当我尝试添加自己的网络时，我总是发现 ValueError :'Observation outside expected value range'。同样在教程08中，当我用其他网络（例如HighwayNetwork或其他）替换RingNetwork时，总是出现ValueError：'Observation outside expected value range'。我不知道如何解决它。
Traceback (most recent call last):
File "/home/vcdc/anaconda3/envs/flow/lib/python3.7/site-packages/ray/tune/trial_runner.py", line 426, in _process_trial
result = self.trial_executor.fetch_result(trial)
File "/home/vcdc/anaconda3/envs/flow/lib/python3.7/site-packages/ray/tune/ray_trial_executor.py", line 378, in fetch_result
result = ray.get(trial_future[0], DEFAULT_GET_TIMEOUT)
File "/home/vcdc/anaconda3/envs/flow/lib/python3.7/site-packages/ray/worker.py", line 1457, in get
raise value.as_instanceof_cause()
ray.exceptions.RayTaskError(ValueError): ray::PPO.train() (pid=5069, ip=192.168.6.112)
File "python/ray/_raylet.pyx", line 636, in ray._raylet.execute_task
File "python/ray/_raylet.pyx", line 619, in ray._raylet.execute_task.function_executor
File "/home/vcdc/anaconda3/envs/flow/lib/python3.7/site-packages/ray/rllib/agents/trainer.py", line 444, in train
raise e
File "/home/vcdc/anaconda3/envs/flow/lib/python3.7/site-packages/ray/rllib/agents/trainer.py", line 433, in train
result = Trainable.train(self)
File "/home/vcdc/anaconda3/envs/flow/lib/python3.7/site-packages/ray/tune/trainable.py", line 176, in train
result = self._train()
File "/home/vcdc/anaconda3/envs/flow/lib/python3.7/site-packages/ray/rllib/agents/trainer_template.py", line 129, in _train
fetches = self.optimizer.step()
File "/home/vcdc/anaconda3/envs/flow/lib/python3.7/site-packages/ray/rllib/optimizers/multi_gpu_optimizer.py", line 140, in step
self.num_envs_per_worker, self.train_batch_size)
File "/home/vcdc/anaconda3/envs/flow/lib/python3.7/site-packages/ray/rllib/optimizers/rollout.py", line 29, in collect_samples
next_sample = ray_get_and_free(fut_sample)
File "/home/vcdc/anaconda3/envs/flow/lib/python3.7/site-packages/ray/rllib/utils/memory.py", line 33, in ray_get_and_free
result = ray.get(object_ids)
ray.exceptions.RayTaskError(ValueError): ray::RolloutWorker.sample() (pid=5065, ip=192.168.6.112)
File "python/ray/_raylet.pyx", line 636, in ray._raylet.execute_task
File "python/ray/_raylet.pyx", line 619, in ray._raylet.execute_task.function_executor
File "/home/vcdc/anaconda3/envs/flow/lib/python3.7/site-packages/ray/rllib/evaluation/rollout_worker.py", line 471, in sample
batches = [self.input_reader.next()]
File "/home/vcdc/anaconda3/envs/flow/lib/python3.7/site-packages/ray/rllib/evaluation/sampler.py", line 56, in next
batches = [self.get_data()]
File "/home/vcdc/anaconda3/envs/flow/lib/python3.7/site-packages/ray/rllib/evaluation/sampler.py", line 99, in get_data
item = next(self.rollout_provider)
File "/home/vcdc/anaconda3/envs/flow/lib/python3.7/site-packages/ray/rllib/evaluation/sampler.py", line 319, in _env_runner
soft_horizon, no_done_at_end)
File "/home/vcdc/anaconda3/envs/flow/lib/python3.7/site-packages/ray/rllib/evaluation/sampler.py", line 407, in _process_observations
policy_id).transform(raw_obs)
File "/home/vcdc/anaconda3/envs/flow/lib/python3.7/site-packages/ray/rllib/models/preprocessors.py", line 166, in transform
self.check_shape(observation)
File "/home/vcdc/anaconda3/envs/flow/lib/python3.7/site-packages/ray/rllib/models/preprocessors.py", line 65, in check_shape
self._obs_space, observation)
ValueError: ('Observation outside expected value range', Box(44,), array([], dtype=float64))
2021-07-11 16:50:33,037 INFO trial_runner.py:530 -- Trial PPO_AccelEnv-v0_078139c6: Attempting to recover trial state from last checkpoint.
2021-07-11 16:50:33,040 INFO ray_trial_executor.py:121 -- Trial PPO_AccelEnv-v0_078139c6: Setting up new remote runner