0

我正在尝试在环境中从此框架 rllib 运行基线 PPO。对于这个环境，我们用健身房包装器包装了一个 Unity 环境。

venv 'multiagent' 是由一个 github repo 提供的，它需要能够模拟。我认为它以某种方式制造了一些问题。

该代码显示了如何在此环境中运行 PPO：

from ray import tune
from ray.rllib.agents.ppo import PPOTrainer
import environment

env = environment.load_env(name="static_worm", no_graphics=True)
tune.run(PPOTrainer, config={"env": env})

我收到以下错误：

Traceback (most recent call last):
  File "rllib_ppo.py", line 6, in <module>
    tune.run(PPOTrainer, config={"env": env})
  File "C:\Users\cvaes\anaconda3\envs\ml-agents\lib\site-packages\ray\tune\tune.py", line 520, in run
    runner.step()
  File "C:\Users\cvaes\anaconda3\envs\ml-agents\lib\site-packages\ray\tune\trial_runner.py", line 497, in step
    next_trial = self._get_next_trial()  # blocking
  File "C:\Users\cvaes\anaconda3\envs\ml-agents\lib\site-packages\ray\tune\trial_runner.py", line 633, in _get_next_trial
    self._update_trial_queue(blocking=wait_for_trial)
  File "C:\Users\cvaes\anaconda3\envs\ml-agents\lib\site-packages\ray\tune\trial_runner.py", line 1090, in _update_trial_queue
    trial = self._search_alg.next_trial()
  File "C:\Users\cvaes\anaconda3\envs\ml-agents\lib\site-packages\ray\tune\suggest\basic_variant.py", line 307, in next_trial
    trial = next(self._trial_iter)
  File "C:\Users\cvaes\anaconda3\envs\ml-agents\lib\site-packages\ray\tune\suggest\basic_variant.py", line 143, in __next__
    lazy_eval=self.lazy_eval)
  File "C:\Users\cvaes\anaconda3\envs\ml-agents\lib\site-packages\ray\tune\suggest\basic_variant.py", line 36, in __init__
    self.iterable = list(iterable)
  File "C:\Users\cvaes\anaconda3\envs\ml-agents\lib\site-packages\ray\tune\suggest\variant_generator.py", line 47, in generate_variants
    for resolved_vars, spec in _generate_variants(unresolved_spec):
  File "C:\Users\cvaes\anaconda3\envs\ml-agents\lib\site-packages\ray\tune\suggest\variant_generator.py", line 178, in _generate_variants
    spec = copy.deepcopy(spec)
  File "C:\Users\cvaes\anaconda3\envs\ml-agents\lib\copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "C:\Users\cvaes\anaconda3\envs\ml-agents\lib\copy.py", line 240, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "C:\Users\cvaes\anaconda3\envs\ml-agents\lib\copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "C:\Users\cvaes\anaconda3\envs\ml-agents\lib\copy.py", line 240, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "C:\Users\cvaes\anaconda3\envs\ml-agents\lib\copy.py", line 180, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "C:\Users\cvaes\anaconda3\envs\ml-agents\lib\copy.py", line 280, in _reconstruct
    state = deepcopy(state, memo)
  File "C:\Users\cvaes\anaconda3\envs\ml-agents\lib\copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "C:\Users\cvaes\anaconda3\envs\ml-agents\lib\copy.py", line 240, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "C:\Users\cvaes\anaconda3\envs\ml-agents\lib\copy.py", line 180, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "C:\Users\cvaes\anaconda3\envs\ml-agents\lib\copy.py", line 280, in _reconstruct
    state = deepcopy(state, memo)
  File "C:\Users\cvaes\anaconda3\envs\ml-agents\lib\copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "C:\Users\cvaes\anaconda3\envs\ml-agents\lib\copy.py", line 240, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "C:\Users\cvaes\anaconda3\envs\ml-agents\lib\copy.py", line 180, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "C:\Users\cvaes\anaconda3\envs\ml-agents\lib\copy.py", line 280, in _reconstru
ct
    state = deepcopy(state, memo)
  File "C:\Users\cvaes\anaconda3\envs\ml-agents\lib\copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "C:\Users\cvaes\anaconda3\envs\ml-agents\lib\copy.py", line 240, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "C:\Users\cvaes\anaconda3\envs\ml-agents\lib\copy.py", line 169, in deepcopy
    rv = reductor(4)
TypeError: can't pickle _thread.lock objects

(ml-agents) C:\Users\cvaes\PycharmProjects\autonomous-systems>

我知道它有事情要做，这不能被多处理/序列化。但是我该如何解决呢？

4

0 回答 0