0

作为前言，我正在使用蒙特卡洛树搜索来运行基于模型的强化学习任务。基本上我有一个代理在一个离散的环境中觅食，代理可以看到它周围的一些空间（为了简单起见，我假设完全了解它的观察空间，所以观察与状态相同）。代理有一个由 MLP 表示的世界的内部转换模型（我正在使用 tf.keras）。基本上，对于树中的每一步，我使用模型来预测给定动作的下一个状态，并让代理根据预测的状态变化计算它将获得多少奖励。从那里开始是熟悉的 MCTS 算法，包括选择、扩展、推出和反向传播。

我想并行运行多次试验以节省时间。我一开始尝试使用香草多处理，但它使用了泡菜，它不能序列化很多东西（包括我的代码）。因此，我使用 pathos.multiprocessing 显然解决了这个问题，因为它使用了莳萝。但是，当我运行我的代码时，而不是伴随香草多处理的“无法腌制”错误，我得到了这个（对不起，跟踪的长度很长，我会删掉一些东西，但我不确定什么是相关的或不是，所以也许只是滚动到底部）：


Traceback (most recent call last):
  File "/Users/~/anaconda3/envs/discrete_foraging/lib/python3.6/site-packages/multiprocess/pool.py", line 424, in _handle_tasks
    put(task)
  File "/Users/~/anaconda3/envs/discrete_foraging/lib/python3.6/site-packages/multiprocess/connection.py", line 209, in send
    self._send_bytes(_ForkingPickler.dumps(obj))
  File "/Users/~/anaconda3/envs/discrete_foraging/lib/python3.6/site-packages/multiprocess/reduction.py", line 54, in dumps
    cls(buf, protocol, *args, **kwds).dump(obj)
  File "/Users/~/anaconda3/envs/discrete_foraging/lib/python3.6/site-packages/dill/_dill.py", line 446, in dump
    StockPickler.dump(self, obj)
  File "/Users/~/anaconda3/envs/discrete_foraging/lib/python3.6/pickle.py", line 409, in dump
    self.save(obj)
  File "/Users/~/anaconda3/envs/discrete_foraging/lib/python3.6/pickle.py", line 476, in save
    f(self, obj) # Call unbound method with explicit self
  File "/Users/~/anaconda3/envs/discrete_foraging/lib/python3.6/pickle.py", line 751, in save_tuple
    save(element)
  File "/Users/~/anaconda3/envs/discrete_foraging/lib/python3.6/pickle.py", line 476, in save
    f(self, obj) # Call unbound method with explicit self
  File "/Users/~/anaconda3/envs/discrete_foraging/lib/python3.6/pickle.py", line 736, in save_tuple
    save(element)
  File "/Users/~/anaconda3/envs/discrete_foraging/lib/python3.6/pickle.py", line 476, in save
    f(self, obj) # Call unbound method with explicit self
  File "/Users/~/anaconda3/envs/discrete_foraging/lib/python3.6/pickle.py", line 736, in save_tuple
    save(element)
  File "/Users/~/anaconda3/envs/discrete_foraging/lib/python3.6/pickle.py", line 476, in save
    f(self, obj) # Call unbound method with explicit self
  File "/Users/~/anaconda3/envs/discrete_foraging/lib/python3.6/pickle.py", line 736, in save_tuple
    save(element)
  File "/Users/~/anaconda3/envs/discrete_foraging/lib/python3.6/pickle.py", line 476, in save
    f(self, obj) # Call unbound method with explicit self
  File "/Users/~/anaconda3/envs/discrete_foraging/lib/python3.6/pickle.py", line 736, in save_tuple
    save(element)
  File "/Users/~/anaconda3/envs/discrete_foraging/lib/python3.6/pickle.py", line 476, in save
    f(self, obj) # Call unbound method with explicit self
  File "/Users/~/anaconda3/envs/discrete_foraging/lib/python3.6/pickle.py", line 751, in save_tuple
    save(element)
  File "/Users/~/anaconda3/envs/discrete_foraging/lib/python3.6/pickle.py", line 521, in save
    self.save_reduce(obj=obj, *rv)
  File "/Users/~/anaconda3/envs/discrete_foraging/lib/python3.6/pickle.py", line 634, in save_reduce
    save(state)
  File "/Users/~/anaconda3/envs/discrete_foraging/lib/python3.6/pickle.py", line 476, in save
    f(self, obj) # Call unbound method with explicit self
  File "/Users/~/anaconda3/envs/discrete_foraging/lib/python3.6/site-packages/dill/_dill.py", line 933, in save_module_dict
    StockPickler.save_dict(pickler, obj)
  File "/Users/~/anaconda3/envs/discrete_foraging/lib/python3.6/pickle.py", line 821, in save_dict
    self._batch_setitems(obj.items())
  File "/Users/~/anaconda3/envs/discrete_foraging/lib/python3.6/pickle.py", line 847, in _batch_setitems
    save(v)
  File "/Users/~/anaconda3/envs/discrete_foraging/lib/python3.6/pickle.py", line 476, in save
    f(self, obj) # Call unbound method with explicit self
  File "/Users/~/anaconda3/envs/discrete_foraging/lib/python3.6/site-packages/dill/_dill.py", line 1119, in save_instancemethod0
    pickler.save_reduce(MethodType, (obj.__func__, obj.__self__), obj=obj)
  File "/Users/~/anaconda3/envs/discrete_foraging/lib/python3.6/pickle.py", line 610, in save_reduce
    save(args)
  File "/Users/~/anaconda3/envs/discrete_foraging/lib/python3.6/pickle.py", line 476, in save
    f(self, obj) # Call unbound method with explicit self
  File "/Users/~/anaconda3/envs/discrete_foraging/lib/python3.6/pickle.py", line 736, in save_tuple
    save(element)
  File "/Users/~/anaconda3/envs/discrete_foraging/lib/python3.6/pickle.py", line 476, in save
    f(self, obj) # Call unbound method with explicit self
  File "/Users/~/anaconda3/envs/discrete_foraging/lib/python3.6/site-packages/dill/_dill.py", line 1408, in save_function
    if not _locate_function(obj): #, pickler._session):
  File "/Users/~/anaconda3/envs/discrete_foraging/lib/python3.6/site-packages/dill/_dill.py", line 856, in _locate_function
    found = _import_module(obj.__module__ + '.' + obj.__name__, safe=True)
  File "/Users/~/anaconda3/envs/discrete_foraging/lib/python3.6/site-packages/dill/_dill.py", line 847, in _import_module
    return getattr(__import__(module, None, None, [obj]), obj)
ValueError: Empty module name

我认为这可能与为什么它不能在香草多处理下腌制有关，但我不确定。这是我尝试运行的代码的相关部分：



def trial_runner(args):
# the function I'm trying to parallelize

if __name__ == '__main__':

    # generate the environment
    env = MultiAgentEnv()

    # acquire some sample data

    # create a list of tuple of arguments of length equal to the number of trials 
    input_data = [
        (env, env.world, env.world.agent, env.world.agent.input_elev_memory, env.world.agent.input_food_memory,
         env.world.agent.input_energy_memory, env.world.agent.input_action_memory,
         env.world.agent.output_elev_memory,
         env.world.agent.output_food_memory, env.world.agent.output_history) for _ in range(ep.num_trials)]

    # run the pool 
    results = ProcessingPool().map(trial_runner, input_data)

trial_runner 是我试图并行化的函数，它运行前言中描述的算法。我有两个想法是：

  1. 可能是因为它调用了许多类和类函数，所以并行化太复杂了。
  2. 它们都导入同一个文件（其中包含一些变量值），并且不同的进程可能会变得混乱。

任何帮助将不胜感激。

4

0 回答 0