2

我正在使用 scikit-learn 解决具有大量标签的多标签分类问题。我遵循了该项目的核心开发人员之一（larsmans）的想法。它给了我一个运行时警告，然后最终出现内存错误。

>>> clf = Pipeline([('chi2', SelectKBest(chi2, k=1000)),('svm', LinearSVC())])
>>> 
>>> multiclf = OneVsRestClassifier(clf, n_jobs=-1)
>>> 
>>> multiclf.fit(Xtr, ytr)
/home/rsivapr/scikit-learn/sklearn/feature_selection/univariate_selection.py:157: 
RuntimeWarning: invalid value encountered in divide
  chisq /= f_exp
/home/rsivapr/scikit-learn/sklearn/feature_selection/univariate_selection.py:157: 
RuntimeWarning: invalid value encountered in divide
  chisq /= f_exp
/home/rsivapr/scikit-learn/sklearn/feature_selection/univariate_selection.py:157:             
RuntimeWarning: invalid value encountered in divide
  chisq /= f_exp

然后在多次这样的警告之后，它失败了：下面的错误转储

Process PoolWorker-21:
Traceback (most recent call last):
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker
    task = get()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get
    return recv()
MemoryError
Exception in thread Thread-1:
Traceback (most recent call last):
  File "/home/rsivapr/anaconda/lib/python2.7/threading.py", line 808, in __bootstrap_inner
    self.run()
  File "/home/rsivapr/anaconda/lib/python2.7/threading.py", line 761, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 325, in _handle_workers
    pool._maintain_pool()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 229, in _maintain_pool
    self._repopulate_pool()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 222, in _repopulate_pool
    w.start()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 130, in start
    self._popen = Popen(self)
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/forking.py", line 121, in __init__
    self.pid = os.fork()
OSError: [Errno 12] Cannot allocate memory

/home/rsivapr/scikit-learn/sklearn/feature_selection/univariate_selection.py:157: RuntimeWarning: invalid value encountered in divide
  chisq /= f_exp
Process PoolWorker-22:
Traceback (most recent call last):
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker
    task = get()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get
    return recv()
MemoryError
Process PoolWorker-23:
Process PoolWorker-30:
/home/rsivapr/scikit-learn/sklearn/feature_selection/univariate_selection.py:157: RuntimeWarning: invalid value encountered in divide
  chisq /= f_exp
Process PoolWorker-31:
Traceback (most recent call last):
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
Traceback (most recent call last):
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
Process PoolWorker-28:
Traceback (most recent call last):
Traceback (most recent call last):
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
Process PoolWorker-32:
Process PoolWorker-29:
Traceback (most recent call last):
Traceback (most recent call last):
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
Process PoolWorker-27:
Process PoolWorker-26:
Traceback (most recent call last):
Process PoolWorker-25:
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
Traceback (most recent call last):
Traceback (most recent call last):
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run
    self.run()
    self.run()
    self.run()
    self.run()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker
    self._target(*self._args, **self._kwargs)
    self._target(*self._args, **self._kwargs)
    self._target(*self._args, **self._kwargs)
    self._target(*self._args, **self._kwargs)
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker
    self.run()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker
    self.run()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker
    self.run()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker
    task = get()
    task = get()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get
    task = get()
    task = get()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get
    task = get()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get
    task = get()
    task = get()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get
    task = get()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get
    return recv()
    return recv()
    return recv()
MemoryError
MemoryError
MemoryError
    return recv()
MemoryError
    return recv()
    return recv()
MemoryError
MemoryError
    return recv()
MemoryError
    return recv()
MemoryError
    self.run()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker
Process PoolWorker-24:
Traceback (most recent call last):
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    task = get()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get
    return recv()
UnpicklingError: invalid load key, 'k'.
    self.run()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker
    task = get()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get
    return recv()
MemoryError
Process PoolWorker-1:
Traceback (most recent call last):
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker
    task = get()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get
    return recv()
MemoryError
Process PoolWorker-2:
Traceback (most recent call last):
Process PoolWorker-3:
Traceback (most recent call last):
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
Process PoolWorker-4:
Traceback (most recent call last):
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
Process PoolWorker-6:
Traceback (most recent call last):
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker
    task = get()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get
    return recv()
MemoryError
    self.run()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker
Process PoolWorker-5:
    task = get()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get
Traceback (most recent call last):
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
    self.run()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker
    task = get()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get
    return recv()
MemoryError
    task = get()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get
    return recv()
MemoryError
    return recv()
MemoryError
    self.run()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker
    task = get()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get
    return recv()
MemoryError
Process PoolWorker-7:
Traceback (most recent call last):
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker
    task = get()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get
    return recv()
UnpicklingError: unpickling stack underflow
Process PoolWorker-8:
Traceback (most recent call last):
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker
    task = get()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get
    return recv()
UnpicklingError: invalid load key, ''.
Process PoolWorker-10:
Traceback (most recent call last):
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker
    task = get()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get
    return recv()
UnpicklingError: invalid load key, '�'.
Process PoolWorker-21:
Traceback (most recent call last):
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker
    task = get()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get
    return recv()
MemoryError
Exception in thread Thread-1:
Traceback (most recent call last):
  File "/home/rsivapr/anaconda/lib/python2.7/threading.py", line 808, in __bootstrap_inner
    self.run()
  File "/home/rsivapr/anaconda/lib/python2.7/threading.py", line 761, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 325, in _handle_workers
    pool._maintain_pool()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 229, in _maintain_pool
    self._repopulate_pool()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 222, in _repopulate_pool
    w.start()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 130, in start
    self._popen = Popen(self)
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/forking.py", line 121, in __init__
    self.pid = os.fork()
OSError: [Errno 12] Cannot allocate memory

/home/rsivapr/scikit-learn/sklearn/feature_selection/univariate_selection.py:157: RuntimeWarning: invalid value encountered in divide
  chisq /= f_exp
Process PoolWorker-22:
Traceback (most recent call last):
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker
    task = get()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get
    return recv()
MemoryError
Process PoolWorker-23:
Process PoolWorker-30:
/home/rsivapr/scikit-learn/sklearn/feature_selection/univariate_selection.py:157: RuntimeWarning: invalid value encountered in divide
  chisq /= f_exp
Process PoolWorker-31:
Traceback (most recent call last):
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
Traceback (most recent call last):
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
Process PoolWorker-28:
Traceback (most recent call last):
Traceback (most recent call last):
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
Process PoolWorker-32:
Process PoolWorker-29:
Traceback (most recent call last):
Traceback (most recent call last):
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
Process PoolWorker-27:
Process PoolWorker-26:
Traceback (most recent call last):
Process PoolWorker-25:
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
Traceback (most recent call last):
Traceback (most recent call last):
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run
    self.run()
    self.run()
    self.run()
    self.run()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker
    self._target(*self._args, **self._kwargs)
    self._target(*self._args, **self._kwargs)
    self._target(*self._args, **self._kwargs)
    self._target(*self._args, **self._kwargs)
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker
    self.run()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker
    self.run()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker
    self.run()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker
    task = get()
    task = get()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get
    task = get()
    task = get()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get
    task = get()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get
    task = get()
    task = get()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get
    task = get()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get
    return recv()
    return recv()
    return recv()
MemoryError
MemoryError
MemoryError
    return recv()
MemoryError
    return recv()
    return recv()
MemoryError
MemoryError
    return recv()
MemoryError
    return recv()
MemoryError
    self.run()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker
Process PoolWorker-24:
Traceback (most recent call last):
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    task = get()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get
    return recv()
UnpicklingError: invalid load key, 'k'.
    self.run()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker
    task = get()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get
    return recv()
MemoryError
Process PoolWorker-1:
Traceback (most recent call last):
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker
    task = get()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get
    return recv()
MemoryError
Process PoolWorker-2:
Traceback (most recent call last):
Process PoolWorker-3:
Traceback (most recent call last):
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
Process PoolWorker-4:
Traceback (most recent call last):
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
Process PoolWorker-6:
Traceback (most recent call last):
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker
    task = get()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get
    return recv()
MemoryError
    self.run()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker
Process PoolWorker-5:
    task = get()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get
Traceback (most recent call last):
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
    self.run()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker
    task = get()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get
    return recv()
MemoryError
    task = get()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get
    return recv()
MemoryError
    return recv()
MemoryError
    self.run()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker
    task = get()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get
    return recv()
MemoryError
Process PoolWorker-7:
Traceback (most recent call last):
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker
    task = get()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get
    return recv()
UnpicklingError: unpickling stack underflow
Process PoolWorker-8:
Traceback (most recent call last):
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker
    task = get()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get
    return recv()
UnpicklingError: invalid load key, ''.
Process PoolWorker-10:
Traceback (most recent call last):
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker
    task = get()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get
    return recv()
UnpicklingError: invalid load key, '�'.
4

1 回答 1

0

看到所有的 MemoryErrors，你可能已经用完了内存。尝试将 n_jobs=-1 替换为 n_jobs=1。这控制了产生的进程数量（以及因此使用的内存量）。

如果这行得通，只要您没有耗尽内存，您就可以增加 n_jobs，即 n_jobs=2、n_jobs=3 等等。

于 2014-09-30T09:27:17.707 回答