0

我正在尝试使用 joblib.Memory 来缓存 DXF 文件（一种 3D CAD 文件格式）的加载，并且发现加载缓存的对象似乎以递归错误结束，这会耗尽 python 堆栈。我创建了一个演示问题的 MWE。

import numpy as np

from ezdxf import recover

import numpy as np

from joblib import Memory
from numpy.lib.function_base import extract
location = './cachedir'
memory = Memory(location, verbose=0)

def _load_dxf_file( filename ):

       print( 'Loading DXF file %s' % filename )

       doc, auditor = recover.readfile(filename)

       print( '    DXF file load complete.' )

       return doc



if __name__ == '__main__':

       load_dxf_file = memory.cache( _load_dxf_file )
       filename = 'cube_mesh_2.dxf'
       doc = load_dxf_file( filename )

示例 DXF 文件（一个简单的立方体）可在此 gist 处获得（太大，无法粘贴到此处）。

https://gist.github.com/jrjbertram/87e31b3bb0ce2d3771dce7f50d2d0fba

该对象第一次加载没有任何问题（数据很好，绘图未包含在 MWE 中。）重新运行脚本以便使用缓存的文件会导致错误，例如：


   1 WARNING:root:[MemorizedFunc(func=<function _load_dxf_file at 0x7fbaee2a38b0>, location=./cachedir/joblib)]: Exception while loading results for      _load_dxf_file('cube_mesh_2.dxf')
   2  Traceback (most recent call last):
   3   File "/Users/bertrjr1/opt/anaconda3/lib/python3.8/site-packages/joblib/memory.py", line 513, in _cached_call
   4     out = self.store_backend.load_item(
   5   File "/Users/bertrjr1/opt/anaconda3/lib/python3.8/site-packages/joblib/_store_backends.py", line 170, in load_item
   6     item = numpy_pickle.load(f)
   7   File "/Users/bertrjr1/opt/anaconda3/lib/python3.8/site-packages/joblib/numpy_pickle.py", line 575, in load
   8     obj = _unpickle(fobj)
   9   File "/Users/bertrjr1/opt/anaconda3/lib/python3.8/site-packages/joblib/numpy_pickle.py", line 504, in _unpickle
  10     obj = unpickler.load()
  11   File "/Users/bertrjr1/opt/anaconda3/lib/python3.8/pickle.py", line 1210, in load
  12     dispatch[key[0]](self)
  13   File "/Users/bertrjr1/opt/anaconda3/lib/python3.8/site-packages/joblib/numpy_pickle.py", line 329, in load_build
  14     Unpickler.load_build(self)
  15   File "/Users/bertrjr1/opt/anaconda3/lib/python3.8/pickle.py", line 1701, in load_build
  16     setstate = getattr(inst, "__setstate__", None)
  17   File "/Users/bertrjr1/opt/anaconda3/lib/python3.8/site-packages/ezdxf/entities/dxfns.py", line 126, in __getattr__
  18     attrib_def: Optional[DXFAttr] = self.dxfattribs.get(key)
  19   File "/Users/bertrjr1/opt/anaconda3/lib/python3.8/site-packages/ezdxf/entities/dxfns.py", line 300, in dxfattribs
  20     return self._entity.DXFATTRIBS
  21   File "/Users/bertrjr1/opt/anaconda3/lib/python3.8/site-packages/ezdxf/entities/dxfns.py", line 126, in __getattr__
  22     attrib_def: Optional[DXFAttr] = self.dxfattribs.get(key)
  23   File "/Users/bertrjr1/opt/anaconda3/lib/python3.8/site-packages/ezdxf/entities/dxfns.py", line 300, in dxfattribs
  24     return self._entity.DXFATTRIBS
  25   File "/Users/bertrjr1/opt/anaconda3/lib/python3.8/site-packages/ezdxf/entities/dxfns.py", line 126, in __getattr__
  26     attrib_def: Optional[DXFAttr] = self.dxfattribs.get(key)
  27   File "/Users/bertrjr1/opt/anaconda3/lib/python3.8/site-packages/ezdxf/entities/dxfns.py", line 300, in dxfattribs
  28     return self._entity.DXFATTRIBS
  29   File "/Users/bertrjr1/opt/anaconda3/lib/python3.8/site-packages/ezdxf/entities/dxfns.py", line 126, in __getattr__
  30     attrib_def: Optional[DXFAttr] = self.dxfattribs.get(key)
  31   File "/Users/bertrjr1/opt/anaconda3/lib/python3.8/site-packages/ezdxf/entities/dxfns.py", line 300, in dxfattribs
  32     return self._entity.DXFATTRIBS
<snip>
1993   File "/Users/bertrjr1/opt/anaconda3/lib/python3.8/site-packages/ezdxf/entities/dxfns.py", line 126, in __getattr__
1994     attrib_def: Optional[DXFAttr] = self.dxfattribs.get(key)
1995 RecursionError: maximum recursion depth exceeded

我怀疑必须有一些关于joblib可以成功缓存和恢复的规则，而ezdxf库必须违反它们？或者这可能是类似于这里描述的泡菜限制：

无法腌制对象：超出最大递归深度

我现在的解决方法是加载DXF文件（真正的需要很长时间并且非常大），执行面部/顶点/三角剖分处理将结果保存到numpy数组，然后使用joblib.Memory缓存numpy数组。但是，就我而言，我需要经常在同一个 DXF 文件上重新运行此三角测量和其他处理，因此能够缓存加载的 DXF 文件本身会很好，这样我就可以避免解析损失（作为 DXF 文件格式是一个非常详细的文本文件，必须全部处理。）

我可能只需要遍历加载的 DXF 文件并以其他格式（列表、字典等？）返回它的修剪版本，然后将其缓存。

欢迎任何建议或想法。

谢谢你，乔希。

4

0 回答 0