4

我有 5 个 numpy 数组a,b,c,d，e全部定义为：

array([1, 2, 3, 4, 5, 6, 7, 8, 9])

我像这样保存这些数组：

np.savez_compressed('tmp/test',a=a,b=b,c=c,d=d,e=e)

这会导致test.npz创建一个文件。

但是，我在尝试加载数据时遇到问题（按照此处的示例）：

>>> f=np.load('tmp/test.npz')
>>> f.files
['a', 'c', 'b', 'e', 'd']
>>> f['a']

给出一大串以以下结尾的错误：

File "C:\Python27\lib\compiler\transformer.py", line 754, in atom_lbrace
return self.com_dictorsetmaker(nodelist[1])
File "C:\Python27\lib\compiler\transformer.py", line 1214, in com_dictorsetmaker
assert nodelist[0] == symbol.dictorsetmaker
AssertionError

我已经考虑pickle改用。但是，这会导致文件大小是 .npz 文件的四倍，所以我想使用savezor savez_compressed。

有谁知道我做错了什么，或者对替代方法的建议？

这是一个会产生错误的脚本：

def saver():
    import numpy as np
    a= np.arange(1,10)
    b=a
    c=a
    d=a
    e=a
    np.savez_compressed('tmp/test',a=a,b=b,c=c,d=d,e=e)
    f=np.load('tmp/test.npz')
    print f.files
    print f['a']

这是完整的回溯：

Traceback (most recent call last):
  File "<pyshell#16>", line 1, in <module>
    saver.saver()
  File "C:\Python27\saver.py", line 14, in saver
    print f['a']
  File "C:\Python27\lib\site-packages\numpy\lib\npyio.py", line 241, in __getitem__
    return format.read_array(value)
  File "C:\Python27\lib\site-packages\numpy\lib\format.py", line 440, in read_array
    shape, fortran_order, dtype = read_array_header_1_0(fp)
  File "C:\Python27\lib\site-packages\numpy\lib\format.py", line 336, in read_array_header_1_0
    d = safe_eval(header)
  File "C:\Python27\lib\site-packages\numpy\lib\utils.py", line 1156, in safe_eval
    ast = compiler.parse(source, mode="eval")
  File "C:\Python27\lib\compiler\transformer.py", line 53, in parse
    return Transformer().parseexpr(buf)
  File "C:\Python27\lib\compiler\transformer.py", line 132, in parseexpr
    return self.transform(parser.expr(text))
  File "C:\Python27\lib\compiler\transformer.py", line 124, in transform
    return self.compile_node(tree)
  File "C:\Python27\lib\compiler\transformer.py", line 159, in compile_node
    return self.eval_input(node[1:])
  File "C:\Python27\lib\compiler\transformer.py", line 194, in eval_input
    return Expression(self.com_node(nodelist[0]))
  File "C:\Python27\lib\compiler\transformer.py", line 805, in com_node
    return self._dispatch[node[0]](node[1:])
  File "C:\Python27\lib\compiler\transformer.py", line 578, in testlist
    return self.com_binary(Tuple, nodelist)
  File "C:\Python27\lib\compiler\transformer.py", line 1082, in com_binary
    return self.lookup_node(n)(n[1:])
  File "C:\Python27\lib\compiler\transformer.py", line 596, in test
    then = self.com_node(nodelist[0])
  File "C:\Python27\lib\compiler\transformer.py", line 805, in com_node
    return self._dispatch[node[0]](node[1:])
  File "C:\Python27\lib\compiler\transformer.py", line 610, in or_test
    return self.com_binary(Or, nodelist)
  File "C:\Python27\lib\compiler\transformer.py", line 1082, in com_binary
    return self.lookup_node(n)(n[1:])
  File "C:\Python27\lib\compiler\transformer.py", line 615, in and_test
    return self.com_binary(And, nodelist)
  File "C:\Python27\lib\compiler\transformer.py", line 1082, in com_binary
    return self.lookup_node(n)(n[1:])
  File "C:\Python27\lib\compiler\transformer.py", line 619, in not_test
    result = self.com_node(nodelist[-1])
  File "C:\Python27\lib\compiler\transformer.py", line 805, in com_node
    return self._dispatch[node[0]](node[1:])
  File "C:\Python27\lib\compiler\transformer.py", line 626, in comparison
    node = self.com_node(nodelist[0])
  File "C:\Python27\lib\compiler\transformer.py", line 805, in com_node
    return self._dispatch[node[0]](node[1:])
  File "C:\Python27\lib\compiler\transformer.py", line 659, in expr
    return self.com_binary(Bitor, nodelist)
  File "C:\Python27\lib\compiler\transformer.py", line 1082, in com_binary
    return self.lookup_node(n)(n[1:])
  File "C:\Python27\lib\compiler\transformer.py", line 663, in xor_expr
    return self.com_binary(Bitxor, nodelist)
  File "C:\Python27\lib\compiler\transformer.py", line 1082, in com_binary
    return self.lookup_node(n)(n[1:])
  File "C:\Python27\lib\compiler\transformer.py", line 667, in and_expr
    return self.com_binary(Bitand, nodelist)
  File "C:\Python27\lib\compiler\transformer.py", line 1082, in com_binary
    return self.lookup_node(n)(n[1:])
  File "C:\Python27\lib\compiler\transformer.py", line 671, in shift_expr
    node = self.com_node(nodelist[0])
  File "C:\Python27\lib\compiler\transformer.py", line 805, in com_node
    return self._dispatch[node[0]](node[1:])
  File "C:\Python27\lib\compiler\transformer.py", line 683, in arith_expr
    node = self.com_node(nodelist[0])
  File "C:\Python27\lib\compiler\transformer.py", line 805, in com_node
    return self._dispatch[node[0]](node[1:])
  File "C:\Python27\lib\compiler\transformer.py", line 695, in term
    node = self.com_node(nodelist[0])
  File "C:\Python27\lib\compiler\transformer.py", line 805, in com_node
    return self._dispatch[node[0]](node[1:])
  File "C:\Python27\lib\compiler\transformer.py", line 715, in factor
    node = self.lookup_node(nodelist[-1])(nodelist[-1][1:])
  File "C:\Python27\lib\compiler\transformer.py", line 727, in power
    node = self.com_node(nodelist[0])
  File "C:\Python27\lib\compiler\transformer.py", line 805, in com_node
    return self._dispatch[node[0]](node[1:])
  File "C:\Python27\lib\compiler\transformer.py", line 739, in atom
    return self._atom_dispatch[nodelist[0][0]](nodelist)
  File "C:\Python27\lib\compiler\transformer.py", line 754, in atom_lbrace
    return self.com_dictorsetmaker(nodelist[1])
  File "C:\Python27\lib\compiler\transformer.py", line 1214, in com_dictorsetmaker
    assert nodelist[0] == symbol.dictorsetmaker
AssertionError
4

2 回答 2

1

无法在 Linux 或 Mac 上重现您的问题（Python 2.7，numpy 1.6.1/1.7.1）

但是，我注意到您使用相对路径来保存文件tmp/test.npz。这是故意的吗？在我的回忆中，最近版本的 Windows 对尝试在某些目录（如“/Program Files/”）中创建的新文件应用程序有一些特殊处理——它会将它们移开，但在某些情况下仍然告诉应用程序它们在那里。这里似乎不太可能，但是您可以尝试保存文件的绝对路径吗？

顺便说一句：作为 ZIP 存档（savez savez_compressed创建）的替代方案，您可以尝试使用“LZMAFile”作为文件对象的泡菜。它提供了非常好的压缩率（但它可能很慢并且在压缩/保存文件时需要更多的内存和时间）；

它用作任何其他文件对象包装器，类似这样（加载腌制数据）：

from lzma import LZMAFile
import cPickle as pickle

if fileName.endswith('.xz'):
    dataFile = LZMAFile(fileName,'r')
else:
    dataFile = file(fileName, 'ro')     
data = pickle.load(dataFile)
于 2013-06-25T20:54:30.730 回答
0

在 MAC OS 和 Windows 上使用 numpy 1.7.1/1.8.0 和 python 2.7.6 时，我确实遇到了同样的问题（AssertionError）。但是在我使用 python 2.7.5 切换到 linux 后，问题就自动修复了。然后我在 MACOS 和 Windows 上重新安装 python 2.7.5，所有问题都消失了。基本上问题在于python而不是numpy，因为编译器正在发送警报。所以版本可能很重要。

但是尽管 npy 是 numpy 的可序列化类型，但我认为即使 savez_compressed 用于大型矩阵，文件也不够小。

希望你的问题和我的一样。

于 2014-02-09T07:09:35.413 回答