我已经查看了Python 文档提供的信息,但我仍然有点困惑。有人可以发布示例代码来编写一个新文件,然后使用 pickle 将字典转储到其中吗?
11 回答
试试这个:
import pickle
a = {'hello': 'world'}
with open('filename.pickle', 'wb') as handle:
pickle.dump(a, handle, protocol=pickle.HIGHEST_PROTOCOL)
with open('filename.pickle', 'rb') as handle:
b = pickle.load(handle)
print(a == b)
上述解决方案没有任何特定于dict
对象的内容。这种相同的方法将适用于许多 Python 对象,包括任意类的实例和任意复杂的数据结构嵌套。例如,用这些行替换第二行:
import datetime
today = datetime.datetime.now()
a = [{'hello': 'world'}, 1, 2.3333, 4, True, "x",
("y", [[["z"], "y"], "x"]), {'today', today}]
也会产生一个结果True
。
有些物体由于其性质而不能腌制。例如,腌制包含打开文件句柄的结构是没有意义的。
import pickle
your_data = {'foo': 'bar'}
# Store data (serialize)
with open('filename.pickle', 'wb') as handle:
pickle.dump(your_data, handle, protocol=pickle.HIGHEST_PROTOCOL)
# Load data (deserialize)
with open('filename.pickle', 'rb') as handle:
unserialized_data = pickle.load(handle)
print(your_data == unserialized_data)
的优点HIGHEST_PROTOCOL
是文件变小。这使得解酸有时更快。
重要提示:pickle 的最大文件大小约为 2GB。
替代方式
import mpu
your_data = {'foo': 'bar'}
mpu.io.write('filename.pickle', data)
unserialized_data = mpu.io.read('filename.pickle')
替代格式
- CSV:超级简单的格式(读写)
- JSON:非常适合编写人类可读的数据;非常常用(读写)
- YAML:YAML 是 JSON 的超集,但更易于阅读(读写,JSON 和 YAML 的比较)
- pickle:一种 Python 序列化格式(读写)
- MessagePack(Python 包):更紧凑的表示(读写)
- HDF5(Python 包):适用于矩阵(读写)
- XML: 也存在 *sigh* (读&写)
对于您的应用程序,以下内容可能很重要:
- 其他编程语言的支持
- 读/写性能
- 紧凑性(文件大小)
另见:数据序列化格式的比较
如果您正在寻找一种制作配置文件的方法,您可能需要阅读我的短文Python 中的配置文件
# Save a dictionary into a pickle file.
import pickle
favorite_color = {"lion": "yellow", "kitty": "red"} # create a dictionary
pickle.dump(favorite_color, open("save.p", "wb")) # save it into a file named save.p
# -------------------------------------------------------------
# Load the dictionary back from the pickle file.
import pickle
favorite_color = pickle.load(open("save.p", "rb"))
# favorite_color is now {"lion": "yellow", "kitty": "red"}
一般来说,腌制 adict
会失败,除非你只有简单的对象,比如字符串和整数。
Python 2.7.9 (default, Dec 11 2014, 01:21:43)
[GCC 4.2.1 Compatible Apple Clang 4.1 ((tags/Apple/clang-421.11.66))] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from numpy import *
>>> type(globals())
<type 'dict'>
>>> import pickle
>>> pik = pickle.dumps(globals())
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 1374, in dumps
Pickler(file, protocol).dump(obj)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 224, in dump
self.save(obj)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 286, in save
f(self, obj) # Call unbound method with explicit self
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 649, in save_dict
self._batch_setitems(obj.iteritems())
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 663, in _batch_setitems
save(v)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 306, in save
rv = reduce(self.proto)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/copy_reg.py", line 70, in _reduce_ex
raise TypeError, "can't pickle %s objects" % base.__name__
TypeError: can't pickle module objects
>>>
即使是非常简单的dict
也经常会失败。这仅取决于内容。
>>> d = {'x': lambda x:x}
>>> pik = pickle.dumps(d)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 1374, in dumps
Pickler(file, protocol).dump(obj)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 224, in dump
self.save(obj)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 286, in save
f(self, obj) # Call unbound method with explicit self
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 649, in save_dict
self._batch_setitems(obj.iteritems())
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 663, in _batch_setitems
save(v)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 286, in save
f(self, obj) # Call unbound method with explicit self
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 748, in save_global
(obj, module, name))
pickle.PicklingError: Can't pickle <function <lambda> at 0x102178668>: it's not found as __main__.<lambda>
但是,如果您使用更好的序列化程序,例如dill
or cloudpickle
,那么大多数字典都可以腌制:
>>> import dill
>>> pik = dill.dumps(d)
或者,如果您想将您dict
的文件保存到文件中...
>>> with open('save.pik', 'w') as f:
... dill.dump(globals(), f)
...
后一个示例与此处发布的任何其他好的答案相同(除了忽略内容的可腌制性之外dict
)。
将 Python 数据(例如字典)转储到 pickle 文件的简单方法。
import pickle
your_dictionary = {}
pickle.dump(your_dictionary, open('pickle_file_name.p', 'wb'))
>>> import pickle
>>> with open("/tmp/picklefile", "wb") as f:
... pickle.dump({}, f)
...
通常最好使用 cPickle 实现
>>> import cPickle as pickle
>>> help(pickle.dump)
Help on built-in function dump in module cPickle:
dump(...)
dump(obj, file, protocol=0) -- Write an object in pickle format to the given file.
See the Pickler docstring for the meaning of optional argument proto.
如果您只想将字典存储在单个文件中,请pickle
像这样使用
import pickle
a = {'hello': 'world'}
with open('filename.pickle', 'wb') as handle:
pickle.dump(a, handle)
with open('filename.pickle', 'rb') as handle:
b = pickle.load(handle)
如果要在多个文件中保存和恢复多个字典以进行缓存和存储更复杂的数据,请使用anycache。它可以完成您需要的所有其他事情pickle
from anycache import anycache
@anycache(cachedir='path/to/files')
def myfunc(hello):
return {'hello', hello}
Anycachemyfunc
根据不同文件的参数存储不同的结果cachedir
并重新加载它们。
有关更多详细信息,请参阅文档。
import pickle
dictobj = {'Jack' : 123, 'John' : 456}
filename = "/foldername/filestore"
fileobj = open(filename, 'wb')
pickle.dump(dictobj, fileobj)
fileobj.close()
仅供参考,熊猫现在有一种保存泡菜的方法。
我觉得更容易。
pd.to_pickle(object_to_save,'/temp/saved_pkl.pickle' )
如果您想在不打开文件的情况下在一行中处理写入或读取:
import joblib
my_dict = {'hello': 'world'}
joblib.dump(my_dict, "my_dict.pickle") # write pickle file
my_dict_loaded = joblib.load("my_dict.pickle") # read pickle file
我发现腌制令人困惑(可能是因为我很厚)。不过,我发现这行得通:
myDictionaryString=str(myDictionary)
然后您可以将其写入文本文件。我放弃了尝试使用 pickle,因为我收到错误告诉我将整数写入 .dat 文件。我很抱歉没有使用泡菜。