我想绘制误差与 keras 中的时代数的关系。为此,我在这里找到了一个非常有用的帖子。它在本地就像一个魅力,但如果我想在某个远程 GPU/集群中运行我的脚本,那么它就没有那么有用了,因为我需要向自己发送数据才能真正制作绘图。有没有办法运行类似的东西,但将错误存储在一些文件/csv/json 中,然后能够使用 rysnc/scp/dropbox 等标准工具将其发送给自己?
现在我唯一的想法是腌制文件,然后将其发送给自己并取消腌制,但它根本不起作用。我得到错误:
Traceback (most recent call last):
File "store_data.py", line 71, in <module>
pickle.dump( history, open( "history.p", "wb" ) )
_pickle.PicklingError: Can't pickle <class 'module'>: attribute lookup module on builtins failed
有没有使用 Keras 和 python 的不同方法?我只需要能够存储这种类型的数据,以便我可以以某种方式将其发送到我的本地计算机。
目前我完全可重现的错误可以按如下方式完成:
存储数据.py
# Visualize training history
from keras.models import Sequential
from keras.layers import Dense, Dropout, Activation, Flatten
from keras.datasets import cifar10
from keras.utils import np_utils
import numpy
import pickle
print('start visualization example')
#params
nb_classes = 10
data_augmentation = False
units_single_layer = 10
actication_func = 'relu'
actication_func = 'sigmoid'
nb_epoch = 3
batch_size = 64
#optimizer = 'adam'
optimizer = 'rmsprop'
# input image dimensions
img_rows, img_cols = 32, 32
# The CIFAR10 images are RGB.
img_channels = 3
# The data, shuffled and split between train and test sets:
(X_train, y_train), (X_test, y_test) = cifar10.load_data()
X_train = X_train.reshape((X_train.shape[0],32*32*3))
X_test = X_test.reshape((X_test.shape[0],32*32*3))
print('X_train shape:', X_train.shape)
print(X_train.shape[0], 'train samples')
print(X_test.shape[0], 'test samples')
# Convert class vectors to binary class matrices.
Y_train = np_utils.to_categorical(y_train, nb_classes)
Y_test = np_utils.to_categorical(y_test, nb_classes)
# create model
print('\n ---- Singled Layer Model ----')
print('units_single_layer: ', units_single_layer)
print('actication_func: ', actication_func)
model = Sequential()
model.add(Dense(units_single_layer, input_shape=(32*32*3,)))
model.add(Activation(actication_func))
model.add(Dense(nb_classes))
model.add(Activation('softmax'))
# Compile model
print('\n ---- Optimizer ----')
print('optimizer: ', optimizer)
print('batch_size: ', optimizer)
print('')
model.compile(loss='categorical_crossentropy',
optimizer=optimizer,
metrics=['accuracy'])
# Fit the model
#history = model.fit(X, Y, validation_split=0.33, nb_epoch=150, batch_size=10, verbose=0)
history = model.fit(X_train, Y_train,
batch_size=batch_size,
nb_epoch=nb_epoch,
validation_data=(X_test, Y_test),
shuffle=True)
pickle.dump( history, open( "history.p", "wb" ) )
并绘制它:
绘图.py
# Visualize training history
from keras.models import Sequential
from keras.layers import Dense, Dropout, Activation, Flatten
from keras.datasets import cifar10
from keras.utils import np_utils
import matplotlib.pyplot as plt
import numpy
import pickle
history = pickle.load( open( "history.p", "rb" ) )
# list all data in history
print(history.history.keys())
# summarize history for accuracy
plt.plot(history.history['acc'])
plt.plot(history.history['val_acc'])
plt.title('model accuracy')
plt.ylabel('accuracy')
plt.xlabel('epoch')
plt.legend(['train', 'test'], loc='upper left')
plt.show()
# summarize history for loss
plt.plot(history.history['loss'])
plt.plot(history.history['val_loss'])
plt.title('model loss')
plt.ylabel('loss')
plt.xlabel('epoch')
plt.legend(['train', 'test'], loc='upper left')
plt.show()
此外,如果您由于框架构建而对 matplot lib 有问题,只需使用已经为您解决该问题的虚拟环境:
python3 -m venv <my-virtual-env-name>
http://matplotlib.org/faq/osx_framework.html#virtualenv
http://matplotlib.org/faq/osx_framework.html
http://matplotlib.org/faq/virtualenv_faq.html
使用python3的-m venv
virtualenv实现,该实现使python成为一个框架。