1

我想绘制误差与 keras 中的时代数的关系。为此,我在这里找到了一个非常有用的帖子。它在本地就像一个魅力,但如果我想在某个远程 GPU/集群中运行我的脚本,那么它就没有那么有用了,因为我需要向自己发送数据才能真正制作绘图。有没有办法运行类似的东西,但将错误存储在一些文件/csv/json 中,然后能够使用 rysnc/scp/dropbox 等标准工具将其发送给自己?

现在我唯一的想法是腌制文件,然后将其发送给自己并取消腌制,但它根本不起作用。我得到错误:

Traceback (most recent call last):
  File "store_data.py", line 71, in <module>
    pickle.dump( history, open( "history.p", "wb" ) )
_pickle.PicklingError: Can't pickle <class 'module'>: attribute lookup module on builtins failed

有没有使用 Keras 和 python 的不同方法?我只需要能够存储这种类型的数据,以便我可以以某种方式将其发送到我的本地计算机。


目前我完全可重现的错误可以按如下方式完成:

存储数据.py

# Visualize training history
from keras.models import Sequential
from keras.layers import Dense, Dropout, Activation, Flatten
from keras.datasets import cifar10
from keras.utils import np_utils

import numpy
import pickle

print('start visualization example')

#params
nb_classes = 10

data_augmentation = False

units_single_layer = 10
actication_func = 'relu'
actication_func = 'sigmoid'

nb_epoch = 3
batch_size = 64
#optimizer = 'adam'
optimizer = 'rmsprop'

# input image dimensions
img_rows, img_cols = 32, 32
# The CIFAR10 images are RGB.
img_channels = 3

# The data, shuffled and split between train and test sets:
(X_train, y_train), (X_test, y_test) = cifar10.load_data()
X_train = X_train.reshape((X_train.shape[0],32*32*3))
X_test = X_test.reshape((X_test.shape[0],32*32*3))
print('X_train shape:', X_train.shape)
print(X_train.shape[0], 'train samples')
print(X_test.shape[0], 'test samples')

# Convert class vectors to binary class matrices.
Y_train = np_utils.to_categorical(y_train, nb_classes)
Y_test = np_utils.to_categorical(y_test, nb_classes)

# create model
print('\n ---- Singled Layer Model ----')
print('units_single_layer: ', units_single_layer)
print('actication_func: ', actication_func)
model = Sequential()

model.add(Dense(units_single_layer, input_shape=(32*32*3,)))
model.add(Activation(actication_func))
model.add(Dense(nb_classes))
model.add(Activation('softmax'))

# Compile model
print('\n ---- Optimizer ----')
print('optimizer: ', optimizer)
print('batch_size: ', optimizer)

print('')
model.compile(loss='categorical_crossentropy',
              optimizer=optimizer,
              metrics=['accuracy'])
# Fit the model
#history = model.fit(X, Y, validation_split=0.33, nb_epoch=150, batch_size=10, verbose=0)
history = model.fit(X_train, Y_train,
          batch_size=batch_size,
          nb_epoch=nb_epoch,
          validation_data=(X_test, Y_test),
          shuffle=True)

pickle.dump( history, open( "history.p", "wb" ) )

并绘制它:

绘图.py

# Visualize training history
from keras.models import Sequential
from keras.layers import Dense, Dropout, Activation, Flatten
from keras.datasets import cifar10
from keras.utils import np_utils

import matplotlib.pyplot as plt

import numpy
import pickle

history = pickle.load( open( "history.p", "rb" ) )

# list all data in history
print(history.history.keys())
# summarize history for accuracy
plt.plot(history.history['acc'])
plt.plot(history.history['val_acc'])
plt.title('model accuracy')
plt.ylabel('accuracy')
plt.xlabel('epoch')
plt.legend(['train', 'test'], loc='upper left')
plt.show()
# summarize history for loss
plt.plot(history.history['loss'])
plt.plot(history.history['val_loss'])
plt.title('model loss')
plt.ylabel('loss')
plt.xlabel('epoch')
plt.legend(['train', 'test'], loc='upper left')
plt.show()

此外,如果您由于框架构建而对 matplot lib 有问题,只需使用已经为您解决该问题的虚拟环境:

python3 -m venv <my-virtual-env-name>

http://matplotlib.org/faq/osx_framework.html#virtualenv

http://matplotlib.org/faq/osx_framework.html

http://matplotlib.org/faq/virtualenv_faq.html

使用python3的-m venvvirtualenv实现,该实现使python成为一个框架。

4

1 回答 1

1

尝试仅腌制损失字典:

pickle.dump( history.history, open( "history.p", "wb" ) )
于 2017-02-24T19:56:38.497 回答