3

我编写了一个简单的 Keras 代码，其中我使用 CNN 来处理时尚 mnist 数据集。一切都很好。我实现了自己的类，分类没问题。

但是，我想使用 Optuna，作为 OptKeras（Keras 的 Optuna 包装器），您可以在此处查看示例：https ://medium.com/@Minyus86/optkeras-112bcc34ec73 。

但是，我的代码有问题。当我尝试在自己的班级中使用 optKeras 时。这是代码：（普通run方法有效，但optuna_run给出错误：AttributeError: type object 'FrozenTrial' has no attribute '_field_types'.

! pip install optkeras
# -*- coding: utf-8 -*- 
#!/usr/bin/env python3

import tensorflow as tf
from tensorflow import keras

from keras.models import Sequential
from keras.layers import Dense, SimpleRNN 
from keras.callbacks import ModelCheckpoint
from keras import backend as K

from sklearn.metrics import r2_score
from sklearn.metrics import mean_squared_error
from sklearn.metrics import mean_absolute_error
from sklearn.preprocessing import MinMaxScaler
from sklearn.model_selection import train_test_split
import sklearn.metrics

import optuna
from optkeras.optkeras import OptKeras

import sys
import math
import numpy
import scipy.io as sio   
import matplotlib.pyplot as plt

class OptunaTest():

  def __init__(self):
    self.fashion_mnist = keras.datasets.fashion_mnist
    (self.train_images, self.train_labels), (self.test_images, self.test_labels) = self.fashion_mnist.load_data()
    self.class_names = ['T-shirt/top', 'Trouser', 'Pullover', 'Dress', 'Coat', 'Sandal', 'Shirt', 'Sneaker', 'Bag', 'Ankle boot']
    self.train_images = self.train_images / 255.0
    self.test_images = self.test_images / 255.0
    self.model = None 
    self.study_name = 'FashionMnist' + '_Simple'
    self.ok = OptKeras(study_name=self.study_name)

  def run(self):
    self.model = keras.Sequential()
    self.model.add(keras.layers.Flatten(input_shape=(28, 28)))
    self.model.add(keras.layers.Dense(128, activation='relu'))
    self.model.add(keras.layers.Dense(10))
    self.model.compile(optimizer='adam', loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True), metrics=['accuracy'])
    self.model.fit(self.train_images, self.train_labels, epochs=5)
    test_loss, test_acc = self.model.evaluate(self.test_images, self.test_labels, verbose=0)
    predictions = self.model.predict(self.test_images)

    INDEX = 10
    print("\nPREDICTION: " + str(predictions[INDEX]))
    print("\nMAX PREDICTION VAL: " + str(numpy.argmax(predictions[INDEX])))
    print("\nLABEL: " + str(self.test_labels[INDEX]))

  def optuna_run(self, trial):
    K.clear_session() 
    
    self.model = keras.Sequential()
    self.model.add(keras.layers.Flatten(input_shape=(28, 28)))
    self.model.add(keras.layers.Dense(units = trial.suggest_categorical('units', [32, 64, 128]), activation = trial.suggest_categorical('activation', ['relu', 'linear'])))
    self.model.add(keras.layers.Dense(units = trial.suggest_categorical('units', [32, 64, 128]), activation = trial.suggest_categorical('activation', ['relu', 'linear'])))

    self.model.compile(optimizer='adam', loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True), metrics=['accuracy'])
    self.model.fit(self.train_images, self.train_labels, epochs=5, callbacks = self.ok.callbacks(trial), verbose = self.ok.keras_verbose)
    test_loss, test_acc = self.model.evaluate(self.test_images, self.test_labels, verbose=0)
    predictions = self.model.predict(self.test_images)
    print(ok.trial_best_value)
    
    INDEX = 10
    print("\nPREDICTION: " + str(predictions[INDEX]))
    print("\nMAX PREDICTION VAL: " + str(numpy.argmax(predictions[INDEX])))
    print("\nLABEL: " + str(self.test_labels[INDEX]))


if __name__ == "__main__":
  ot = OptunaTest()
  ot.run()

  ot.ok.optimize(ot.optuna_run,  timeout = 60)

代码也可以在这里找到：https ://colab.research.google.com/drive/1uibWa80BdjatA5Kcw27eMUsS7SmwxaDk?usp=sharing 。

完整的错误信息：

[W 2020-06-30 11:09:26,959] Setting status of trial#0 as TrialState.FAIL because of the following error: AttributeError("type object 'FrozenTrial' has no attribute '_field_types'",)
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/optkeras/optkeras.py", line 230, in synch_with_optuna
    self.best_trial = self.study.best_trial
  File "/usr/local/lib/python3.6/dist-packages/optuna/study.py", line 97, in best_trial
    return copy.deepcopy(self._storage.get_best_trial(self._study_id))
  File "/usr/local/lib/python3.6/dist-packages/optuna/storages/in_memory.py", line 293, in get_best_trial
    raise ValueError("No trials are completed yet.")
ValueError: No trials are completed yet.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/optuna/study.py", line 734, in _run_trial
    result = func(trial)
  File "/usr/local/lib/python3.6/dist-packages/optkeras/optkeras.py", line 130, in fun_tf
    return fun(trial)
  File "<ipython-input-11-45495c9f2ae9>", line 65, in optima_run
    self.model.fit(self.train_images, self.train_labels, epochs=10, callbacks = self.ok.callbacks(trial), verbose = self.ok.keras_verbose)
  File "/usr/local/lib/python3.6/dist-packages/optkeras/optkeras.py", line 172, in callbacks
    self.synch_with_optuna()
  File "/usr/local/lib/python3.6/dist-packages/optkeras/optkeras.py", line 232, in synch_with_optuna
    self.best_trial = get_trial_default()
  File "/usr/local/lib/python3.6/dist-packages/optkeras/optkeras.py", line 367, in get_trial_default
    num_fields = optuna.structs.FrozenTrial._field_types.__len__()
AttributeError: type object 'FrozenTrial' has no attribute '_field_types'

---------------------------------------------------------------------------

ValueError                                Traceback (most recent call last)

/usr/local/lib/python3.6/dist-packages/optkeras/optkeras.py in synch_with_optuna(self)
    229         try:
--> 230             self.best_trial = self.study.best_trial
    231         except:

12 frames

ValueError: No trials are completed yet.


During handling of the above exception, another exception occurred:

AttributeError                            Traceback (most recent call last)

/usr/local/lib/python3.6/dist-packages/optkeras/optkeras.py in get_trial_default()
    365 
    366 def get_trial_default():
--> 367     num_fields = optuna.structs.FrozenTrial._field_types.__len__()
    368     assert num_fields in (10, 11, 12)
    369     if num_fields == 12: # possible future version

AttributeError: type object 'FrozenTrial' has no attribute '_field_types'
4

1 回答 1

3

似乎 optkeras（我得到的版本是 0.0.7）与 optuna 库不是最新的是问题的原因。通过进行以下更改，我能够使其与 optuna 1.5.0 一起使用：

get_default_trial首先，在运行代码之前，您需要像这样进行猴子补丁：

import optkeras
optkeras.optkeras.get_trial_default = lambda: optuna.trial.FrozenTrial(
            None, None, None, None, None, None, None, None, None, None, None)

这样做之后，我说错了Callback：

AttributeError: 'OptKeras' object has no attribute '_implements_train_batch_hooks'

要解决这个问题，您必须手动编辑 optkeras.py，但不要太多 - 只需添加tensorflow.到前两行导入，即制作它们：

import tensorflow.keras.backend as K
from tensorflow.keras.callbacks import Callback, CSVLogger, ModelCheckpoint

代替：

import keras.backend as K
from keras.callbacks import Callback, CSVLogger, ModelCheckpoint

如果您在安装后无法更改代码，这可能会有点问题 - 我可能只是建议复制 optkeras 库的完整代码（它只是一个文件 optkeras.py）并在您的脚本中使用该固定版本或类似的东西。不幸的是，我没有看到猴子修补这个导入问题的好方法。也就是说，我认为即使从 python 中即时更改（即optkeras.py在导入 optkeras 之前从 python 中更改行）或复制 optkeras.py（也来自 withing python 脚本）+ 即时替换字符串也很容易，然后从新位置导入。

完成后，我只需要：

  • 修复代码中的错字（print(ok.trial_best_value)应该是print(self.ok.trial_best_value)）
  • 添加validation_split=0.1到self.model.fit调用（或者您可以使用其他东西进行调整 - 仅使用现有代码示例回调不会获得val_loss价值，因为没有验证集并且 optkerasval_loss默认使用 - 请参阅构造函数monitor的参数OptKeras）。我的猜测是，您可能想要创建一个固定的验证集，或者监控训练损失loss而不是val_loss.
  • return test_loss在方法末尾添加optuna_run。

在所有这些更改之后，一切似乎都在工作。

于 2020-07-05T02:50:27.550 回答