1

在学习了几个教程和 Stackoverflow 问题之后，我设法在我的自定义环境中学习了 Double Dueling DQN Agent。

现在我无法使用它的优势（）参数保存和加载代理的网络模型。如何保存和加载此类自定义序列化网络？

Python 3.8.5 张量流 2.3.1 Keras 2.4.3

到目前为止，我的方法很简单：

class DuelingDeepQNAgent():
    ....
    def load_model(self, model_name):
        self.q_network = load_model(model_name)
        self.target_network = load_model(model_name)

    def save_model(self, model_name):
        self.q_network.save(model_name, save_format='tf')

加载保存的模型并继续学习会导致该错误：

agent = MyAgent.DuelingDeepQNAgent()
agent.load_model(model_name)
.... 
# within learning process
q_values = self.q_network.advantage(state)

AttributeError：“DuelingDeepQNetwork”对象没有属性“优势”

网络：

class DuelingDeepQNetwork(keras.Model):
    def __init__(self, input_shape, output_shape, n_filter=64, kernel_size=3, n_neurons1=256, n_neurons2=128):
        super(DuelingDeepQNetwork, self).__init__()

        self.input_layer = keras.layers.InputLayer(input_shape=(None, *input_shape))
        # self.lstm1 = keras.layers.LSTM(units=n_neurons1, activation='relu', stateful=True, return_state=True)
        self.conv1d1 = keras.layers.Conv1D(filters=n_filter, kernel_size=kernel_size, padding='same', activation='relu')
        self.pool1 = keras.layers.AvgPool1D(pool_size=2)
        self.conv1d2 = keras.layers.Conv1D(filters=n_filter, kernel_size=kernel_size, padding='same', activation='relu')
        self.pool2 = keras.layers.AvgPool1D(pool_size=2)
        self.flat1 = keras.layers.Flatten()
        self.dens_1 = keras.layers.Dense(n_neurons1, activation='relu')
        self.dens_2 = keras.layers.Dense(n_neurons2, activation='relu')

        self.V = keras.layers.Dense(1, activation=None)  # Value layer
        self.A = keras.layers.Dense(output_shape, activation=None)  # Advantage layer

    def call(self, state):
        x = self.input_layer(state)
        x = self.conv1d1(x)
        x = self.pool1(x)
        x = self.conv1d2(x)
        x = self.pool2(x)
        x = self.flat1(x)
        x = self.dens_1(x)
        x = self.dens_2(x)

        A = self.A(x)
        V = self.V(x)
        Q = V + (A - tf.math.reduce_mean(A, axis=1, keepdims=True))

        return Q

    def advantage(self, state):
        x = self.input_layer(state)
        x = self.conv1d1(x)
        x = self.pool1(x)
        x = self.conv1d2(x)
        x = self.pool2(x)
        x = self.flat1(x)
        x = self.dens_1(x)
        x = self.dens_2(x)

        A = self.A(x)
        return A
4

1 回答 1

0

编辑：我之前的答案是对代码的缺失部分做出假设，但我相信以下实际上是正确的答案。

我实际上在类似的问题上做了更多的工作，您可能遇到的问题是，当您加载保存的 H5 模型时，它不会加载自定义类的关联函数。您需要做的是仅保存/加载权重以确保保留类方法。

def save_model(self):
     self.q_network.save_weights("model_name") #no *.h5 extension
def load_model(self):
    self.q_network.load_weights("model_name")
于 2021-07-20T21:36:30.393 回答