0

我正在尝试对产生无意义预测的模型进行故障排除。在此过程中，我添加了另一个密集层，并且火车时间与没有它时保持相同。所以我添加了 4 x 512 个神经元密集层，时间是一样的..

我使用 TPU v3，但仍然 - 计算 4 个额外的完全连接的网络，每个网络有 512 个神经元，我预计需要一些额外的时间。

任何人都可以请帮助解释这一点，或者至少给出一个调查的想法

编辑： LSTM + 1 Dense 的可训练参数数 = 4 百万；LSTM + 4 密集 = 5 百万

model = tf.keras.Sequential()
model.add(layers.LSTM(neurons, input_shape=(window_size, inputs_n), return_sequences=True)) 
model.add(layers.Dropout(0.2))
model.add(layers.LSTM(neurons))
model.add(layers.Dense(neurons, activation='relu'))
model.add(layers.Dropout(0.2))
model.add(layers.Dense(neurons, activation='relu'))
model.add(layers.Dropout(0.2))
model.add(layers.Dense(neurons, activation='relu'))
model.add(layers.Dropout(0.2))
model.add(layers.Dense(neurons, activation='relu'))
model.add(layers.Dropout(0.2))
model.add(layers.Dense(outputs_n, activation=activation))

TPU_ADDRESS1 = 'grpc://10.240.2.2:8470'

opt = tf.train.AdamOptimizer(learning_rate)

model.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['categorical_accuracy'])
print(model.metrics_names)

tpu_model = tf.contrib.tpu.keras_to_tpu_model(model, 
        strategy=tf.contrib.tpu.TPUDistributionStrategy(
            tf.contrib.cluster_resolver.TPUClusterResolver(tpu = [TPU_ADDRESS1])))

Layer (type)                 Output Shape              Param #   
=================================================================
lstm_input (InputLayer)      (None, 1024, 7)           0         
_________________________________________________________________
lstm (LSTM)                  (None, 1024, 512)         1064960   
_________________________________________________________________
dropout (Dropout)            (None, 1024, 512)         0         
_________________________________________________________________
lstm_1 (LSTM)                (None, 512)               2099200   
_________________________________________________________________
dense (Dense)                (None, 512)               262656    
_________________________________________________________________
dropout_1 (Dropout)          (None, 512)               0         
_________________________________________________________________
dense_1 (Dense)              (None, 512)               262656    
_________________________________________________________________
dropout_2 (Dropout)          (None, 512)               0         
_________________________________________________________________
dense_2 (Dense)              (None, 512)               262656    
_________________________________________________________________
dropout_3 (Dropout)          (None, 512)               0         
_________________________________________________________________
dense_3 (Dense)              (None, 512)               262656    
_________________________________________________________________
dropout_4 (Dropout)          (None, 512)               0         
_________________________________________________________________
dense_4 (Dense)              (None, 4)                 2052      
=================================================================
Total params: 4,216,836
Trainable params: 4,216,836
Non-trainable params: 0
_________________________________________________________________
4

0 回答 0