我需要使用 Tensorflow/Keras 创建一个自定义训练循环(因为我想拥有多个优化器并告诉每个优化器应该处理哪些权重)。
尽管这个教程和那个教程对这个问题也很清楚,但他们忽略了一个非常重要的点:我如何预测训练阶段以及如何预测验证阶段?
假设我的模型有Dropout
层或BatchNormalization
层。无论是在训练还是在验证中,它们肯定以完全不同的方式工作。
我如何调整这些教程?这是一个虚拟示例(可能包含一两段伪代码):
# Iterate over epochs.
for epoch in range(3):
# Iterate over the batches of the dataset.
for step, (x_batch_train, y_batch_train) in enumerate(train_dataset):
with tf.GradientTape() as tape:
#model with two outputs
#IMPORTANT: must be in training phase (use dropouts, calculate batch statistics)
logits1, logits2 = model(x_batch_train) #must be "training"
loss_value1 = loss_fn1(y_batch_train[0], logits1)
loss_value2 = loss_fn2(y_batch_train[1], logits2)
grads1 = tape.gradient(loss_value1, model.trainable_weights[selection1])
grads2 = tape.gradient(loss_value2, model.trainable_weights[selection2])
optimizer1.apply_gradients(zip(grads1, model.trainable_weights[selection1]))
optimizer2.apply_gradients(zip(grads2, model.trainable_weights[selection2]))
# Run a validation loop at the end of each epoch.
for x_batch_val, y_batch_val in val_dataset:
##Important: must be validation phase
#dropouts are off: calculate all neurons and divide value
#batch norms use previously calculated statistics
val_logits1, val_logits2 = model(x_batch_val)
#.... do the evaluations