1

考虑一个具有 2 个隐藏层的神经网络。我想计算第二个隐藏层参数的梯度wrt损失,修改它,然后使用修改后的梯度反向传播并找到第一个隐藏层的梯度。我目前正在这样做:

with tf.GradientTape(persistent=True) as tape1:
    # Calculate forward pass
    pred = model(X_data)
    # Calculate loss
    loss = tf.keras.losses.binary_crossentropy(y_data, pred)
#calculate gradients of 2nd hidden layer    
model_gradients_second_layer = tape1.gradient(loss,model.trainable_variables[2:])

#modify model_gradients_later_layer say multiply by 2
model_gradients_second_layer_modified = 2*model_gradients_second_layer 

# calculate gradients of 1st hidden layer
model_gradients_first_layer = tape1.gradient(model.trainable_variables[2],model.trainable_variables[0:2],model_gradients_second_layer_modified [0])    

model_gradients_all_layers = model_gradients_first_layer + model_gradients_second_layer_modified 
optimizer.apply_gradients(zip(model_gradients_all_layers , model.trainable_variables))

但是 model_gradients_first_layer 以 None 的形式出现。

4

0 回答 0