以下代码是对我的训练 cicle 的 3 个不同测试,在前两种情况下,梯度返回 Null 向量,在最后一种情况下有效。
这是我第一次尝试,根据网上的信息,with语句记录了其中的可训练变量。
for epoch in range(epochsize):
for batch in batchlist:
loss_value = tf.constant(0.)
mini_batch_losses = []
for seqref in batch:
seqref=int(seqref)
with tf.GradientTape() as tape:
X_train,y_train = loadvalue(seqref) #caricamento elementi
logits = model(X_train, training=True)
loss_value = loss_fn(y_train, logits)
mini_batch_losses.append(loss_value)
loss_avg = tf.reduce_mean(mini_batch_losses)
print("batch " + str(seqref) + " losses:" + str(loss_avg.numpy()))
grads = tape.gradient(loss_avg, model.trainable_weights)
optimizer.apply_gradients(grads_and_vars=zip(grads, model.trainable_weights))
但是由于某些原因它不起作用,所以我尝试(随机)将 with 语句放在前面,但它也不起作用
for epoch in range(epochsize):
for batch in batchlist:
loss_value = tf.constant(0.)
mini_batch_losses = []
with tf.GradientTape() as tape:
for seqref in batch:
seqref=int(seqref)
X_train,y_train = loadvalue(seqref) #caricamento elementi
logits = model(X_train, training=True)
loss_value = loss_fn(y_train, logits)
mini_batch_losses.append(loss_value)
loss_avg = tf.reduce_mean(mini_batch_losses)
print("batch " + str(seqref) + " losses:" + str(loss_avg.numpy()))
grads = tape.gradient(loss_avg, model.trainable_weights)
optimizer.apply_gradients(grads_and_vars=zip(grads, model.trainable_weights))
最后,包括 with 语句中的平均值,它终于起作用了
for epoch in range(epochsize):
for batch in batchlist:
loss_value = tf.constant(0.)
mini_batch_losses = []
with tf.GradientTape() as tape:
for seqref in batch:
seqref=int(seqref)
X_train,y_train = loadvalue(seqref) #caricamento elementi
logits = model(X_train, training=True)
loss_value = loss_fn(y_train, logits)
mini_batch_losses.append(loss_value)
loss_avg = tf.reduce_mean(mini_batch_losses)
print("batch " + str(seqref) + " losses:" + str(loss_avg.numpy()))
grads = tape.gradient(loss_avg, model.trainable_weights)
optimizer.apply_gradients(grads_and_vars=zip(grads, model.trainable_weights))
主要问题是我不知道它是好用还是正常工作..所以我需要更详细地了解 gradientTape 是如何工作的。你能帮助我吗?