softmax - 缩放 sparse_softmax_cross_entropy_with_logits

Question

我如何缩放损失来自sparse_softmax_cross_entropy_with_logits的梯度。例如，我试图除以 128，如下所示，但我发现错误：

new_gradients = [(grad/128, var) for (grad, var) in gradients] 
TypeError: unsupported operand type(s) for /: 'IndexedSlices' and 'int'

我使用的代码如下：

loss = tf.nn.sparse_softmax_cross_entropy_with_logits(logits=logits, labels=labels)

gradients = opt.compute_gradient(loss)

new_gradients = [(grad/128, var) for (grad, var) in gradients]

train_step = opt.appy_gradients(new_gradients)

Answer 1

我找到了解决问题的方法如下：

new_gradients = [(grad/128, var) for (grad, var) in gradients]

应该

new_gradients = [(tf.div(grad, 128), var) for (grad, var) in gradients]