python - tensorflow训练后，vgg-net返回nan

Question

我将 vgg-19 网络训练为 classfy cifar10，训练一次后，仅一次，vgg-net 返回nan。

0 [[  4.45161677e+09   2.87961518e+10   4.20765041e+10 ...,          -2.33432433e+10
1.83500431e+10  -1.12923648e+10]
 [  1.18354002e+10   3.38799473e+10   5.86873242e+10 ...,  -4.18343895e+10
2.79392338e+10  -1.61746637e+10]
 [  1.26074880e+09   2.22301839e+10   5.25488333e+10 ...,  -2.92738212e+10
2.51925299e+10  -1.48290714e+10]
 ..., 
 [  1.05694116e+10   2.16351908e+10   5.02961357e+10 ...,  -3.12492278e+10
2.42959094e+10  -1.26112993e+10]
 [  4.72429568e+09   2.75032003e+10   5.14044682e+10 ...,  -3.51395635e+10
2.18048840e+10  -1.46147287e+10]
 [  2.97774285e+09   1.89559747e+10   4.06387917e+10 ...,  -2.35828470e+10
1.96148122e+10  -9.55916698e+09]]
1 [[ nan  nan  nan ...,  nan  nan  nan]
 [ nan  nan  nan ...,  nan  nan  nan]
 [ nan  nan  nan ...,  nan  nan  nan]
 ..., 
 [ nan  nan  nan ...,  nan  nan  nan]
 [ nan  nan  nan ...,  nan  nan  nan]
 [ nan  nan  nan ...,  nan  nan  nan]]

我tf.train.GradientDescentOptimizer用来训练vgg net，活动函数是relu，tf.random_normal初始化权重并tf.nn.xw_plus_b用作全连接层。所以我想知道，为什么 vgg-netnan在训练后返回。

Answer 1

降低学习率可以解决这个数值稳定性问题。