1

我正在使用 TensorFlow Keras 构建和拟合深度神经网络 (DNN)。我读过的文献鼓励我使用 Dropout 层。但是，当我添加dropout值为 的图层时0.5，我的渐变体面进展表现得很奇怪。具体来说，训练准确度和验证准确度似乎有一定的硬性限制。

这是什么效果？为什么会这样？任何意见或反馈将不胜感激。

_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
dropout_1 (Dropout)          (None, 136)               0         
_________________________________________________________________
dense_1 (Dense)              (None, 84)                11508     
_________________________________________________________________
dropout_2 (Dropout)          (None, 84)                0         
_________________________________________________________________
batch_normalization_1 (Batch (None, 84)                336       
_________________________________________________________________
dense_2 (Dense)              (None, 42)                3570      
_________________________________________________________________
batch_normalization_2 (Batch (None, 42)                168       
_________________________________________________________________
dropout_3 (Dropout)          (None, 42)                0         
_________________________________________________________________
dense_3 (Dense)              (None, 21)                903       
=================================================================
Total params: 16,485
Trainable params: 16,233
Non-trainable params: 252

matplotlib.pyplot 精度

这是一些代码：

InputCount = 136
OutputCount = 21
H1_Count = 84
H2_Count = 42
H3_Count = 0
H4_Count = 0
initializer = 'he_normal'
activation = ['' ,'elu','elu','elu','elu']
dropout    = [0.5,0.5  ,0.5  ,0.5  ,0.5  ]
batch_norm = 1

optimizer = optimizers.Adam(lr=0.001, beta_1=0.9, beta_2=0.999, epsilon=None, decay=0.0, amsgrad=False)
loss = 'mse' 

epochs = 500
batch_size = 10000  #Stochatic mini-batch size

model = Sequential()

if (dropout[0] > 0) :
    model.add(Dropout(dropout[0], input_shape=(InputCount,)) )
    model.add(Dense(H1_Count, activation=activation[1], kernel_initializer=initializer ))
    if (dropout[1] > 0) : model.add(Dropout(dropout[1]))
else :
    model.add(Dense(H1_Count, activation=activation[1], kernel_initializer=initializer, input_shape=(InputCount,) ))
    if (dropout[1] > 0) : model.add(Dropout(dropout[1]))

if (batch_norm == 1) : model.add(BatchNormalization())

if (H2_Count > 0) :
    model.add(Dense(H2_Count,activation=activation[2], kernel_initializer=initializer))
    if (batch_norm == 1) : model.add(BatchNormalization())
    if (dropout[2] > 0) : model.add(Dropout(dropout[2]))

if (H3_Count > 0) :
    model.add(Dense(H3_Count, activation=activation[3], kernel_initializer=initializer))
    if (batch_norm == 1) : model.add(BatchNormalization())
    if (dropout[3] > 0) : model.add(Dropout(dropout[3]))

if (H4_Count > 0) :
    model.add(Dense(H4_Count, activation=activation[4], kernel_initializer=initializer))
    if (batch_norm == 1) : model.add(BatchNormalization())
    if (dropout[4] > 0) : model.add(Dropout(dropout[4]))

model.add(Dense(OutputCount, activation='linear'))

model.summary()
4

1 回答 1

1

Dropout 是一种很好的正则化技术。但是 dropout 的值是一种超参数，你必须调整它。

dropout 所做的基本上是关闭指定百分比的神经元，以便网络可以看到哪些未关闭的神经元正在恢复——无论它是否是无用的信息。每次随机关闭一组不同的神经元。

所以，我认为你的问题是关闭神经元的百分比太高（50%）。将其调低至 20%-30%。另一个错误是在最后一层和第一层之前使用 dropout。删除这 2 个 dropout 层并调低 dropout 的百分比（辍学率），你应该没问题 :)

于 2019-01-07T14:30:22.340 回答