我正在尝试为多层前馈神经网络实现反向传播算法，但我在让它收敛到好的结果时遇到了问题。原因是，梯度下降卡在均方根误差的板上。
正如您在图中看到的，前 70 个 epoch 左右的 rms 值几乎没有变化。因此梯度下降找到了一个最小值并停止。为了解决这个问题，我设置了一个要求，即除了变化率低于给定值之外，rms 误差必须低于 0.3。但是，我认为这并不好，因为我认为我的实现有问题。
以下是红宝石代码：
def train eta, criteria
rms = 1
old_rms = 0
rms_window = Array.new 20, 0
new_avg = 10
old_avg = 0
diff = 100
epoch = 0
@data[:training].shuffle!
while (diff > criteria || rms > 0.3) do
#while (diff > criteria) do
rms = 0
old_avg = new_avg
new_avg = 0
classification_error = 0
sample_num = 0
@data[:training].each_with_index do |s, s_i|
# Forward Propagation
inputs = [1, s[1], s[2]]
@hidden_layers.each_with_index do |hl, hl_i|
outputs = Array.new
# Bias Term
outputs << 1
# Compute the output for each neuron
hl.each do |p|
outputs << p.compute_output(inputs)
end
inputs = outputs
end
# Compute System Outputs
outputs = Array.new
@outputs.each do |p|
outputs << p.compute_output(inputs)
end
# Comput Errors
errors = Array.new
desired = @desired_values[s[0]-1]
@outputs.length.times do |x|
errors[x] = desired[x] - outputs[x]
rms += errors[x]**2
end
decision = outputs.each_with_index.max[1]
if decision+1 != s[0]
classification_error += 1
end
# Back Propagation
gradients = Array.new
local_gradient = Array.new
next_layer = Array.new
@outputs.each_with_index do |o, i|
local_gradient << errors[i] * o.activation_prime(o.output)
o.weights.length.times do |x|
o.weights[x] += eta * local_gradient[i] * o.inputs[x]
end
end
gradients << local_gradient
next_layer = @outputs
@hidden_layers.reverse_each do |hl|
local_gradient = Array.new
hl.each do |p|
gradient = 0
gradients.last.each_with_index do |g, i|
gradient += g * next_layer[i].weights[p.index+1]
end
gradient *= p.activation_prime(p.output)
local_gradient << gradient
p.weights.each_index do |x|
p.weights[x] += eta * gradient * p.inputs[x]
end
end
gradients << local_gradient
next_layer = hl
end
if s_i == 0
#puts "Epoch: #{epoch}\nOutputs: #{outputs}\nGradients:\n#{gradients[0]}\n#{gradients[1]}\n#{gradients[2]}\n\n"
#puts "Epoch #{epoch}\nError: #{errors}\nSE: #{rms}"
end
end
rms = Math::sqrt(rms / (@data[:training].length * 4))
rms_window[0] = rms
rms_window.rotate!
rms_window.each do |x|
new_avg += x
end
new_avg /= 20
diff = (new_avg - old_avg).abs
@rms << rms
epoch += 1
if classification_error == 0
break
end
#puts "RMS: #{rms}\tDiff: \t#{diff}\tClassification: #{classification_error}\n\n"
end
self.rms_plot "Plot"
self.grid_eval "Test", 250
end
显示的图表是一个 2 隐藏层网络，每个隐藏层有 5 个神经元。有 2 个输入和 4 个输出。也许这是正常行为，但对我来说似乎有些不对劲。任何帮助将不胜感激。