1

在计算误差导数时，我正在使用以下工作，但不确定原因。

double errorDerivative = (-output * (1-output) *(desiredOutput - output));

当我从第一个输出中删除减号时，它会失败并达到最大时期限制。我假设这就是不使用减号运算符的http://homepages.gold.ac.uk/nikolaev/311imlti.htm示例。

double errorDerivative2 = (output * (1-output) *(desiredOutput - output));

我目前正在查看一个修改现有的 BackPropagation 实现，它使用随机梯度下降并希望让它使用标准的反向传播算法。目前，它看起来像这样。

public void applyBackpropagation(double expectedOutput[]) {

        // error check, normalize value ]0;1[
        /*for (int i = 0; i < expectedOutput.length; i++) {
            double d = expectedOutput[i];
            if (d < 0 || d > 1) {
                if (d < 0)
                    expectedOutput[i] = 0 + epsilon;
                else
                    expectedOutput[i] = 1 - epsilon;
            }
        }*/

        int i = 0;
        for (Neuron n : outputLayer) {
            System.out.println("neuron");
            ArrayList<Connection> connections = n.getAllInConnections();
            for (Connection con : connections) {
                double output = n.getOutput();
                System.out.println("final output is "+output);
                double ai = con.leftNeuron.getOutput();
                System.out.println("ai output is "+ai);
                double desiredOutput = expectedOutput[i];

                double errorDerivative = (-output * (1-output) *(desiredOutput - output));
                double errorDerivative2 = (output * (1-output) *(desiredOutput - output));
                System.out.println("errorDerivative is "+errorDerivative);
                System.out.println("errorDerivative my one is "+(output * (1-output) *(desiredOutput - output)));
                double deltaWeight = -learningRate * errorDerivative2;
                double newWeight = con.getWeight() + deltaWeight;
                con.setDeltaWeight(deltaWeight);
                con.setWeight(newWeight + momentum * con.getPrevDeltaWeight());
            }
            i++;
        }

        // update weights for the hidden layer
        for (Neuron n : hiddenLayer) {
            ArrayList<Connection> connections = n.getAllInConnections();
            for (Connection con : connections) {
                double output = n.getOutput();
                double ai = con.leftNeuron.getOutput();
                double sumKoutputs = 0;
                int j = 0;
                for (Neuron out_neu : outputLayer) {
                    double wjk = out_neu.getConnection(n.id).getWeight();
                    double desiredOutput = (double) expectedOutput[j];
                    double ak = out_neu.getOutput();
                    j++;
                    sumKoutputs = sumKoutputs
                            + (-(desiredOutput - ak) * ak * (1 - ak) * wjk);
                }

                double partialDerivative = output * (1 - output) * ai * sumKoutputs;
                double deltaWeight = -learningRate * partialDerivative;
                double newWeight = con.getWeight() + deltaWeight;
                con.setDeltaWeight(deltaWeight);
                con.setWeight(newWeight + momentum * con.getPrevDeltaWeight());
            }
        }
    }
4

1 回答 1

2

抱歉，我不会查看您的代码 - 没时间了，您将不得不带着更具体的问题回来，然后我可以帮助您。

errorDerivative2 起作用的原因可能是您正在使用权重更新规则，例如
deltaW = learningRate*errorDerivative2*input

通常，您所说的“errorDerivative2”被称为delta，并被定义为
-output * (1-output) *(desiredOutput - output)
具有 sigmoid 传递函数的神经元

使用权重更新规则
deltaW = -learningRate*delta*input

所以基本上它在没有减号的情况下对你有用，errorDerivative2因为你在另一个地方也遗漏了一个减号..

于 2012-02-29T09:34:26.633 回答