3

I have built a simple recurrent neural network that predicts a very noisy signal from a series of 15 inputs (statistical breakdowns of the signal).

From what I can tell in the pybrain source (pybrain\supervised\trainers\backprop.py), the error function is hardcoded in the _calcDerivs function as the sum of the squared errors divided by the total targets (MSE). The division happens in the train function.

In my case, it is most important that the network predict the direction of signal change over the exact change amount, so I want to penalize the NN when it predicts down but signal moves up and vice-versa. I've been experimenting with passing _calcDerivs not only the current target, but also the previous target and outputs, which I use to calculate a weight based on whether or not the target guessed the direction correctly, but the network fails to converge using both rprop and backprop. This whole thing is very hack-y to me.

My question is: Is there a best way to modify the default performance function? Is all of the performance function code kept in _calcDerivs or am I missing something?

4

1 回答 1

2

好的，所以我找到了问题的根源和可能的解决方案，尽管 PyBrain 中没有。

我的问题的根源在于我的自定义成本（又名性能等）功能的衍生......或缺乏它。正在使用的成本函数似乎是：

0.5 * (error ** 2) # 1/2 the average square error

它的导数很简单：

error

由于我使用更复杂的导数实现了更复杂的误差函数，并且我没有更改硬编码的导数（无论它应该去哪里），梯度下降无法对误差梯度采取合理的步骤。

我找到的解决方案是使用neurolab，这使得以模块化方式实现自定义错误函数变得更加容易。尽管在核心文件中需要进行一些修改，但我只需要更改大约三四行核心代码。（具体来说，我ff_grad_step在 tool.py 和 core.py 中的类的最后一行进行了修改。Train我通过在 error.py 中创建一个新函数并在 net.py 中使我的网络挂钩来实现我的自定义成本函数。）

我希望对于处于类似情况的其他人来说，这不是我自己的问题太具体，但对于学习神经网络如此关键的东西来说，这是一个巨大的痛苦！

于 2013-06-08T05:56:00.810 回答