0

我试图用 Matconvnet DagNN 来理解 DNN。我有一个基于以下最后两层网络的问题，该网络使用欧几里得损失进行回归

net.addLayer('fc9', dagnn.Conv('size', [1 1 4096 1], 'hasBias', true, 'stride', [1,1], 'pad', [0 0 0 0]), {'drop8'}, {'prediction'},  {'conv10f'  'conv10b'});
 net.addLayer('l2_loss', dagnn.L2Loss(), {'prediction', 'label'}, {'objective'});

L2Loss 的代码在哪里

function Y=vl_nnL2(X,c,dzdy)
 c=reshape(c,size(X));
 if nargin == 2 || (nargin == 3 && isempty(dzdy))
    diff_xc=(bsxfun(@minus, X,(c)));
    Y=diff_xc.^2;
 elseif nargin == 3 && ~isempty(dzdy)
    Y=(X-c).*dzdy;
 end
end

X是fc9层的输出，是长度为100（batch size）的特征向量，c是标签。

  1. 在损失函数中，如何比较两者？X 是一个激活，一个不是概率的向量。我猜。C是标签，整数值范围从..0-10。那么如何比较和减去它们呢？不知道这两者有没有关系？
  2. 此外，反向传播如何比较 fc9 输出和标签以实现最小化？

*-----------新修改的L2回归函数

function Y=vl_nnL2_(X,c,dzdy)
    c=reshape(c,size(X));
    [~,chat] = max(X,[],3) ;
    [~,lchat] = max(c,[],3) ; 
if nargin == 2 || (nargin == 3 && isempty(dzdy))
      t = (chat-lchat).^ 2 ;
     Y=sum(sum(t));
elseif nargin == 3 && ~isempty(dzdy)
  ch=squeeze(chat);
  aa1=repmat(ch',35,1);
  lch=squeeze(lchat);
  aa2=repmat(lch',35,1);
  t = (chat-lchat);
  Y = dzdy.*(aa1-aa2)*2;
Y = single(reshape(Y,size(X)));

end
end

enter image description here

4

1 回答 1

0

"if nargin == 2 || (nargin == 3 && isempty(dzdy))" checks if it's forward mode.

In the forward mode, you compute (prediction - label).^2:

diff_xc=(bsxfun(@minus, X,(c)));
Y=diff_xc.^2;

The derivative of L2 loss w.r.t. prediction is 2*(prediction - label). Thus we have 

Y=(X-c).*dzdy;

in your code. Here the author of your code isn't rigorous enough to put the constant 2*. But in general it will work since it's just a constant scaling factor on your gradients. dzdy is the gradient from downstream layers. If this layer is the last one, dzdy=1, which manually provided by MatConvnet.

c must be of the same size as X since its' regression.

More comments coming. Let me know if you have other questions. I'm pretty familiar with MatConvNet.

于 2017-05-21T22:19:23.730 回答