介绍
根据千层面文档:“该层应插入线性变换(例如 DenseLayer 或 Conv2DLayer)及其非线性之间。便利函数 batch_norm() 修改现有层以在其非线性之前插入批量归一化。”
然而,千层面也有实用功能:
千层面.layers.batch_norm
但是,由于我的实施,我无法使用该功能。
我的问题是:我应该如何以及在哪里添加 BatchNormLayer?
class lasagne.layers.BatchNormLayer(incoming, axes='auto', epsilon=1e-4, alpha=0.1, beta=lasagne.init.Constant(0), gamma=lasagne.init.Constant(1), mean=lasagne.init.Constant(0), inv_std=lasagne.init.Constant(1), **kwargs)
我可以在卷积层之后添加它吗?还是我应该在 maxpool 之后添加?我是否必须手动消除图层的偏差?
使用的方法 我只是这样使用它:
try:
import lasagne
import theano
import theano.tensor as T
input_var = T.tensor4('inputs')
target_var = T.fmatrix('targets')
network = lasagne.layers.InputLayer(shape=(None, 1, height, width), input_var=input_var)
from lasagne.layers import BatchNormLayer
network = BatchNormLayer(network,
axes='auto',
epsilon=1e-4,
alpha=0.1,
beta=lasagne.init.Constant(0),
gamma=lasagne.init.Constant(1),
mean=lasagne.init.Constant(0),
inv_std=lasagne.init.Constant(1))
network = lasagne.layers.Conv2DLayer(
network, num_filters=60, filter_size=(3, 3), stride=1, pad=2,
nonlinearity=lasagne.nonlinearities.rectify,
W=lasagne.init.GlorotUniform())
network = lasagne.layers.Conv2DLayer(
network, num_filters=60, filter_size=(3, 3), stride=1, pad=1,
nonlinearity=lasagne.nonlinearities.rectify,
W=lasagne.init.GlorotUniform())
network = lasagne.layers.MaxPool2DLayer(incoming=network, pool_size=(2, 2), stride=None, pad=(0, 0),
ignore_border=True)
network = lasagne.layers.DenseLayer(
lasagne.layers.dropout(network, p=0.5),
num_units=32,
nonlinearity=lasagne.nonlinearities.rectify)
network = lasagne.layers.DenseLayer(
lasagne.layers.dropout(network, p=0.5),
num_units=1,
nonlinearity=lasagne.nonlinearities.sigmoid)
return network, input_var, target_var
参考:
https://github.com/Lasagne/Lasagne/blob/master/lasagne/layers/normalization.py#L120-L320
http://lasagne.readthedocs.io/en/latest/modules/layers/normalization.html