2

我有一个带有一个输入和一个输出的玩具数据集，输入是随机的 0 或 1，输出等于之前 3 个样本的输入

Input, Output
0,1
1,1
0,0
1,0
0,1
0,0

型号 1：

lstm = nn.Sequential()
lstm:add(nn.LSTM(1,20,8))
lstm:add(nn.Linear(20,100))
lstm:add(nn.Sigmoid())
lstm:add(nn.Linear(100,1))
lstm:add(nn.Sigmoid())

型号 2：

lstm = nn.Sequencer(
   nn.Sequential()
      :add(nn.LSTM(1,100))
      :add(nn.Linear(100,1))
      :add(nn.Sigmoid())
   )

标准：

criterion = nn.BCECriterion()
trainer = nn.StochasticGradient(lstm, criterion)
trainer.learningRate = .01
trainer.maxIteration = 25

第一个模型不学习，第二个模型不起作用（可能是因为我的数据对于 Sequencer 来说是错误的类型？）

编辑：

训练：

lstm:training()
print("Training")
for epoch=1, 5 do
  err = 0
  for i = 1,data:size() do
    input = {}
    output = {}
    table.insert(input, data[i][1])
    table.insert(output, data[i][2])
    --input = data[i][1]
    --output = data[i][2]
    err = err + criterion:forward(lstm:updateOutput(input)[1], output)
    lstm:zeroGradParameters()
    lstm:backward(input, {criterion:backward(lstm.output[1], output)})
    lstm.modules[1]:backwardThroughTime()
    lstm:updateParameters(0.01)
    lstm.modules[1]:forget()
  end
  print (err / data:size())
end

我现在在通过训练运行 Model 2 时遇到错误。错误

AbstractRecurrent.lua:61：缺少 gradInput

在线上发生

lstm.modules[1]:backwardThroughTime()

4

1 回答 1

2

查看将 LSTM 与 Sequencer 结合使用的玩具示例。该模型除了尝试预测随机数之外什么都不做。

require 'nn'
require 'torch'
require 'rnn'

lstm = nn.Sequencer(
   nn.Sequential()
      :add(nn.LSTM(1,100))
      :add(nn.Linear(100,1))
      :add(nn.Sigmoid())
   )

-- Encapsulate thr criterion using a Sequencer
-- Just provide the entire sequence as input and the corresponding
-- target sequence as expected output
criterion = nn.SequencerCriterion(nn.BCECriterion())

data = torch.zeros(10,2)
for i=1,data:size(1) do
  data[i][1] = torch.uniform()
  data[i][2] = torch.uniform()
end

local inputs, targets = {},{}

for epoch=1,5 do
  lstm:training()

  for i=1,data:size(1) do
    --useful for minibatch
    inputs[1] = torch.zeros(1)
    inputs[1][1] = data[i][1]

    targets[1] = torch.zeros(1)
    targets[1][1] = data[i][2]

    local output = lstm:forward(inputs)
    local err = criterion:forward(output, targets)

    local gradOutputs = criterion:backward(output, targets)
    -- Sequencer handles the backwardThroughTime internally
    lstm:backward(inputs, gradOutputs)
    lstm:updateParameters(0.01)
    lstm:zeroGradParameters()

    inputs = {}
    targets = {}
  end
end
于 2015-11-26T05:36:55.783 回答