learn = text_classifier_learner(data_clas, AWD_LSTM, drop_mult=0.7)
learn.fit_one_cycle(1, 1e-2)
我已经如上所述训练了 fastai 模型。我可以得到如下预测
preds, targets = learn.get_preds()
但相反,我想要模型的倒数第二层嵌入learn
(这种做法在 CNN 模型中很常见)。你能帮我怎么做吗?
learn = text_classifier_learner(data_clas, AWD_LSTM, drop_mult=0.7)
learn.fit_one_cycle(1, 1e-2)
我已经如上所述训练了 fastai 模型。我可以得到如下预测
preds, targets = learn.get_preds()
但相反,我想要模型的倒数第二层嵌入learn
(这种做法在 CNN 模型中很常见)。你能帮我怎么做吗?
我不确定你是否想要一个分类器,但无论如何......
learn.model
为您提供模型架构。然后learn.model[0]
将是learn.model[1]
模型的另一部分的编码器。
例子:
要访问 SequentialEx(下面的架构)中的第一个线性层,您可以使用以下命令进行操作
learn.model[0].layers[0].ff.layers[0]
SequentialRNN(
(0): TransformerXL(
(encoder): Embedding(60004, 410)
(pos_enc): PositionalEncoding()
(drop_emb): Dropout(p=0.03)
(layers): ModuleList(
(0): DecoderLayer(
(mhra): MultiHeadRelativeAttention(
(attention): Linear(in_features=410, out_features=1230, bias=False)
(out): Linear(in_features=410, out_features=410, bias=False)
(drop_att): Dropout(p=0.03)
(drop_res): Dropout(p=0.03)
(ln): LayerNorm(torch.Size([410]), eps=1e-05, elementwise_affine=True)
(r_attn): Linear(in_features=410, out_features=410, bias=False)
)
(ff): SequentialEx(
(layers): ModuleList(
(0): Linear(in_features=410, out_features=2100, bias=True)
(1): ReLU(inplace)
(2): Dropout(p=0.03)
(3): Linear(in_features=2100, out_features=410, bias=True)
(4): Dropout(p=0.03)
(5): MergeLayer()
(6): LayerNorm(torch.Size([410]), eps=1e-05, elementwise_affine=True)
)
)
)