2

我在这里关注 seq2seq 教程。

我想使用预训练的向量。我已经编辑了代码以获取单词的向量而不是索引。以下是代码：

#This piece of code loads the vectors from a json file {'word':[vector]..}
class Lang:
    def __init__(self, name, savedVectorsFile):
        def getSavedVectors(filename):
            import json
            word2vec = {}
            with open(filename) as json_data:
                word2vec = json.load(json_data)
            return word2vec

        self.name = name
        self.word2vector = {}
        self.word2count = {}
        self.index2word = {0: "SOS", 1: "EOS"}
        self.n_words = 2  # Count SOS and EOS

        self.get_saved_vector = getSavedVectors(savedVectorsFile)
        self.word2vector['unknown'] = self.get_saved_vector['unknown']

    def addSentence(self, sentence):
        for word in sentence.split(' '):
            self.addWord(word)

    def addWord(self, word):
        if word not in self.word2vector:
            self.word2vector[word] = self.get_saved_vector[word]
            self.word2count[word] = 1
            self.index2word[self.n_words] = word
            self.n_words += 1
        else:
            self.word2count[word] += 1




# this piece deals with returning a vector of given word, all vectors are just concatenated into one giant vector
def vectorFromSentence(lang, sentence):
    vectors = []
    for word in sentence.split(' '):
        if word not in lang.word2vector:
            vectors += (lang.word2vector["unknown"])
        else:
            vectors += (lang.word2vector[word])
    return vectors



# in the train method, I am passing  a vector instead of index to the encoder
    for ei in range(0, input_length, VEC_SIZE):
        encoder_output, encoder_hidden = encoder(input_variable[ei*VEC_SIZE:(ei+1)*VEC_SIZE], encoder_hidden)
        encoder_outputs[ei] = encoder_output[0][0]

现在，我不知道应该如何更改我的编码器以合并向量而不是索引。这是我现在的编码器：

def __init__(self, input_size, hidden_size, n_layers=1):
    super(EncoderRNN, self).__init__()
    self.n_layers = n_layers
    self.hidden_size = hidden_size

    self.embedding = nn.Embedding(input_size, hidden_size)
    self.gru = nn.GRU(hidden_size, hidden_size)

我得到这个错误

TypeError: torch.index_select received an invalid combination of arguments - got (torch.cuda.FloatTensor, int, torch.cuda.FloatTensor), but expected (torch.cuda.FloatTensor source, int dim, torch.cuda.LongTensor index)

我也尝试使用 VEC_SIZE 而不是 input_size，但无济于事。

以下是跟踪：

Traceback (most recent call last):
  File "scapula_generation_pretrained_vectors.py", line 710, in <module>
    trainIters(encoder1, attn_decoder1, 50000, print_every=1000)
  File "scapula_generation_pretrained_vectors.py", line 566, in trainIters
    decoder, encoder_optimizer, decoder_optimizer, criterion)
  File "scapula_generation_pretrained_vectors.py", line 472, in train
    encoder_output, encoder_hidden = encoder(input_variable[ei*VEC_SIZE:(ei+1)*VEC_SIZE], encoder_hidden)
  File "/home/sagar/anaconda3/lib/python3.6/site-packages/torch/nn/modules/module.py", line 224, in __call__
    result = self.forward(*input, **kwargs)
  File "scapula_generation_pretrained_vectors.py", line 214, in forward
    embedded = self.embedding(input).view(1, 1, -1)
  File "/home/sagar/anaconda3/lib/python3.6/site-packages/torch/nn/modules/module.py", line 224, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/sagar/anaconda3/lib/python3.6/site-packages/torch/nn/modules/sparse.py", line 94, in forward
    self.scale_grad_by_freq, self.sparse
  File "/home/sagar/anaconda3/lib/python3.6/site-packages/torch/nn/_functions/thnn/sparse.py", line 53, in forward
    output = torch.index_select(weight, 0, indices.view(-1))

如何编写编码器和解码器来占用 word2vec 嵌入？涉及到几件事，比如我的编码器输出gru是否必须是一个大小的向量VEC_SIZE，在解码器中 - 我的损失必须使用一些相似性度量来计算。我想我会选择余弦相似度，但在此之前我必须确保解码器获取编码器的输出并生成一个大小为 的向量VEC_SIZE。

如果有人已经在教程中完成了这项作业并且有现成的代码，我将不胜感激？

4

0 回答 0