tensorflow - 如何在 Tensorflow 中使用词嵌入进行预测

Question

我正在尝试完成 Tensorflow 教程，并且在尝试增强RNN/语言模型教程时遇到了困难，以便我可以预测句子中的下一个单词。本教程使用词嵌入作为词的表示。

由于模型在词嵌入上学习，我假设我添加的任何类型的预测都会输出相同的嵌入。我不知道如何将这些嵌入转换回数据集中的单词 id。我见过的唯一示例保留了与 wordid 映射相反的内存数据结构 -> 嵌入并将其用于查找。这显然不适用于所有问题。有没有更好的办法？

Answer 1

假设您同时拥有word_to_idx词汇idx_to_word表，这是您所做的伪代码

想象一下预测的输入是“这是样本”

batch_size = 1
num_steps = 3 # i.e each step for the word in "this is sample"
hidden_size = 1500
vocab_size = 10000
translate the `word_to_idx` for the input `"this is sample"`
get the word embeddings for the each word in the input
Input to model will be word embedding of size 1x1500 at each time step
Output of model at each time step will be of size 1x1500
y_pred is output at last step from model for the given input 
adding projection to output (i.e y_pred x Weights(hidden_size, vocab_size) + bias(vocab_size, )  = 1x10000)
now sample the output through the below function to get the index with max probability
generate idx_to_word from the index we just got
use the generated word along with the previous input to generate next word until you get `<eos>` or some predefined sentence stopping.

这是从此处采样的示例：

def sample(a, temperature=1.0):
    # helper function to sample an index from a probability array
    a = np.log(a) / temperature
    a = np.exp(a) / np.sum(np.exp(a))
    return np.argmax(np.random.multinomial(1, a, 1))