6

在尝试复制Incorporating Discrete Translation Lexicons into Neural MT中的第 3.1 节时paddle-paddle

我试图有一个静态矩阵，我需要将其加载到seqToseq训练管道中，例如：

>>> import numpy as np
>>> x = np.random.rand(3,2)  
>>> x
array([[ 0.64077103,  0.03278357],
       [ 0.47133411,  0.16309775],
       [ 0.63986919,  0.07130613]])
# where there is 3 target words and 2 source words, 
# and each cell in the matrix represents some co-occurrence probabilities.

在seqToseq_net演示中，这个矩阵需要乘以gru_decoder_with_attention. 原始演示：

def gru_decoder_with_attention(enc_vec, enc_proj, current_word):
    decoder_mem = memory(name='gru_decoder',
                         size=decoder_size,
                         boot_layer=decoder_boot)

    # This attention context layer would have been 
    # a vector of size |src_vocab| x 1
    context = simple_attention(encoded_sequence=enc_vec,
                               encoded_proj=enc_proj,
                               decoder_state=decoder_mem, )

    with mixed_layer(size=decoder_size * 3) as decoder_inputs:
        decoder_inputs += full_matrix_projection(input=context)
        decoder_inputs += full_matrix_projection(input=current_word)

    gru_step = gru_step_layer(name='gru_decoder',
                              input=decoder_inputs,
                              output_mem=decoder_mem,
                              size=decoder_size)

    with mixed_layer(size=target_dict_dim,
                     bias_attr=True,
                     act=SoftmaxActivation()) as out:
        out += full_matrix_projection(input=gru_step)
    return out

目标是通过将注意力层与静态矩阵相乘来影响注意力层：

def gru_decoder_with_attention(enc_vec, enc_proj, current_word):
    decoder_mem = memory(name='gru_decoder',
                         size=decoder_size,
                         boot_layer=decoder_boot)

    # This attention context layer would have been 
    # of size |src_vocab| x 1
    context = simple_attention(encoded_sequence=enc_vec,
                               encoded_proj=enc_proj,
                               decoder_state=decoder_mem, )

    # This static matrix layer, x, would have been
    # of size |trg_vocab| x |src_vocab|
    static_matrix = some_sort_of_layer(x)

    # This should yield a vector of size
    # |trg_vocab| x 1
    static_matrix_multiply_context = some_sort_of_operation_layer( static_matrix, context)

    with mixed_layer(size=decoder_size * 3) as decoder_inputs:
        # 
        decoder_inputs += full_matrix_projection(input= static_matrix_multiply_context)
        decoder_inputs += full_matrix_projection(input=current_word)

我尝试过查看代码Paddle/python/trainer_config_helps并浏览所有演示代码，并且我还询问了PaddlePaddle 的 gitter。但是我找不到如何加载不需要在训练过程中更新并与 Paddle 层之一交互的自定义​​静态矩阵。

如何加载矩阵以更改 seqToseq 演示中的注意力层？

在上面的例子中some_sort_of_layer应该是什么？some_sort_of_operation_layer

4

0 回答 0