tf.contrib.crf似乎不支持长度为 1 的序列。
例如，如果我在https://github.com/tensorflow/tensorflow/tree/master/tensorflow/contrib/crf ( mirror ) 上运行示例并替换num_words = 20为num_words = 1：
import numpy as np
import tensorflow as tf
# Data settings.
num_examples = 10
num_words = 1
num_features = 100
num_tags = 5
# Random features.
x = np.random.rand(num_examples, num_words, num_features).astype(np.float32)
# Random tag indices representing the gold sequence.
y = np.random.randint(num_tags, size=[num_examples, num_words]).astype(np.int32)
# All sequences in this example have the same length, but they can be variable in a real model.
sequence_lengths = np.full(num_examples, num_words - 1, dtype=np.int32)
# Train and evaluate the model.
with tf.Graph().as_default():
with tf.Session() as session:
# Add the data to the TensorFlow graph.
x_t = tf.constant(x)
y_t = tf.constant(y)
sequence_lengths_t = tf.constant(sequence_lengths)
# Compute unary scores from a linear layer.
weights = tf.get_variable("weights", [num_features, num_tags])
matricized_x_t = tf.reshape(x_t, [-1, num_features])
matricized_unary_scores = tf.matmul(matricized_x_t, weights)
unary_scores = tf.reshape(matricized_unary_scores,
[num_examples, num_words, num_tags])
# Compute the log-likelihood of the gold sequences and keep the transition
# params for inference at test time.
log_likelihood, transition_params = tf.contrib.crf.crf_log_likelihood(
unary_scores, y_t, sequence_lengths_t)
# Add a training op to tune the parameters.
loss = tf.reduce_mean(-log_likelihood)
train_op = tf.train.GradientDescentOptimizer(0.01).minimize(loss)
# Train for a fixed number of iterations.
session.run(tf.global_variables_initializer())
for i in range(1000):
tf_unary_scores, tf_transition_params, _ = session.run(
[unary_scores, transition_params, train_op])
if i % 100 == 0:
correct_labels = 0
total_labels = 0
for tf_unary_scores_, y_, sequence_length_ in zip(tf_unary_scores, y,
sequence_lengths):
# Remove padding from the scores and tag sequence.
tf_unary_scores_ = tf_unary_scores_[:sequence_length_]
y_ = y_[:sequence_length_]
# Compute the highest scoring sequence.
viterbi_sequence, _ = tf.contrib.crf.viterbi_decode(
tf_unary_scores_, tf_transition_params)
# Evaluate word-level accuracy.
correct_labels += np.sum(np.equal(viterbi_sequence, y_))
total_labels += sequence_length_
accuracy = 100.0 * correct_labels / float(total_labels)
print("Accuracy: %.2f%%" % accuracy)
我收到错误消息：
Traceback (most recent call last):
File "C:\Anaconda\envs\py35\lib\site-packages\tensorflow\python\client\session.py", line 1022, in _do_call
return fn(*args)
File "C:\Anaconda\envs\py35\lib\site-packages\tensorflow\python\client\session.py", line 1004, in _run_fn
status, run_metadata)
File "C:\Anaconda\envs\py35\lib\contextlib.py", line 66, in __exit__
next(self.gen)
File "C:\Anaconda\envs\py35\lib\site-packages\tensorflow\python\framework\errors_impl.py", line 469, in raise_exception_on_not_ok_status
pywrap_tensorflow.TF_GetCode(status))
tensorflow.python.framework.errors_impl.UnimplementedError: TensorArray has size zero, but element shape <unknown> is not fully defined. Currently only static shapes are supported when packing zero-size TensorArrays.
[[Node: gradients/rnn/TensorArrayUnstack/TensorArrayScatter/TensorArrayScatterV3_grad/TensorArrayGatherV3 = TensorArrayGatherV3[_class=["loc:@rnn/TensorArray_1"], dtype=DT_FLOAT, element_shape=<unknown>, _device="/job:localhost/replica:0/task:0/cpu:0"](gradients/rnn/TensorArrayUnstack/TensorArrayScatter/TensorArrayScatterV3_grad/TensorArrayGrad/TensorArrayGradV3, rnn/TensorArrayUnstack/range, gradients/rnn/TensorArrayUnstack/TensorArrayScatter/TensorArrayScatterV3_grad/TensorArrayGrad/gradient_flow)]]
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\Francky\Documents\GitHub\nlp\neurodeid\test\CRF_v2.py", line 47, in <module>
[unary_scores, transition_params, train_op])
File "C:\Anaconda\envs\py35\lib\site-packages\tensorflow\python\client\session.py", line 767, in run
run_metadata_ptr)
File "C:\Anaconda\envs\py35\lib\site-packages\tensorflow\python\client\session.py", line 965, in _run
feed_dict_string, options, run_metadata)
File "C:\Anaconda\envs\py35\lib\site-packages\tensorflow\python\client\session.py", line 1015, in _do_run
target_list, options, run_metadata)
File "C:\Anaconda\envs\py35\lib\site-packages\tensorflow\python\client\session.py", line 1035, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.UnimplementedError: TensorArray has size zero, but element shape <unknown> is not fully defined. Currently only static shapes are supported when packing zero-size TensorArrays.
[[Node: gradients/rnn/TensorArrayUnstack/TensorArrayScatter/TensorArrayScatterV3_grad/TensorArrayGatherV3 = TensorArrayGatherV3[_class=["loc:@rnn/TensorArray_1"], dtype=DT_FLOAT, element_shape=<unknown>, _device="/job:localhost/replica:0/task:0/cpu:0"](gradients/rnn/TensorArrayUnstack/TensorArrayScatter/TensorArrayScatterV3_grad/TensorArrayGrad/TensorArrayGradV3, rnn/TensorArrayUnstack/range, gradients/rnn/TensorArrayUnstack/TensorArrayScatter/TensorArrayScatterV3_grad/TensorArrayGrad/gradient_flow)]]
Caused by op 'gradients/rnn/TensorArrayUnstack/TensorArrayScatter/TensorArrayScatterV3_grad/TensorArrayGatherV3', defined at:
File "C:\Users\Francky\Documents\GitHub\nlp\neurodeid\test\CRF_v2.py", line 41, in <module>
train_op = tf.train.GradientDescentOptimizer(0.01).minimize(loss)
File "C:\Anaconda\envs\py35\lib\site-packages\tensorflow\python\training\optimizer.py", line 288, in minimize
grad_loss=grad_loss)
File "C:\Anaconda\envs\py35\lib\site-packages\tensorflow\python\training\optimizer.py", line 354, in compute_gradients
colocate_gradients_with_ops=colocate_gradients_with_ops)
File "C:\Anaconda\envs\py35\lib\site-packages\tensorflow\python\ops\gradients_impl.py", line 482, in gradients
in_grads = grad_fn(op, *out_grads)
File "C:\Anaconda\envs\py35\lib\site-packages\tensorflow\python\ops\tensor_array_grad.py", line 186, in _TensorArrayScatterGrad
grad = g.gather(indices)
File "C:\Anaconda\envs\py35\lib\site-packages\tensorflow\python\ops\tensor_array_ops.py", line 348, in gather
element_shape=element_shape)
File "C:\Anaconda\envs\py35\lib\site-packages\tensorflow\python\ops\gen_data_flow_ops.py", line 2226, in _tensor_array_gather_v3
element_shape=element_shape, name=name)
File "C:\Anaconda\envs\py35\lib\site-packages\tensorflow\python\framework\op_def_library.py", line 763, in apply_op
op_def=op_def)
File "C:\Anaconda\envs\py35\lib\site-packages\tensorflow\python\framework\ops.py", line 2395, in create_op
original_op=self._default_original_op, op_def=op_def)
File "C:\Anaconda\envs\py35\lib\site-packages\tensorflow\python\framework\ops.py", line 1264, in __init__
self._traceback = _extract_stack()
...which was originally created as op 'rnn/TensorArrayUnstack/TensorArrayScatter/TensorArrayScatterV3', defined at:
File "C:\Users\Francky\Documents\GitHub\nlp\neurodeid\test\CRF_v2.py", line 37, in <module>
unary_scores, y_t, sequence_lengths_t)
File "C:\Anaconda\envs\py35\lib\site-packages\tensorflow\contrib\crf\python\ops\crf.py", line 156, in crf_log_likelihood
log_norm = crf_log_norm(inputs, sequence_lengths, transition_params)
File "C:\Anaconda\envs\py35\lib\site-packages\tensorflow\contrib\crf\python\ops\crf.py", line 123, in crf_log_norm
dtype=dtypes.float32)
File "C:\Anaconda\envs\py35\lib\site-packages\tensorflow\python\ops\rnn.py", line 545, in dynamic_rnn
dtype=dtype)
File "C:\Anaconda\envs\py35\lib\site-packages\tensorflow\python\ops\rnn.py", line 663, in _dynamic_rnn_loop
for ta, input_ in zip(input_ta, flat_input))
File "C:\Anaconda\envs\py35\lib\site-packages\tensorflow\python\ops\rnn.py", line 663, in <genexpr>
for ta, input_ in zip(input_ta, flat_input))
File "C:\Anaconda\envs\py35\lib\site-packages\tensorflow\python\ops\tensor_array_ops.py", line 400, in unstack
indices=math_ops.range(0, num_elements), value=value, name=name)
File "C:\Anaconda\envs\py35\lib\site-packages\tensorflow\python\ops\tensor_array_ops.py", line 428, in scatter
name=name)
File "C:\Anaconda\envs\py35\lib\site-packages\tensorflow\python\ops\gen_data_flow_ops.py", line 2492, in _tensor_array_scatter_v3
name=name)
File "C:\Anaconda\envs\py35\lib\site-packages\tensorflow\python\framework\op_def_library.py", line 763, in apply_op
op_def=op_def)
UnimplementedError (see above for traceback): TensorArray has size zero, but element shape <unknown> is not fully defined. Currently only static shapes are supported when packing zero-size TensorArrays.
[[Node: gradients/rnn/TensorArrayUnstack/TensorArrayScatter/TensorArrayScatterV3_grad/TensorArrayGatherV3 = TensorArrayGatherV3[_class=["loc:@rnn/TensorArray_1"], dtype=DT_FLOAT, element_shape=<unknown>, _device="/job:localhost/replica:0/task:0/cpu:0"](gradients/rnn/TensorArrayUnstack/TensorArrayScatter/TensorArrayScatterV3_grad/TensorArrayGrad/TensorArrayGradV3, rnn/TensorArrayUnstack/range, gradients/rnn/TensorArrayUnstack/TensorArrayScatter/TensorArrayScatterV3_grad/TensorArrayGrad/gradient_flow)]]
num_words = 2或num_words = 5工作。我在
https://github.com/tensorflow/tensorflow/blob/e121667dc609de978a223c56ee906368d2c4ceef/tensorflow/contrib/crf/python/ops/crf.py#L121（mirror ）上看到已经将1递减sequence_length：
# Compute the alpha values in the forward algorithm in order to get the
# partition function.
forward_cell = CrfForwardRnnCell(transition_params)
_, alphas = rnn.dynamic_rnn(
cell=forward_cell,
inputs=rest_of_input,
sequence_length=sequence_lengths - 1,
initial_state=first_input,
dtype=dtypes.float32)
log_norm = math_ops.reduce_logsumexp(alphas, [1])
return log_norm
但是，更改sequence_lengths = np.full(num_examples, num_words - 1, dtype=np.int32)为 时sequence_lengths = np.full(num_examples, num_words, dtype=np.int32)并不能解决问题num_words = 1。
如何解决这个问题，使 CRF 层支持长度为 1 的序列？
在 Windows 7 SP1 x64 Ultimate 上使用 TensorFlow 1.0.0 和在 Ubuntu 14.04.4 LTS x64 上使用 TensorFlow-GPU 1.0.0 进行测试。我在 TensorFlow 存储库中创建了一个问题，但由于不活动而关闭：https ://github.com/tensorflow/tensorflow/issues/7751 （镜像）