2

问题设置

我正在关注本教程。本教程首先加载一个监督数据集（使用tfds.loadwith as_supervised=True）：

(train_data, test_data), info = tfds.load(
    'imdb_reviews/subwords8k', 
    split = (tfds.Split.TRAIN, tfds.Split.TEST), 
    with_info=True, as_supervised=True)

然后本教程建议像这样洗牌和填充数据集：

电影评论可以有不同的长度。我们将使用该padded_batch方法来标准化评论的长度。

train_batches = train_data.shuffle(1000).padded_batch(10)
test_batches = test_data.shuffle(1000).padded_batch(10)

...但不幸的是，该padded_batch方法需要一个额外的参数，该教程似乎已经忘记了：

Traceback (most recent call last):
  File "imdb_reviews.py", line 14, in <module>
    train_batches = train_data.shuffle(1000).padded_batch(10)
TypeError: padded_batch() missing 1 required positional argument: 'padded_shapes'

重要假设

尽管错误堆栈说这padded_shapes是缺少的参数，但从教程中我认为推断缺少的参数实际上是公平的batch_size（应该先于padded_shapes）。

我试过的

我认为这可能很容易解决，如下所示：

batch_sz = 100 # arbitrary number
train_batches = train_data.shuffle(1000).padded_batch(batch_sz, ([10],[None]))
test_batches = test_data.shuffle(1000).padded_batch(batch_sz, ([10],[None]))

...但我的解决方案显然是错误的：

Traceback (most recent call last):
  File "imdb_reviews.py", line 15, in <module>
    train_batches = train_data.shuffle(1000).padded_batch(batch_sz, ([10],[None]))
  File "/home/ggiuffre/.local/lib/python3.7/site-packages/tensorflow_core/python/data/ops/dataset_ops.py", line 2298, in padded_batch
    batch_size, padded_shapes, padding_values, drop_remainder))
  File "/home/ggiuffre/.local/lib/python3.7/site-packages/tensorflow_core/python/data/ops/dataset_ops.py", line 1481, in padded_batch
    drop_remainder)
  File "/home/ggiuffre/.local/lib/python3.7/site-packages/tensorflow_core/python/data/ops/dataset_ops.py", line 3813, in __init__
    _padded_shape_to_tensor(padded_shape, input_component_shape))
  File "/home/ggiuffre/.local/lib/python3.7/site-packages/tensorflow_core/python/data/ops/dataset_ops.py", line 3741, in _padded_shape_to_tensor
    % (padded_shape_as_shape, input_component_shape))
ValueError: The padded shape (None,) is not compatible with the corresponding input component shape ().

替换None给()我ValueError: Padded shape [()] must be a 1-D tensor of tf.int64 values, but its shape was (1, 0).

替换None给1我ValueError: The padded shape (1,) is not compatible with the corresponding input component shape ().

问题

我应该赋予这个padded_shapes论点什么价值？或者，更一般地说，我在这里做错了什么？

非常感谢您的帮助。

4

2 回答 2

5

看看这个博客。https://medium.com/@a.ydobon/tensorflow-2-0-word-embeddings-part3-964b2b9caf94

它建议

padded_shapes = ([None],())
train_batches = train_data.shuffle(1000).padded_batch(10,padded_shapes=padded_shapes)
test_batches = test_data.shuffle(1000).padded_batch(10,padded_shapes=padded_shapes)

这对我有用。

于 2020-02-14T16:33:45.683 回答
1

这也很好用

batch_sz = 100 # arbitrary number
train_batches = train_data.shuffle(1000)
test_batches = test_data.shuffle(1000)

train_batches = train_batches.padded_batch(batch_sz, train_batches.output_shapes)
test_batches = test_batches.padded_batch(batch_sz, test_batches.output_shapes)

比你可以定义你的 LSTM 层和适合 train_batches ，test_batches

于 2020-04-14T15:10:48.150 回答