3

我在 google 的 colab 上写了一个 jupyter-notebook 来微调（用于文本分类）一个我已经在阿拉伯语上预训练过的 BERT 版本。当训练开始时，我无法绕过这个错误。

我在github上按照google给出的notebook

模型构建代码：

model_fn = model_fn_builder(
  bert_config=modeling.BertConfig.from_json_file(CONFIG_FILE),
  num_labels=len(label_list),
  init_checkpoint=INIT_CHECKPOINT,
  learning_rate=LEARNING_RATE,
  num_train_steps=num_train_steps,
  num_warmup_steps=num_warmup_steps,
  use_tpu=True,
  use_one_hot_embeddings=True
)


tpu_cluster_resolver = tf.contrib.cluster_resolver.TPUClusterResolver(TPU_ADDRESS)

run_config = tf.contrib.tpu.RunConfig(
    cluster=tpu_cluster_resolver,
    model_dir=OUTPUT_DIR,
    save_checkpoints_steps=SAVE_CHECKPOINTS_STEPS,
    tpu_config=tf.contrib.tpu.TPUConfig(
        iterations_per_loop=ITERATIONS_PER_LOOP,
        num_shards=NUM_TPU_CORES,
        per_host_input_for_training=tf.contrib.tpu.InputPipelineConfig.PER_HOST_V2))

estimator = tf.contrib.tpu.TPUEstimator(
    use_tpu=USE_TPU,
    model_fn=model_fn,
    config=run_config,
    train_batch_size=TRAIN_BATCH_SIZE,
    eval_batch_size=EVAL_BATCH_SIZE,
    predict_batch_size=PREDICT_BATCH_SIZE,)

train_input_fn = input_fn_builder(
    features=train_features,
    seq_length=MAX_SEQ_LENGTH,
    is_training=True,
    drop_remainder=False)

#tf.reset_default_graph()
print(f'Beginning Training!')
current_time = datetime.now()
estimator.train(input_fn=train_input_fn, max_steps=TRAIN_STEPS)
print("Training took time ", datetime.now() - current_time)

错误代码：

/usr/local/lib/python3.6/dist-packages/tensorflow/python/tpu/tpu_sharding.py in _unshard_shape(self, shape)
    214                        (shape.as_list(), self._shard_dimension))
    215     dims = shape.as_list()
--> 216     dims[self._shard_dimension] *= self._number_of_shards
    217     return tensor_shape.as_shape(dims)
    218 

TypeError: unsupported operand type(s) for *=: 'NoneType' and 'int'

参数和其余代码都在这个 colab 笔记本的共享副本中：colab_link

4

1 回答 1

1

为了社区的利益，在本节中提及答案（即使在评论部分中已回答）。

在函数中设置参数，drop_remainder解决了这个问题。Trueinput_fn_builder

相应的代码片段如下所示：

train_input_fn = input_fn_builder(
    features=train_features,
    seq_length=MAX_SEQ_LENGTH,
    is_training=True,
    drop_remainder=False)
于 2019-12-30T08:07:37.657 回答