0

这是日志中唯一的错误异常,所有 Dataflow 工作人员在处理 3.5 天后关闭。它通过了一半以上的负载。这个错误是什么意思?不确定这是否是增加资源后可能会解决的内存问题。用户代码不会引起异常,因为所有内容都在一个毯子 try...except 块中。

Workflow failed. Causes: S04:Reshuffle/ReshufflePerKey/GroupByKey/Read+Reshuffle/ReshufflePerKey/GroupByKey/GroupByWindow+Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)+Reshuffle/RemoveRandomKeys+ParDo(EnrichCompanies)+ParDo(LogCompanyPipelineRun) failed., The job failed because a work item has failed 4 times. Look in previous log entries for the cause of each one of the 4 failures. For more information, see https://cloud.google.com/dataflow/docs/guides/common-errors. The work item was attempted on these workers: 

  company-batch-enrichment--02161750-u9wk-harness-3nj5
      Root cause: The worker lost contact with the service.,

  company-batch-enrichment--02161750-u9wk-harness-3nj5
      Root cause: The worker lost contact with the service.,

  company-batch-enrichment--02161750-u9wk-harness-3nj5
      Root cause: The worker lost contact with the service.,

  company-batch-enrichment--02161750-u9wk-harness-3nj5
      Root cause: The worker lost contact with the service.

以下是作业的资源指标。 在此处输入图像描述

4

0 回答 0