0

我在“logName”中发现了很多与 GCP Dataflow 作业相关的奇怪错误日志：“...shuffler”和“...shuffler-startup”

数据流作业没有任何问题，并且“logName”中没有错误：“...worker”

以下是“logName”中带有 INFO 级别的完整错误行：“...shuffler-startup”

E0627 19:09:03.584041 487 curl_http_transport.cc:411] Call to url=<SUBSCRIPTION_NAME>:pull failed with curl status UNKNOWN: . curl=16

E0624 12:56:32.971646 195 curl_http_transport.cc:411] Call to url=<JOB_ID>/workItems:reportStatus failed with curl status UNKNOWN: . curl=16

E0613 09:23:08.731521 205 curl_http_transport.cc:410] Call to url=<JOB_ID>/workItems:reportStatus failed with curl status UNKNOWN: . curl=65: necessary data rewind wasn't possible

E0616 17:11:53.604693 481 curl_http_transport.cc:410] Call to url=<SUBSCRIPTION_NAME>:pull failed with curl status DEADLINE_EXCEEDED: Timed out. curl=28: Operation timed out after 120000 milliseconds with 0 bytes received

E0529 02:48:14.860606 55 curl_http_transport.cc:410] Call to url=<SUBSCRIPTION_NAME>:pull failed with curl status UNKNOWN: . curl=56: BoringSSL SSL_read: SSL_ERROR_SYSCALL, errno 104

E0627 19:51:37.833993 38041 curl_http_transport.cc:411] Call to url=<SUBSCRIPTION_NAME>:acknowledge failed with curl status UNKNOWN: . curl=16

和

以下是“logName”中具有 ERROR 级别的错误行示例：“...shuffler”

Call to url=<SUBSCRIPTION_NAME>:pull failed with curl status UNKNOWN: . curl=56: BoringSSL SSL_read: Unknown error 104, errno 104

Call to url=<SUBSCRIPTION_NAME>:pull failed with curl status UNKNOWN: . curl=65: necessary data rewind wasn't possible

Call to url=<SUBSCRIPTION_NAME>:pull failed with curl status UNKNOWN: . curl=16

4

1 回答 1

1

Apache Beam 和大多数分布式数据处理库和引擎有时可能会触发错误。这是因为这些系统很复杂，涉及数百台机器和数千个线程（或更多！）。

如果您的管道成功，那么您不必担心。这些引擎和库旨在承受这些错误并从中恢复。

此特定错误是 Dataflow 服务的临时问题。

如果这导致您的管道失败，请立即尝试再次运行它，因为该问题现在应该已经得到缓解。

于 2020-06-29T23:23:50.720 回答