1

我正在尝试通过在本地运行火花流作业来访问事件中心数据。我在为eventhubs.checkpoint.dir. 我尝试设置低于值

  • wasbs://container_name@storage_name.blob.core.windows.net/
  • https://container_name@storage_name.blob.core.windows.net/
  • https://storage_name.blob.core.windows.net/continer_name/

每个都导致与以下类似的错误：

ERROR ReceiverTracker: Deregistered receiver for stream 0: Restarting receiver with delay 2000ms: Error handling message; restarting receiver -   java.io.IOException: No FileSystem for scheme: https
    at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2421)
    at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2428)
    at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:88)
    at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2467)
    at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2449)
    at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:367)
4

1 回答 1

1

您可以将 eventthubs.checkpoint.dir 设置为一个字符串值，该值将是一个有效的 wasb 文件夹名称。例如，我将其设置为“/myeventhubspark”。该文件夹将在 Spark 集群的默认容器中自动创建。请务必在文件夹名称前加上正斜杠，如下所示 -

“eventhubs.checkpoint.dir”->“/myeventhubspark”

于 2015-09-09T23:11:40.507 回答