0

我正在运行一个 3 节点 Spark 2.1.1 独立集群,spark.authenticate=true并且 Livy 服务器无法连接到它。这是我的一部分 livy.conf

livy.spark.authenticate = true
livy.spark.master = spark://HOSTNAME:25000
livy.spark.authenticate.secret = SECRET

在日志中我可以看到由于某种原因安全性被声明为DISABLED ,我猜这会导致这个错误跟踪......

INFO  SecurityManager:54 - SecurityManager: authentication disabled; ui acls disabled; users  with view permissions: Set(vva); groups with view permissions: Set(); users  with modify permissions: Set(vva); groups with modify permissions: Set()
INFO StandaloneAppClient$ClientEndpoint: Connecting to master spark://x.x.xxx.xx:25000...
INFO TransportClientFactory: Successfully created connection to /x.x.xxx.xx:25000 after 9 ms (0 ms spent in bootstraps)
WARN StandaloneAppClient$ClientEndpoint: Failed to connect to master x.x.xxx.xx:25000
org.apache.spark.SparkException: Exception thrown in awaitResult:
.........
Caused by: java.lang.RuntimeException: java.lang.IllegalStateException: Expected SaslMessage, received something else (maybe your client does not have SASL enabled?)

PS。我已经在单节点环境中进行了相同的测试,并且似乎工作正常。

谢谢。

4

1 回答 1

0

发现有些 spark 参数不能livy.confspark-defaults.conf.

于 2017-11-08T08:10:00.773 回答