0

使用这个 jar 列表，我可以成功地将 SQuirrel SQL 连接到 Spark 2.2.1：

commons-logging-1.1.3.jar
hadoop-common-2.7.3.jar
hive-exec-1.2.1.spark2.jar
hive-jdbc-1.2.1.spark2.jar
hive-metastore-1.2.1.spark2.jar
http-client-1.0.4.jar
httpclient-4.5.2.jar
httpcore-4.4.4.jar
libfb303-0.9.3.jar
libthrift-0.9.3.jar
log4j-1.2.17.jar
slf4j-api-1.7.16.jar
slf4j-log4j12-1.7.16.jar
spark-hive-thriftserver_2.11-2.2.1.jar
spark-hive_2.11-2.2.1.jar
spark-network-common_2.11-2.2.1.jar

我认为上述罐子是不必要的。但是当尝试将 JMeter 3.3 连接到与他们相同的 Spark 2.2.1 ThriftServer 时，我收到以下错误消息

enter code here Cannot create PoolableConnectionFactory (Method not supported)

JDBC 配置在这里： 在此处输入图像描述

Jmeter 的完整回复在这里：

Thread Name: test 1-1
Sample Start: 2018-04-03 13:34:43 CST
Load time: 511
Connect Time: 510
Latency: 0
Size in bytes: 62
Sent bytes:0
Headers size in bytes: 0
Body size in bytes: 62
Sample Count: 1
Error Count: 1
Data type ("text"|"bin"|""): text
Response code: null 0
Response message: java.sql.SQLException: Cannot create PoolableConnectionFactory (Method not supported)

Response headers:


SampleResult fields:
ContentType: text/plain
DataEncoding: UTF-8

我也尝试使用较新的 Hive JDBC driver 2.3.0，但显然它不适用于 Spark 2.2.1beeline或包括 Jmeter 在内的任何其他版本。

将 Beeline 与 Hive JDBC 驱动程序 2.3.0 一起使用时的错误消息如下：

$ beeline -u jdbc:hive2://<hostip>:10000/tpch_sf100_orc -n rxxxds          
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/apache-hive-2.3.0-bin/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/apache-tez-0.9.0-bin/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/hadoop-2.9.0/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Connecting to jdbc:hive2://<hostip>:10000/tpch_sf100_orc
18/04/03 13:41:58 [main]: ERROR jdbc.HiveConnection: Error opening session
org.apache.thrift.TApplicationException: Required field 'client_protocol' is unset! Struct:TOpenSessionReq(client_protocol:null, configuration:{set:hiveconf:hive.server2.thrift.resultset.default.fetch.size=1000, use:database=tpch_sf100_orc})
        at org.apache.thrift.TApplicationException.read(TApplicationException.java:111) ~[hive-exec-2.3.0.jar:2.3.0]
        at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:79) ~[hive-exec-2.3.0.jar:2.3.0]
        at org.apache.hive.service.rpc.thrift.TCLIService$Client.recv_OpenSession(TCLIService.java:168) ~[hive-exec-2.3.0.jar:2.3.0]
        at org.apache.hive.service.rpc.thrift.TCLIService$Client.OpenSession(TCLIService.java:155) ~[hive-exec-2.3.0.jar:2.3.0]
        at org.apache.hive.jdbc.HiveConnection.openSession(HiveConnection.java:680) [hive-jdbc-2.3.0.jar:2.3.0]
        at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:200) [hive-jdbc-2.3.0.jar:2.3.0]
        at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:107) [hive-jdbc-2.3.0.jar:2.3.0]
        at java.sql.DriverManager.getConnection(DriverManager.java:664) [?:1.8.0_112]
        at java.sql.DriverManager.getConnection(DriverManager.java:208) [?:1.8.0_112]
        at org.apache.hive.beeline.DatabaseConnection.connect(DatabaseConnection.java:145) [hive-beeline-2.3.0.jar:2.3.0]
        at org.apache.hive.beeline.DatabaseConnection.getConnection(DatabaseConnection.java:209) [hive-beeline-2.3.0.jar:2.3.0]
        at org.apache.hive.beeline.Commands.connect(Commands.java:1641) [hive-beeline-2.3.0.jar:2.3.0]
        at org.apache.hive.beeline.Commands.connect(Commands.java:1536) [hive-beeline-2.3.0.jar:2.3.0]
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_112]
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_112]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_112]
        at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_112]
        at org.apache.hive.beeline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:56) [hive-beeline-2.3.0.jar:2.3.0]
        at org.apache.hive.beeline.BeeLine.execCommandWithPrefix(BeeLine.java:1274) [hive-beeline-2.3.0.jar:2.3.0]
        at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:1313) [hive-beeline-2.3.0.jar:2.3.0]
        at org.apache.hive.beeline.BeeLine.connectUsingArgs(BeeLine.java:867) [hive-beeline-2.3.0.jar:2.3.0]
        at org.apache.hive.beeline.BeeLine.initArgs(BeeLine.java:776) [hive-beeline-2.3.0.jar:2.3.0]
        at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:1010) [hive-beeline-2.3.0.jar:2.3.0]
        at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:519) [hive-beeline-2.3.0.jar:2.3.0]
        at org.apache.hive.beeline.BeeLine.main(BeeLine.java:501) [hive-beeline-2.3.0.jar:2.3.0]
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_112]
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_112]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_112]
        at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_112]
        at org.apache.hadoop.util.RunJar.run(RunJar.java:239) [hadoop-common-2.9.0.jar:?]
        at org.apache.hadoop.util.RunJar.main(RunJar.java:153) [hadoop-common-2.9.0.jar:?]
18/04/03 13:41:58 [main]: WARN jdbc.HiveConnection: Failed to connect to <hostip>:10000
Error: Could not open client transport with JDBC Uri: jdbc:hive2://<hostip>:10000/tpch_sf100_orc: Could not establish connection to jdbc:hive2://<hostip>:10000/tpch_sf100_orc: Required field 'client_protocol' is unset! Struct:TOpenSessionReq(client_protocol:null, configuration:{set:hiveconf:hive.server2.thrift.resultset.default.fetch.size=1000, use:database=tpch_sf100_orc}) (state=08S01,code=0)
Beeline version 2.3.0 by Apache Hive

还有什么可以将 JMeter 连接到 Spark？

4

2 回答 2

0

您还可以构建自己的“JDBC 采样器”并从原始代码中添加/调整几行代码以使其正常工作。4 年前，我不得不为 Hive 和 Phoenix 支持这样做，并且它奏效了。它需要一个当时没有实现的 Timeout 方法。

以下是来源： https ://github.com/csalperwyck/JMeterJDBCSamplerWithOutTimeOut

于 2018-08-24T12:35:25.570 回答
0

这很可能是由于客户端和服务器库不匹配，您需要使用与服务器上相同的版本。

因此，确定您的服务器上正在运行哪个版本的 hive 并下载适当hive-jdbc的 jar 和所有依赖项（您可以使用Maven Dependency Plugin检索它们）并使用正确的版本更新JMeter Classpath中的那些。

最好使用“干净”的 JMeter 安装来避免可能的 jar 地狱，因此现在是升级到JMeter 4.0的好时机

于 2018-04-03T08:31:59.610 回答