1

我正在尝试从 Bluemix Spark 上的 iPython 笔记本作为服务连接到 BigInsights on Cloud 集群上的 BigSQL。

我的笔记本是这样的：

import os
cwd = os.getcwd()

with open('certificate', 'w') as f:
    f.write('''-----BEGIN CERTIFICATE-----
XXXXXXXXXXXXXXX
-----END CERTIFICATE-----''')

!rm truststore.jks
!keytool -import -trustcacerts -alias biginsights -file certificate -keystore truststore.jks -storepass mypassword -noprompt

host = 'ehaasp-xxxxx-master-2.bi.services.bluemix.net'
user = 'biadmin'
pwd  = 'xxxxxxx'
url  = 'jdbc:db2://{0}:51000/bigsql:user={1};password={2};sslConnection=true;sslTrustStoreLocation={3}/truststore.jks;Password=mypassword;'.format(host, user, pwd, cwd)

df = sqlContext.read.format('jdbc').options(url=url,
         driver='com.ibm.db2.jcc.DB2Driver', dbtable='GOSALESDW.SLS_SALES_FACT').load()

我收到的例外是：

Py4JJavaError: An error occurred while calling o111.load.
: com.ibm.db2.jcc.am.DisconnectNonTransientConnectionException: [jcc][t4][2030][11211][4.19.26] A communication error occurred during operations on the connection's underlying socket, socket input stream, 
or socket output stream.  Error location: Reply.fill() - socketInputStream.read (-1).  Message: RSA premaster secret error. ERRORCODE=-4499, SQLSTATE=08001
    at com.ibm.db2.jcc.am.kd.a(Unknown Source)
    at com.ibm.db2.jcc.t4.a.a(Unknown Source)
    at com.ibm.db2.jcc.t4.a.a(Unknown Source)
    at com.ibm.db2.jcc.t4.a.a(Unknown Source)
    at com.ibm.db2.jcc.t4.a.b(Unknown Source)
    at com.ibm.db2.jcc.t4.a.a(Unknown Source)
    at com.ibm.db2.jcc.t4.a.f(Unknown Source)
    at com.ibm.db2.jcc.t4.a.flush_(Unknown Source)
    at com.ibm.db2.jcc.am.Agent.flowOutsideUOW(Unknown Source)
    at com.ibm.db2.jcc.t4.b.b(Unknown Source)
    at com.ibm.db2.jcc.t4.b.b(Unknown Source)
    at com.ibm.db2.jcc.t4.b.a(Unknown Source)
    at com.ibm.db2.jcc.t4.b.a(Unknown Source)
    at com.ibm.db2.jcc.t4.b.a(Unknown Source)
    at com.ibm.db2.jcc.t4.b.<init>(Unknown Source)
    at com.ibm.db2.jcc.DB2SimpleDataSource.getConnection(Unknown Source)
    at com.ibm.db2.jcc.DB2SimpleDataSource.getConnection(Unknown Source)
    at com.ibm.db2.jcc.DB2Driver.connect(Unknown Source)
    at com.ibm.db2.jcc.DB2Driver.connect(Unknown Source)
    at java.sql.DriverManager.getConnection(DriverManager.java:583)
    at java.sql.DriverManager.getConnection(DriverManager.java:199)
    at org.apache.spark.sql.jdbc.JDBCRDD$.resolveTable(JDBCRDD.scala:118)
    at org.apache.spark.sql.jdbc.JDBCRelation.<init>(JDBCRelation.scala:128)
    at org.apache.spark.sql.jdbc.DefaultSource.createRelation(JDBCRelation.scala:113)
    at org.apache.spark.sql.sources.ResolvedDataSource$.apply(ddl.scala:269)
    at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:114)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:95)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:56)
    at java.lang.reflect.Method.invoke(Method.java:620)
    at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:231)
    at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:379)
    at py4j.Gateway.invoke(Gateway.java:259)
    at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:133)
    at py4j.commands.CallCommand.execute(CallCommand.java:79)
    at py4j.GatewayConnection.run(GatewayConnection.java:207)
    at java.lang.Thread.run(Thread.java:801)
Caused by: javax.net.ssl.SSLKeyException: RSA premaster secret error
    at com.ibm.jsse2.z.<init>(z.java:85)
    at com.ibm.jsse2.B.a(B.java:20)
    at com.ibm.jsse2.B.a(B.java:275)
    at com.ibm.jsse2.A.t(A.java:225)
    at com.ibm.jsse2.A.a(A.java:452)
    at com.ibm.jsse2.aq.a(aq.java:30)
    at com.ibm.jsse2.aq.h(aq.java:790)
    at com.ibm.jsse2.aq.a(aq.java:371)
    at com.ibm.jsse2.h.write(h.java:20)
    at com.ibm.db2.jcc.t4.eb.b(Unknown Source)
    at com.ibm.db2.jcc.t4.eb.a(Unknown Source)
    ... 31 more
Caused by: java.security.InvalidKeyException: Illegal key size or default parameters
    at javax.crypto.Cipher.a(Unknown Source)
    at javax.crypto.Cipher.a(Unknown Source)
    at javax.crypto.Cipher.a(Unknown Source)
    at javax.crypto.Cipher.init(Unknown Source)
    at com.ibm.jsse2.z.<init>(z.java:122)
    ... 41 more

尝试使用Hive 2.0 jdbc连接时出现类似错误。

4

2 回答 2

1

我看到根据您找到的链接中的说明，需要在 java 目录下的 spark 服务上安装无限制权限策略文件。在 bluemix 用户的 spark 服务租户下，没有创建/浏览 java 目录的权限。（锁定）

n [4]:
!echo $JAVA_HOME
/usr/local/src/spark160master/ibm-java-x86_64-80
In [5]:
!ls $JAVA_HOME/Security/
ls: cannot access /usr/local/src/spark160master/ibm-java-x86_64-80/Security/: No such file or directory
In [6]:
!touch $JAVA_HOME/Security/policydummy.txt
touch: cannot touch '/usr/local/src/spark160master/ibm-java-x86_64-80/Security/policydummy.txt': No such file or directory

这已被视为一项要求，并将进行审查以进行进一步调查。

谢谢，查尔斯。

于 2016-06-13T23:02:29.643 回答
0

现在已解决此问题。

有关示例笔记本，请参见此处：

在此处输入图像描述

该笔记本可在 GitHub 中找到

于 2016-06-27T19:36:30.180 回答