1

在 Azure 上，我在 ADLS (gen2) 上有数据库 (delta)，Databricks 可以访问它。

我正在使用 Simba JDBC 驱动程序连接到数据块并从数据库获取数据的客户端应用程序进行性能测试。当我连续加载大约 600 万条数据时，由于数据块的错误，该过程在 100 万条数据后中断。我检查了这次集群没有关闭，但我的客户端无法连接并抛出错误。

我的 Databricks 集群配置是：

工作器类型：Standard_DS4_v2 - 28 GB，8 核

驱动类型：Standard_DS5_v2 - 56 gb , 16 core

我还尝试将驱动程序类型增加到（驱动程序类型：Standard_DS5_v2 - 56 gb , 16 cores）但失败了

错误：

[Access(81040)00-ids.databricks] ERROR <date time> com.denodo.vdb.engine.wrapper.raw.jdbc.adapter.SQLPlugin [] - Error validating connection
java.sql.SQLException: [Simba][SparkJDBCDriver](500593) Communication link failure. Failed to connect to server. Reason: com.simba.spark.jdbc42.internal.apache.http.NoHttpResponseException: <Databricks URL> failed to respond.
        at com.simba.spark.hivecommon.api.HS2Client.handleTTransportException(Unknown Source) ~[SparkJDBC42.jar:?]
        at com.simba.spark.hivecommon.api.HS2Client.executeStatementInternal(Unknown Source) ~[SparkJDBC42.jar:?]
        at com.simba.spark.hivecommon.api.HS2Client.executeStatement(Unknown Source) ~[SparkJDBC42.jar:?]
        at com.simba.spark.hivecommon.dataengine.HiveJDBCNativeQueryExecutor.executeQuery(Unknown Source) ~[SparkJDBC42.jar:?]
        at com.simba.spark.hivecommon.dataengine.HiveJDBCNativeQueryExecutor.<init>(Unknown Source) ~[SparkJDBC42.jar:?]
        at com.simba.spark.hivecommon.dataengine.HiveJDBCDataEngine.prepare(Unknown Source) ~[SparkJDBC42.jar:?]
        at com.simba.spark.jdbc.common.SStatement.executeNoParams(Unknown Source) ~[SparkJDBC42.jar:?]
        at com.simba.spark.jdbc.common.SStatement.execute(Unknown Source) ~[SparkJDBC42.jar:?]
        at com.simba.spark.hivecommon.jdbc42.Hive42Statement.execute(Unknown Source) ~[SparkJDBC42.jar:?]
        at org.apache.commons.dbcp2.DelegatingStatement.execute(DelegatingStatement.java:194) ~[commons-dbcp2.jar:2.7.0]
        at com.denodo.vdb.engine.wrapper.raw.jdbc.adapter.SQLPlugin.executeValidationQuery(Unknown Source) [denodo-vdp-server.jar:8.0.0]
        at com.denodo.vdb.misc.connection.jdbc.JDBCConnectionFactory.a3(Unknown Source) [denodo-vdp-server.jar:8.0.0]
        at com.denodo.vdb.misc.connection.jdbc.JDBCConnectionFactory.validateConnection(Unknown Source) [denodo-vdp-server.jar:8.0.0]
        at org.apache.commons.dbcp2.PoolableConnectionFactory.validateObject(PoolableConnectionFactory.java:648) [commons-dbcp2.jar:2.7.0]
        at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:472) [commons-pool2.jar:2.7.0]
        at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:349) [commons-pool2.jar:2.7.0]
        at com.denodo.vdb.misc.connection.jdbc.JDBCConnectionPool.borrowObject(Unknown Source) [denodo-vdp-server.jar:8.0.0]
        at com.denodo.vdb.misc.connection.jdbc.JDBCConnectionPool.getCredentialedConnection(Unknown Source) [denodo-vdp-server.jar:8.0.0]
        at com.denodo.vdb.misc.datasource.credentialed.CredentialedDatasource.getCredentialedConnection(Unknown Source) [denodo-vdp-server.jar:8.0.0]
        at com.denodo.vdb.misc.datasource.JDBCDataSource.c3(Unknown Source) [denodo-vdp-server.jar:8.0.0]
        at com.denodo.vdb.misc.datasource.JDBCDataSource.getConnection(Unknown Source) [denodo-vdp-server.jar:8.0.0]
        at com.denodo.vdb.misc.datasource.JDBCXADataSource.getNonXAConnection(Unknown Source) [denodo-vdp-server.jar:8.0.0]
        at com.denodo.vdb.misc.datasource.JDBCXADataSource.getNonXAConnection(Unknown Source) [denodo-vdp-server.jar:8.0.0]
        at com.denodo.vdb.misc.connection.jdbc.JDBCRoute.getConnection(Unknown Source) [denodo-vdp-server.jar:8.0.0]
        at com.denodo.vdb.engine.wrapper.raw.jdbc.JDBCAccess.doRun(Unknown Source) [denodo-vdp-server.jar:8.0.0]
        at com.denodo.vdb.engine.wrapper.RawAccess.run(Unknown Source) [denodo-vdp-server.jar:8.0.0]
        at com.denodo.vdb.engine.thread.WorkWrapper.a3(Unknown Source) [denodo-vdp-server.jar:8.0.0]
        at com.denodo.vdb.engine.thread.ReusableThread.d3(Unknown Source) [denodo-vdp-server.jar:8.0.0]
        at com.denodo.vdb.engine.thread.ReusableThread.run(Unknown Source) [denodo-vdp-server.jar:8.0.0]
Caused by: com.simba.spark.support.exceptions.ErrorException: [Simba][SparkJDBCDriver](500593) Communication link failure. Failed to connect to server. Reason: com.simba.spark.jdbc42.internal.apache.http.NoHttpResponseException: <Databricks URL> failed to respond.
        
Caused by: com.simba.spark.jdbc42.internal.apache.thrift.transport.TTransportException: com.simba.spark.jdbc42.internal.apache.http.NoHttpResponseException: <Databricks URL> failed to respond
        at com.simba.spark.hivecommon.api.TETHttpClient.flushUsingHttpClient(Unknown Source) ~[SparkJDBC42.jar:?]
        at com.simba.spark.hivecommon.api.TETHttpClient.flush(Unknown Source) ~[SparkJDBC42.jar:?]
4

0 回答 0