1

上周六，我们开始遇到管道损坏问题，每天清晨通过石英调度程序运行一项工作。下面是这方面的一些细节。

该作业由一个调用基于 Talend 的作业的本机 java 类组成。这些基于 Talend 的工作中的每一个都遇到了管道损坏错误。奇怪的是，当我运行通过调度程序提交的同一个 java 类时，没有抛出任何错误并且作业成功运行。此外，此作业流在 Tomcat 中运行。

在 Talend 工作中我应该做些什么来先查看是否有可用的连接？正如您将在以下用于 tMysqlConnection 组件的 jdbc 参数中看到的那样，自动重新连接选项设置为 on。

jdbc参数如下是“noDatetimeStringSync=true&connectTimeout=0&socketTimeout=0&autoReconnect=true”

下面是今天早上作业失败的堆栈跟踪。

非常感谢任何输入。谢谢你。

    Exception in component tMysqlConnection_1  
    com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: The last packet successfully received     from the server was 344,265,715 milliseconds ago.  The last packet sent successfully to the server was 344,265,792 milliseconds ago. is longer than the server configured value of 'wait_timeout'. You should consider either expiring and/or testing connection validity before use in your application, increasing the server configured values for client timeouts, or using the Connector/J connection property 'autoReconnect=true' to avoid this problem.
    at sun.reflect.GeneratedConstructorAccessor840.newInstance(Unknown Source)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:532)
    at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
    at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:1117)
    at com.mysql.jdbc.MysqlIO.send(MysqlIO.java:3871)
    at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2484)
    at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2664)
    at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2788)
    at com.mysql.jdbc.ConnectionImpl.setAutoCommit(ConnectionImpl.java:5282)
    at umaxbalancing_etl.csxdemand_masterservice_0_1.CSXDemand_MasterService.tMysqlConnection_1Process(CSXDemand_MasterService.java:2381)
    at umaxbalancing_etl.csxdemand_masterservice_0_1.CSXDemand_MasterService.tFileList_2Process(CSXDemand_MasterService.java:2284)
    at umaxbalancing_etl.csxdemand_masterservice_0_1.CSXDemand_MasterService.tFileExist_1Process(CSXDemand_MasterService.java:1146)
    at umaxbalancing_etl.csxdemand_masterservice_0_1.CSXDemand_MasterService.tJava_2Process(CSXDemand_MasterService.java:1054)
    at umaxbalancing_etl.csxdemand_masterservice_0_1.CSXDemand_MasterService.tJava_1Process(CSXDemand_MasterService.java:972)
    at umaxbalancing_etl.csxdemand_masterservice_0_1.CSXDemand_MasterService.runJobInTOS(CSXDemand_MasterService.java:4370)
    at umaxbalancing_etl.csxdemand_masterservice_0_1.CSXDemand_MasterService.runJob(CSXDemand_MasterService.java:4079)
    at com.rez1.dws.mid.etl.engine.extract.talend.RunTalendDailyJobs.runCSXDemandExtract(RunTalendDailyJobs.java:45)
    at com.rez1.dws.mid.etl.engine.extract.talend.RunTalendDailyJobs.dailyJobs(RunTalendDailyJobs.java:28)
    at com.rez1.dws.mid.etl.engine.master.service.DWMasterETLServiceImpl.processTalendDailyJobs(DWMasterETLServiceImpl.java:401)
    at com.rez1.dws.mid.etl.engine.master.service.DWMasterETLServiceImpl$$FastClassByCGLIB$$13fc5e62.invoke(<generated>)
    at net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:149)
    at org.springframework.aop.framework.Cglib2AopProxy$CglibMethodInvocation.invokeJoinpoint(Cglib2AopProxy.java:688)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
    at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:91)
    at com.rez1.common.processmonitor.ProcessMonitor.handleNestedInvocation(ProcessMonitor.java:440)
    at com.rez1.common.processmonitor.ProcessMonitor.handleJoinPoint(ProcessMonitor.java:404)
    at com.rez1.op.core.performance.aspect.OperationalProgressMonitorAspect.operationalMonitor(OperationalProgressMonitorAspect.java:66)
    at sun.reflect.GeneratedMethodAccessor36.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:616)
    at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:621)
    at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:610)
    at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:65)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:161)
    at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:91)
    at com.rez1.dws.mid.etl.engine.aspect.DWMasterETLExceptionMonitorAspect.monitorForExceptions(DWMasterETLExceptionMonitorAspect.java:44)
    at sun.reflect.GeneratedMethodAccessor57.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:616)
    at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:621)
    at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:610)
    at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:65)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
    at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:89)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
    at org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor.intercept(Cglib2AopProxy.java:621)
    at com.rez1.dws.mid.etl.engine.master.service.DWMasterETLServiceImpl$$EnhancerByCGLIB$$ca0f48f4.processTalendDailyJobs(<generated>)
    at com.rez1.apipublish.dws.facade.DataWarehouseFacadeImpl.talendDailyJob(DataWarehouseFacadeImpl.java:169)
    at com.rez1.apipublish.dws.facade.DataWarehouseFacadeImpl$$FastClassByCGLIB$$fcc756cf.invoke(<generated>)
    at net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:149)
    at org.springframework.aop.framework.Cglib2AopProxy$CglibMethodInvocation.invokeJoinpoint(Cglib2AopProxy.java:688)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
    at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:91)
    at com.rez1.common.processmonitor.ProcessMonitor.handleFirstInvocation(ProcessMonitor.java:369)
    at com.rez1.common.processmonitor.ProcessMonitor.handleJoinPoint(ProcessMonitor.java:402)
    at com.rez1.op.core.performance.aspect.OperationalProgressMonitorAspect.operationalMonitor(OperationalProgressMonitorAspect.java:66)
    at sun.reflect.GeneratedMethodAccessor36.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:616)
    at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:621)
    at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:610)
    at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:65)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:161)
    at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:89)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
    at org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor.intercept(Cglib2AopProxy.java:621)
    at com.rez1.apipublish.dws.facade.DataWarehouseFacadeImpl$$EnhancerByCGLIB$$521e5519.talendDailyJob(<generated>)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:616)
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:309)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
    at org.springframework.remoting.support.RemoteInvocationTraceInterceptor.invoke(RemoteInvocationTraceInterceptor.java:77)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
    at $Proxy75.talendDailyJob(Unknown Source)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:616)
    at org.springframework.remoting.support.RemoteInvocation.invoke(RemoteInvocation.java:205)
    at org.springframework.remoting.support.DefaultRemoteInvocationExecutor.invoke(DefaultRemoteInvocationExecutor.java:38)
    at org.springframework.remoting.support.RemoteInvocationBasedExporter.invoke(RemoteInvocationBasedExporter.java:78)
    at org.springframework.remoting.support.RemoteInvocationBasedExporter.invokeAndCreateResult(RemoteInvocationBasedExporter.java:114)
    at org.springframework.remoting.httpinvoker.HttpInvokerServiceExporter.handleRequest(HttpInvokerServiceExporter.java:73)
    at org.springframework.web.servlet.mvc.HttpRequestHandlerAdapter.handle(HttpRequestHandlerAdapter.java:49)
    at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:790)
    at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:719)
    at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:644)
    at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:560)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:637)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
    at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:852)
    at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
    at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
    at java.lang.Thread.run(Thread.java:636)
Caused by: java.net.SocketException: Broken pipe
    at java.net.SocketOutputStream.socketWrite0(Native Method)
    at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:109)
    at java.net.SocketOutputStream.write(SocketOutputStream.java:153)
    at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
    at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
    at com.mysql.jdbc.MysqlIO.send(MysqlIO.java:3852)
4

1 回答 1

1

终于到了我遇到的破管错误的底部。

由于我没有从 Talend 作业中关闭某些数据库连接，因此引发了损坏的管道错误。这些有争议的连接是在本机 java 中定义的连接（通过 tJava 组件）。我错误地认为通过引用先前定义的连接（用于连接属性），连接资源将被关闭和释放。

考虑到上述情况，我实际上是在创建一个如下所列的新连接，并且必须专门提交该连接，然后将其关闭。这里要注意的项目是连接上提交的使用。仅在关闭连接时仍会引发损坏的管道错误。直到添加提交后，损坏的管道错误才停止，这是由于连接被正确关闭。

事后诸葛亮...当使用 Talend 组件（而不是本机 Java 代码）进行连接定义和关闭该连接时，自动提交功能可用于 tMysqlCONnection 以及提交选项可用于关闭连接选项在 tMysqlCommit 组件上可用。这两个选项可以防止管道损坏错误，这就是为什么我只遇到 tJava 组件使用的连接出现损坏管道错误的原因。现在有道理了。

// code snippet from the tJava component using native java to read a table.
java.sql.Connection tableConnection = null;
tableConnection = (java.sql.Connection) globalMap.get("conn_tMysqlConnection_1");

Statement selectStatement = tableConnection.createStatement();

// Execute statement to get the row count from the table.
ResultSet resultSet = selectStatement.executeQuery("select count(*) from tableA");

// Position the result set to allow for access of its information.
resultSet.next();

// Get the row count from the result set.
context.numberOfRows = resultSet.getInt(1);

resultSet.close();
selectStatement.close();
tableConnection.commit();
tableConnection.close();

而且...如果我没有提到我发现需要通过以下链接上的文章使用提交，那我将是失职。 http://mikeschubert.com/2006/08/03/javanetsocketex/

于 2013-09-18T12:59:25.593 回答