java - 表观死锁为未分配的挂起任务创建紧急线程

Question

我正在使用带有 mybatis 的 mysql，我在我们的实时服务器上遇到了这个错误

com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDetector@6538f8f2 
-- APPARENT DEADLOCK!!! Creating emergency threads for unassigned pending tasks!

我不明白为什么会出现这个错误是因为我的 C3P0 设置？我的 C3P0 设置是这样的

----开始更新-----

下面是我的 spring-servlet.xml 配置

我将数据源 bean更新为

<bean id="datasource" class="com.mchange.v2.c3p0.ComboPooledDataSource"
destroy-method="close" p:driverClass="com.mysql.jdbc.Driver"
p:jdbcUrl="jdbc:mysql://localhost/jdb" p:user="root" p:password="root" 
p:acquireIncrement="10" 
p:idleConnectionTestPeriod="60"
p:maxPoolSize="100" 
    p:maxStatements="0" 
    p:minPoolSize="10" 
    p:initialPoolSize="10"
    p:statementCacheNumDeferredCloseThreads="1" />
   <!-- Declare a transaction manager -->

<bean id="transactionManager"
class="org.springframework.jdbc.datasource.DataSourceTransactionManager"
p:dataSource-ref="datasource" />


<bean id="sqlSessionFactory" class="org.mybatis.spring.SqlSessionFactoryBean">
    <property name="dataSource" ref="datasource" />
</bean>

<!-- scan for mappers and will automatically scan the whole classpath for xmls -->
<bean class="org.mybatis.spring.mapper.MapperScannerConfigurer">
    <property name="sqlSessionFactory" ref="sqlSessionFactory" />
    <property name="basePackage" value="com.mycom.myproject.db.mybatis.dao" />
</bean>

从我的 Dao 类中，我调用 mapper 方法，例如

 myDao.updateRecords()

这是我的服务类方法

@Override
public List<UserDetailedBean> selectAllUsersDetail(long groupId, List<Long> ids) {

    List<UserDetailedBean> usersDetailList = null;

    try {
        usersDetailList = userDao.selectAllUsersDetail(groupId, ids);
    } catch (Exception e) {
        e.printStackTrace();
    }

    return usersDetailList;
}

在 Dao 类中，我只注入映射器。

@Resource
private UserMapper userMapper;

@Override
public List<UserDetailedBean> selectAllUsersDetail(long groupId, List<Long> ids) {
    return userMapper.selectAllUsersDetail(groupId,ids);
}

---结束更新-----

如果需要任何其他信息，请告诉我。

这是完整的堆栈跟踪

[ WARN] 2013-01-08 20:13:39       com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDetector@70497e11 -- APPARENT DEADLOCK!!! Creating emergency threads for unassigned pending tasks!
[ WARN] 2013-01-08 20:13:39 com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDetector@70497e11 -- APPARENT DEADLOCK!!! Complete Status: 
 Managed Threads: 3
 Active Threads: 3
 Active Tasks: 
     com.mchange.v2.resourcepool.BasicResourcePool$AsyncTestIdleResourceTask@2e81b8c5 (com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#0)
     com.mchange.v2.resourcepool.BasicResourcePool$AsyncTestIdleResourceTask@4689a55d (com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#2)
     com.mchange.v2.resourcepool.BasicResourcePool$AsyncTestIdleResourceTask@76c7a0d8 (com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#1)
 Pending Tasks: 
     com.mchange.v2.resourcepool.BasicResourcePool$AsyncTestIdleResourceTask@2c1101d4
     com.mchange.v2.resourcepool.BasicResourcePool$AsyncTestIdleResourceTask@108f1be6
     com.mchange.v2.resourcepool.BasicResourcePool$AsyncTestIdleResourceTask@2370a188
     com.mchange.v2.resourcepool.BasicResourcePool$AsyncTestIdleResourceTask@377cf9e5
     com.mchange.v2.resourcepool.BasicResourcePool$AsyncTestIdleResourceTask@6dfa45d8
     com.mchange.v2.resourcepool.BasicResourcePool$AsyncTestIdleResourceTask@49ffa050
     com.mchange.v2.resourcepool.BasicResourcePool$AsyncTestIdleResourceTask@2d760a24
    Pool thread stack traces:
 Thread[com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#0,5,main]
     java.net.SocketInputStream.socketRead0(Native Method)
     java.net.SocketInputStream.read(SocketInputStream.java:150)
     java.net.SocketInputStream.read(SocketInputStream.java:121)
     com.mysql.jdbc.util.ReadAheadInputStream.fill(ReadAheadInputStream.java:114)
     com.mysql.jdbc.util.ReadAheadInputStream.readFromUnderlyingStreamIfNecessary(ReadAheadInputStream.java:161)
     com.mysql.jdbc.util.ReadAheadInputStream.read(ReadAheadInputStream.java:189)
     com.mysql.jdbc.MysqlIO.readFully(MysqlIO.java:2549)
     com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:3002)
     com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:2991)
     com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3532)
     com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2002)
     com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2163)
     com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2618)
     com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2568)
     com.mysql.jdbc.StatementImpl.executeQuery(StatementImpl.java:1557)
     com.mysql.jdbc.DatabaseMetaData$9.forEach(DatabaseMetaData.java:4984)
     com.mysql.jdbc.IterateBlock.doForAll(IterateBlock.java:51)
     com.mysql.jdbc.DatabaseMetaData.getTables(DatabaseMetaData.java:4962)
     com.mchange.v2.c3p0.impl.DefaultConnectionTester.activeCheckConnectionNoQuery(DefaultConnectionTester.java:185)
     com.mchange.v2.c3p0.impl.DefaultConnectionTester.activeCheckConnection(DefaultConnectionTester.java:62)
     com.mchange.v2.c3p0.AbstractConnectionTester.activeCheckConnection(AbstractConnectionTester.java:67)
     com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.testPooledConnection(C3P0PooledConnectionPool.java:368)
     com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.refurbishIdleResource(C3P0PooledConnectionPool.java:310)
     com.mchange.v2.resourcepool.BasicResourcePool$AsyncTestIdleResourceTask.run(BasicResourcePool.java:1999)
     com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:547)
 Thread[com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#2,5,main]
     java.net.SocketInputStream.socketRead0(Native Method)
     java.net.SocketInputStream.read(SocketInputStream.java:150)
     java.net.SocketInputStream.read(SocketInputStream.java:121)
     com.mysql.jdbc.util.ReadAheadInputStream.fill(ReadAheadInputStream.java:114)
     com.mysql.jdbc.util.ReadAheadInputStream.readFromUnderlyingStreamIfNecessary(ReadAheadInputStream.java:161)
     com.mysql.jdbc.util.ReadAheadInputStream.read(ReadAheadInputStream.java:189)
     com.mysql.jdbc.MysqlIO.readFully(MysqlIO.java:2549)
     com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:3002)
     com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:2991)
     com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3532)
     com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2002)
     com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2163)
     com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2618)
     com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2568)
     com.mysql.jdbc.StatementImpl.executeQuery(StatementImpl.java:1557)
     com.mysql.jdbc.DatabaseMetaData$9.forEach(DatabaseMetaData.java:4984)
     com.mysql.jdbc.IterateBlock.doForAll(IterateBlock.java:51)
     com.mysql.jdbc.DatabaseMetaData.getTables(DatabaseMetaData.java:4962)
     com.mchange.v2.c3p0.impl.DefaultConnectionTester.activeCheckConnectionNoQuery(DefaultConnectionTester.java:185)
     com.mchange.v2.c3p0.impl.DefaultConnectionTester.activeCheckConnection(DefaultConnectionTester.java:62)
     com.mchange.v2.c3p0.AbstractConnectionTester.activeCheckConnection(AbstractConnectionTester.java:67)
     com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.testPooledConnection(C3P0PooledConnectionPool.java:368)
     com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.refurbishIdleResource(C3P0PooledConnectionPool.java:310)
     com.mchange.v2.resourcepool.BasicResourcePool$AsyncTestIdleResourceTask.run(BasicResourcePool.java:1999)
     com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:547)
 Thread[com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#1,5,main]
     java.net.SocketInputStream.socketRead0(Native Method)
     java.net.SocketInputStream.read(SocketInputStream.java:150)
     java.net.SocketInputStream.read(SocketInputStream.java:121)
     com.mysql.jdbc.util.ReadAheadInputStream.fill(ReadAheadInputStream.java:114)
     com.mysql.jdbc.util.ReadAheadInputStream.readFromUnderlyingStreamIfNecessary(ReadAheadInputStream.java:161)
     com.mysql.jdbc.util.ReadAheadInputStream.read(ReadAheadInputStream.java:189)
     com.mysql.jdbc.MysqlIO.readFully(MysqlIO.java:2549)
     com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:3002)
     com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:2991)
     com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3532)
     com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2002)
     com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2163)
     com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2618)
     com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2568)
     com.mysql.jdbc.StatementImpl.executeQuery(StatementImpl.java:1557)
     com.mysql.jdbc.DatabaseMetaData$9.forEach(DatabaseMetaData.java:4984)
     com.mysql.jdbc.IterateBlock.doForAll(IterateBlock.java:51)
     com.mysql.jdbc.DatabaseMetaData.getTables(DatabaseMetaData.java:4962)
     com.mchange.v2.c3p0.impl.DefaultConnectionTester.activeCheckConnectionNoQuery(DefaultConnectionTester.java:185)
     com.mchange.v2.c3p0.impl.DefaultConnectionTester.activeCheckConnection(DefaultConnectionTester.java:62)
     com.mchange.v2.c3p0.AbstractConnectionTester.activeCheckConnection(AbstractConnectionTester.java:67)
     com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.testPooledConnection(C3P0PooledConnectionPool.java:368)
     com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.refurbishIdleResource(C3P0PooledConnectionPool.java:310)
     com.mchange.v2.resourcepool.BasicResourcePool$AsyncTestIdleResourceTask.run(BasicResourcePool.java:1999)
     com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:547)

- -更新 - -

当我将 p:statementCacheNumDeferredCloseThreads="1" 添加到 datasouce bean 时，出现以下错误

     Error creating bean with name 'sqlSessionFactory' defined in ServletContext resource [/WEB-INF/spring-servlet.xml]: 
     Cannot resolve reference to bean 'datasource' while setting bean property 'dataSource'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'datasource' defined in ServletContext resource [/WEB-INF/spring-servlet.xml]: 
   Error setting property values; nested exception is org.springframework.beans.NotWritablePropertyException: Invalid property 'statementCacheNumDeferredCloseThreads' of bean class [com.mchange.v2.c3p0.ComboPooledDataSource]: 
   Bean property 'statementCacheNumDeferredCloseThreads' is not writable or has an invalid setter method. Does the parameter type of the setter match the return type of the getter?

score 18 · Accepted Answer

来自http://www.mchange.com/projects/c3p0/#other_ds_configuration

numHelperThreads和maxAdministrativeTaskTime有助于配置 DataSource 线程池的行为。默认情况下，每个 DataSource 只有三个关联的帮助线程。如果性能似乎在重负载下拖累，或者如果您通过 JMX 或直接检查 PooledDataSource 观察到“待处理任务”的数量通常大于零，请尝试增加 numHelperThreads。maxAdministrativeTaskTime对于遇到无限期挂起的任务和“APPARENT DEADLOCK”消息的用户可能很有用。（更多信息请参见附录 A。）

maxAdministrativeTaskTime 默认值：0 秒之前 c3p0 的线程池将尝试中断一个明显挂起的任务。很少有用。c3p0 的许多功能不是由客户端线程执行的，而是由内部线程池异步执行的。c3p0 的异步性直接增强了客户端性能，并通过确保在非锁持有线程中执行慢速 jdbc 操作来最小化持有关键锁的时间长度。但是，如果其中一些任务“挂起”，即它们在很长一段时间内既没有成功也没有失败并出现异常，c3p0 的线程池可能会耗尽并且管理任务会被备份。如果任务只是很慢，解决问题的最佳方法是增加线程数，通过numHelperThreads. 但是如果任务有时会无限期挂起，您可以使用此参数在任务超过设定的时间限制时强制调用任务线程的 interrupt() 方法。[c3p0 最终将通过发出“明显的死锁”信号（您将在日志中将其视为警告）、替换线程池任务线程和中断（）原始线程来从挂起的任务中恢复。但是让池进入表观死锁然后恢复意味着在某些时期内，c3p0 的性能会受到损害。因此，如果您看到这些消息，增加 numHelperThreads 并设置 maxAdministrativeTaskTime 可能会有所帮助. maxAdministrativeTaskTime 应该足够大，以使任何从数据库获取连接、测试连接或两次破坏连接的合理尝试都可以预期在设定的时间内成功或失败。零（默认）意味着任务从不中断，这是大多数情况下最好和最安全的策略。如果任务很慢，请分配更多线程。如果任务永远挂起，请尝试找出原因，同时设置 maxAdministrativeTaskTime 可能会有所帮助。

The default is 3 for numHelperThreads , increase this to 8-10 

setting maxAdministrativeTaskTime will help

score 11 · Accepted Answer

请查看解决问题的后续步骤：

增加p:maxStatementsComboPooledDataSource。
设置p:maxStatements为 0。例如，在 Firebird 中，此 hack 可用于 ComboPooledDataSource。
确保在应用程序中关闭SqlSession 。更加注意集中执行数据库操作。在我的 mySql JDBC 驱动程序版本中：当对象被垃圾收集时， mysql-connector-java 5.1.8连接会自动关闭。因此，如果您不大量使用数据库，则在您的情况下连接不应该泄漏。尽管如此，您必须确保您关闭了 myBatis SqlSession，它包装了与 DB 的 jdbc 连接。
同样相应的 JDBC3 连接和语句池，您可以尝试在 c3p0 配置中将statementCacheNumDeferredCloseThreads设置为1。

score 1 · Accepted Answer

在我的情况下，应用程序的内存太少是原因。使用的数据库是 H2 或 SQLite（都在这个应用程序中使用）。

第一个症状是上面报告的这些 WARN 日志行：

12006925 [C3P0PooledConnectionPoolManager[identityToken->2rvy8f9szmpczp1k2dm1g|33af2d37]-AdminTaskTimer] WARN  com.mchange.v2.async.ThreadPoolAsynchronousRunner  - com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDetector@6d3a9c65 -- APPARENT DEADLOCK!!! Creating emergency threads for unassigned pending tasks!
12016284 [C3P0PooledConnectionPoolManager[identityToken->2rvy8f9szmpczp1k2dm1g|3d98d1b]-AdminTaskTimer] WARN  com.mchange.v2.async.ThreadPoolAsynchronousRunner  - com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDetector@44565f94 -- APPARENT DEADLOCK!!! Creating emergency threads for unassigned pending tasks!
12051847 [C3P0PooledConnectionPoolManager[identityToken->2rvy8f9szmpczp1k2dm1g|5703a6aa]-AdminTaskTimer] WARN  com.mchange.v2.async.ThreadPoolAsynchronousRunner  - com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDetector@c9f37e2 -- APPARENT DEADLOCK!!! Creating emergency threads for unassigned pending tasks!
12085128 [C3P0PooledConnectionPoolManager[identityToken->2rvy8f9szmpczp1k2dm1g|4e50d42b]-AdminTaskTimer] WARN  com.mchange.v2.async.ThreadPoolAsynchronousRunner  - com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDetector@6f1927b7 -- APPARENT DEADLOCK!!! Creating emergency threads for unassigned pending tasks!
12085128 [C3P0PooledConnectionPoolManager[identityToken->2rvy8f9szmpczp1k2dm1g|78fa7f84]-AdminTaskTimer] WARN  com.mchange.v2.async.ThreadPoolAsynchronousRunner  - com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDetector@22c22b50 -- APPARENT DEADLOCK!!! Creating emergency threads for unassigned pending tasks!
12172644 [C3P0PooledConnectionPoolManager[identityToken->2rvy8f9szmpczp1k2dm1g|e8e88fa]-AdminTaskTimer] WARN  com.mchange.v2.async.ThreadPoolAsynchronousRunner  - com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDetector@745a644f -- APPARENT DEADLOCK!!! Creating emergency threads for unassigned pending tasks!

很长一段时间后出现异常，包括揭示：

Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded

问题是可重现的。为应用程序提供更多内存（-Xmx8G）修复了它。

java - 表观死锁为未分配的挂起任务创建紧急线程

3 回答 3

Related

Reference