Ehcache version
2.10.6
Once in a while, Ehcache servers are getting down because of the threads getting blocked. Please find below the details about the bucket configuration and thread details which are blocking each other.
There are multiple buckets: Bucket configuration
<cache name="bucket1"
maxBytesLocalHeap="300m"
maxBytesLocalDisk="15g"
timeToIdleSeconds="240"
timeToLiveSeconds="240"
eternal="false"
diskSpoolBufferSizeMB="30"
memoryStoreEvictionPolicy="LFU"
diskExpiryThreadIntervalSeconds="360"
transactionalMode="off"
diskPersistent="true">
</cache>
Thread dump
http-nio-8080-exec-26
Stack Trace is:
java.lang.Thread.State: BLOCKED (on object monitor)
at net.sf.ehcache.store.cachingtier.OnHeapCachingTier$Fault.get(OnHeapCachingTier.java:332)
- **waiting to lock <0x0000000604407968>** (a net.sf.ehcache.store.cachingtier.OnHeapCachingTier$Fault)
at net.sf.ehcache.store.cachingtier.OnHeapCachingTier$Fault.access$200(OnHeapCachingTier.java:312)
at net.sf.ehcache.store.cachingtier.OnHeapCachingTier.getValue(OnHeapCachingTier.java:300)
at net.sf.ehcache.store.cachingtier.OnHeapCachingTier.access$000(OnHeapCachingTier.java:62)
at net.sf.ehcache.store.cachingtier.OnHeapCachingTier$1.evicted(OnHeapCachingTier.java:84)
at net.sf.ehcache.store.cachingtier.PooledBasedBackEnd$1.removed(PooledBasedBackEnd.java:195)
at net.sf.ehcache.util.concurrent.ConcurrentHashMap.internalReplace(ConcurrentHashMap.java:1346)
at net.sf.ehcache.util.concurrent.ConcurrentHashMap.removeAndNotify(ConcurrentHashMap.java:2647)
at net.sf.ehcache.store.cachingtier.PooledBasedBackEnd.remove(PooledBasedBackEnd.java:113)
at net.sf.ehcache.store.cachingtier.OnHeapCachingTier.remove(OnHeapCachingTier.java:207)
at net.sf.ehcache.store.CacheStore.put(CacheStore.java:134)
at net.sf.ehcache.Cache.putInternal(Cache.java:1616)
at net.sf.ehcache.Cache.put(Cache.java:1542)
at net.sf.ehcache.Cache.put(Cache.java:1507)......
http-nio-8080-exec-174
Stack Trace is:
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:502)
at net.sf.ehcache.util.concurrent.ConcurrentHashMap$Node.tryAwaitLock(ConcurrentHashMap.java:687)
- locked <0x00000005fa31d820> (a net.sf.ehcache.util.concurrent.ConcurrentHashMap$Node)
at net.sf.ehcache.util.concurrent.ConcurrentHashMap.internalReplace(ConcurrentHashMap.java:1311)
at net.sf.ehcache.util.concurrent.ConcurrentHashMap.remove(ConcurrentHashMap.java:2643)
at net.sf.ehcache.store.cachingtier.PooledBasedBackEnd.evict(PooledBasedBackEnd.java:159)
at net.sf.ehcache.store.cachingtier.PooledBasedBackEnd$PoolParticipant.evict(PooledBasedBackEnd.java:251)
at net.sf.ehcache.pool.impl.FromLargestCachePoolEvictor.freeSpace(FromLargestCachePoolEvictor.java:49)
at net.sf.ehcache.pool.impl.AtomicPoolAccessor.add(AtomicPoolAccessor.java:71)
at net.sf.ehcache.pool.impl.AbstractPoolAccessor.add(AbstractPoolAccessor.java:67)
at net.sf.ehcache.store.disk.Segment.put(Segment.java:427)
at net.sf.ehcache.store.disk.DiskStore.putFaulted(DiskStore.java:229)
at net.sf.ehcache.store.CacheStore$3.call(CacheStore.java:143)
at net.sf.ehcache.store.CacheStore$3.call(CacheStore.java:137)
at net.sf.ehcache.store.cachingtier.OnHeapCachingTier$Fault.get(OnHeapCachingTier.java:335)
**- locked <0x0000000604407968> (a net.sf.ehcache.store.cachingtier.OnHeapCachingTier$Fault)**
at net.sf.ehcache.store.cachingtier.OnHeapCachingTier$Fault.access$200(OnHeapCachingTier.java:312)
at net.sf.ehcache.store.cachingtier.OnHeapCachingTier.get(OnHeapCachingTier.java:176)
at net.sf.ehcache.store.CacheStore.put(CacheStore.java:137)
at net.sf.ehcache.Cache.putInternal(Cache.java:1616)
at net.sf.ehcache.Cache.put(Cache.java:1542)
at net.sf.ehcache.Cache.put(Cache.java:1507)
.......