I have a ExecutorService with a pool size of about 10.
On some occasions the thread will require a semaphore which has 3 permits (connections to an external system).
Not all threads need to perform this task - hence sometimes the thread will complete without the need of an external system connection.
Eventually the thread executor ends up with 3 threads using the external system connection and 7 more waiting on acquiring a connection.
Meanwhile, there is 20,000 more threads waiting to complete tasks which may or may not need the external connection.
This is a waste of resources, as the other 20,000 threads may as well be executing at this time.
I have thought about this, but can't find an appropriate solution (possible due to my lack of experience on the concurrency libraries).
A few key notes:
It is NOT possible to identify if the thread will need the external system connection before execution
The external system connection needs to be performed as part of the threads transaction. In other words, there is no foreseeable way to remove this component of work from the thread and create a separate pool of threads.
So, is there a way I can allow 7 threads to execute other tasks while 3 use the connection to another system and X number of threads are waiting on that connection?