There is one fixed thread pool (let it be with size=100), that I want to use for all tasks across my app. It is used to limit server load.
Task = web crawler, that submits first job to thread pool.
That job can generate more jobs, and so on.
One job = one HTTP I/O request.
Problem
Suppose that there is only one executing task, that generated 10000 jobs.
Those jobs are now queued in thread pool queue, and all 100 threads are used for their execution.
Suppose that I now submit a second task.
The first job of the second task is 10001th in the queue.
It will be executed only after the 10000 jobs that the first task queued up.
So, this is a problem - I don't want the second task to wait so long to start its first job.
Idea
The first idea on my mind is to create a custom BlockingQueue and pass it to the thread pool constructor.
That queue will hold several blocking queues, one for each task.
Its take method will then choose a random queue and take an item from it.
My problem with this is that I don't see how to remove an empty queue from this list when its task is finished. This would mean some or all workers could get blocked on the take method, waiting for jobs from tasks that are finished.
Is this the best way to solve this problem?
I was unable to find any patterns for it in books or on the Internet :(
Thank you!