HiveBrain v1.2.0
Get Started
← Back to all entries
patternjavaMinor

Design of thread pool reuse with Executors.newFixedThreadPool in Java

Submitted by: @import:stackexchange-codereview··
0
Viewed 0 times
withnewfixedthreadpooldesignexecutorsjavathreadreusepool

Problem

In my application I have code which should run every hour on a new items and execute some expensive operation with multithreading:

`int i = 1;

//noinspection InfiniteLoopStatement
while (true) {

try {
Logger.printIterationNum("Text analysis", "beg", i);

Document query = new Document("fetchStatus", new Document("$lte", fetchStatusParam));
ArrayList unfetchedEvents = dbC_Events.find(query).projection(
fields(include("_id"), include("link"), include("title"), include("summary"))
).into(new ArrayList<>());

// get full text and images of the events
this.getFullTextImg(dbC_Events, unfetchedEvents);

Logger.printIterationNum("Text analysis", "end", i++);

// make a pause between executions
Pauser.rest(Pauser.Units.SECONDS, PAUSE_TIME_OUT);
} catch (Exception e) {
i = 1;
e.printStackTrace();
}
}

private void getFullTextImg(MongoCollection dbC_Events, ArrayList unfetchedEvents) {

int cpuCoresNum = SYS_MAX_NUM_CPU_CORES;
int itemsPerCore = unfetchedEvents.size() / cpuCoresNum;

// prevent infinite thread generation if num of unfetched events is less than number of CPU cores
if (itemsPerCore == 0) {
itemsPerCore = unfetchedEvents.size();
}

// update status of queried items
DBAgent.updateItemsStatus(dbC_Events, unfetchedEvents, FetchStatus.IN_PROCESS_FETCH.getID());

int itNum = 1;
int upperIndex = 0;

ExecutorService service = Executors.newFixedThreadPool(cpuCoresNum);

for (int i = 0; i {
try {
Thread.currentThread().setName("im_evFullTextImg_#" + String.format("%03d", treadID));
this.fetcher(dbC_Events, new ArrayList<>(unfetchedEvents.subList(indexFrom, indexTo)));
} catch (Exception e) {
e.printStackTrace();
}
});
}

// wait until all threads will be finished
service.shutdown();
try {
s

Solution

Answers:

-
It is better to reuse the ExecutorService. Destroying threads and then recreating this is expensive. (The only scenario where destroying and recreating a pool might be advisable is when the pool is going to be unused for a long time, and physical memory usage is a problem. But even then, some flavors of executor are able of shrinking the thread pool size when there is no demand.)

-
Well yes, that would be a problem. But that really means that you should not call shutdown on an executor if you still may need to submit new tasks in it.

-
You can't reuse an executor after calling shutdown on it.

-
There should be no memory leaks ... apart from the fact that the idle threads themselves are using memory (for thread stacks), and will continue to use it until the executor is shut down. For a fixed sized / bounded sized pool, that memory usage by threads is bounded. Most people would not call a bounded memory usage a "leak".

In fact, not reusing an executor is likely to lead to a (real) memory leaks= if you somehow neglect to call shutdown on the executor.

Context

StackExchange Code Review Q#124789, answer score: 2

Revisions (0)

No revisions yet.