Add additional synchronization around *CachesCleanup function

This commit fixes the issue when test_malloc_pools TestFixedBufferPool
fails sporadically on test_malloc_pools.cpp:437

utils::NativePar

Add additional synchronization around *CachesCleanup function

This commit fixes the issue when test_malloc_pools TestFixedBufferPool
fails sporadically on test_malloc_pools.cpp:437

utils::NativeParallelFor( p, FixedPoolUse(p, pool, MAX_OBJECT/p/2,1) );

The main problem was that threads were not aware of cleanup operations
done by SoftCachesCleanup() and HardCachesCleanup() functions.
So while one thread was doing a cleanup, the second one
returns ENOMEM as it couldn't find any memory to allocate in the
backend (this test creates fixedSizePool)

The change adds additional flags and counters that indicate cleanup
operation is in progress, allowing the thread to wait for those
operations to complete before deciding if there is free memory to
allocate or not and return this information to the caller.

In the scenario reproduced by the test, memory for the allocation can
be reclaimed from the LOC. This operation happens in the
loc.cleanAll() or loc.regularCleanup functions called respectively, by
hardCachesCleanup() or softCachesCleanup() functions.
In those functions, free memory is removed from LOC and added to the
backend bin. Between those operations, other threads are unaware that
this memory block is free and returns ENOMEM to the user.

To fix this problem, this patch adds two flags to indicate that
cleanup (soft or hard) is in progress, and threads should wait before
returning NULL to the user.

But there was one more problem: backend::clean() function.
This function removes memory from the bins to try to release or
coalesce it. The problem is that other threads were not aware of
this operation. It causes a 'funny' situation when:

Thread 1: Performs HardCachescleanup function
Thread 2: wait "waitTillBlockReleased()"
Thread 1: Returns memory from LOC
Thread 2: exits waitTillBlockReleased() as it detected new memory in
backend
Thread 1: starts backend::clean() - removes memory from the bin to try
coalesce it
Thread 2: Fail memory allocation as there is no free block in
bins, and returns ENOMEM
Thread 1: Finish coalescing and return memory back to bins

To resolve this problem, this patch adds a counter which allows
threads to detect when clean operation is done so that the threads
will wait for the complition of there is no memory available.

Signed-off-by: Łukasz Plewa <[email protected]>

show more ...

Commit oneTBB source code 82ff8707

Commit oneTBB source code 9ed4a21c

Commit oneTBB source code 4ebd2d06