Skip to content

Auto-port 5.0: Adaptive: Fix concurrency issue in adaptive allocator#16777

Merged
chrisvest merged 1 commit into
5.0from
auto-port-pr-16767-to-5.0
May 8, 2026
Merged

Auto-port 5.0: Adaptive: Fix concurrency issue in adaptive allocator#16777
chrisvest merged 1 commit into
5.0from
auto-port-pr-16767-to-5.0

Conversation

@netty-project-bot
Copy link
Copy Markdown
Contributor

Auto-port of #16767 to 5.0
Cherry-picked commit: bd866c3


Motivation:
We have a few code paths where Chunk.remainingCapacity() is called without regard for exclusive access to the given chunk. The BuddyChunk implementation unfortunately drained the freeList in this call, leading to racy modifications of the buddies array, that in turn can manifest as unpredictable bugs.

Modification:
Make the BuddyChunk.remainingCapacity non-modifying. Instead of draining the freeList, we sum up the capacity held by it and add it to the capacity accounted for in the Chunk super class.

This operation needs a weakPeekReduce method on the MpscIntQueue. This method performs a reduction operation on a given range of entries, but without removing entries or incrementing the consumer index.

A regression test is added, which successfully captured the issue before it was fixed.

Also did a few test cleanups.

Result:
No more corruptions in the buddies array.

This fixes the rare occurrences of assertion errors like the following:

java.lang.AssertionError
	at io.netty.buffer.AdaptivePoolingAllocator$Magazine.allocate(AdaptivePoolingAllocator.java:916)
	at io.netty.buffer.AdaptivePoolingAllocator$Magazine.tryAllocate(AdaptivePoolingAllocator.java:854)
	at io.netty.buffer.AdaptivePoolingAllocator$MagazineGroup.allocate(AdaptivePoolingAllocator.java:421)
	at io.netty.buffer.AdaptivePoolingAllocator.allocate(AdaptivePoolingAllocator.java:271)

Motivation:
We have a few code paths where `Chunk.remainingCapacity()` is called
without regard for exclusive access to the given chunk. The `BuddyChunk`
implementation unfortunately drained the `freeList` in this call,
leading to racy modifications of the `buddies` array, that in turn can
manifest as unpredictable bugs.

Modification:
Make the `BuddyChunk.remainingCapacity` non-modifying. Instead of
draining the `freeList`, we sum up the capacity held by it and add it to
the capacity accounted for in the `Chunk` super class.

This operation needs a `weakPeekReduce` method on the `MpscIntQueue`.
This method performs a reduction operation on a given range of entries,
but without removing entries or incrementing the consumer index.

A regression test is added, which successfully captured the issue before
it was fixed.

Also did a few test cleanups.

Result:
No more corruptions in the buddies array.

This fixes the rare occurrences of assertion errors like the following:

```
java.lang.AssertionError
	at io.netty.buffer.AdaptivePoolingAllocator$Magazine.allocate(AdaptivePoolingAllocator.java:916)
	at io.netty.buffer.AdaptivePoolingAllocator$Magazine.tryAllocate(AdaptivePoolingAllocator.java:854)
	at io.netty.buffer.AdaptivePoolingAllocator$MagazineGroup.allocate(AdaptivePoolingAllocator.java:421)
	at io.netty.buffer.AdaptivePoolingAllocator.allocate(AdaptivePoolingAllocator.java:271)
```

(cherry picked from commit bd866c3)
@chrisvest chrisvest merged commit 9840681 into 5.0 May 8, 2026
10 of 13 checks passed
@chrisvest chrisvest deleted the auto-port-pr-16767-to-5.0 branch May 8, 2026 22:55
@chrisvest chrisvest added this to the 5.0.0.Final milestone May 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants