Repository navigation
Fixed - IndexOutOfBoundsException in ContainsDecoder.decode during lock watchdog renewal - #7403
Open
dengwn1999 wants to merge 1 commit into
Open
dengwn1999 wants to merge 1 commit into
dengwn1999 wants to merge 1 commit into
Conversation
…ck watchdog renewal ContainsDecoder aliased the caller-supplied collection when it was a List. LockTask.renew() and ReadLockTask.renew() pass their mutable keys list to the decoder and then shrink that same list inside the completion callback (keys.removeAll(existingNames)). When the reply is decoded after the callback already ran - delayed Redis response, command retry or connection switch - the decoder walks parts (whose size equals the number of KEYS sent) while reading from a shortened args list, throwing IndexOutOfBoundsException. The EVAL future is then never completed, and since RenewalTask only re-schedules from the whenComplete callback, the watchdog renewal chain for that RedissonClient stalls until the command times out. Held locks can expire while still in use. Snapshot the collection in the constructor instead of aliasing it. Decoding semantics are unchanged for every existing caller.
Member
|
Please sign DCO |
mrniko
reviewed
Oct 28, 2026
mrniko
left a comment
Member
There was a problem hiding this comment.
If the reply is decoded after that callback has already run — delayed Redis response, command retry, connection switch — decode() iterates over parts (whose size always equals the number of KEYS sent, since the Lua script returns one element per key) while reading from the now-shorter args list, and throws IndexOutOfBoundsException:
How is it possible? The callback runs on the decoder’s own output. existingNames is the list that ContainsDecoder.decode() returns. The order is:
- Redis sends the reply.
- CommandDecoder calls ContainsDecoder.decode(parts).
- The returned list completes the future.
- thenCompose(existingNames -> keys.removeAll(...)) runs.
The callback can’t run before step 2 because its input doesn’t exist yet. So “decoded after the callback already ran” would need the same reply to be decoded a second time, after its future had already completed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
ContainsDecoderkept a live reference to the caller-suppliedCollectionwhenever it happened to be aList:LockTask.renew()andReadLockTask.renew()both pass their mutablekeyslist to the decoder and then shrink that same list inside the completion callback:If the reply is decoded after that callback has already run — delayed Redis response, command retry, connection switch —
decode()iterates overparts(whose size always equals the number ofKEYSsent, since the Lua script returns one element per key) while reading from the now-shorterargslist, and throwsIndexOutOfBoundsException:Because the exception is thrown while decoding, the
EVALfuture is never completed.RenewalTaskschedules the next round only from thewhenCompletecallback, so the watchdog renewal chain for thatRedissonClientstalls until the command itself times out. Held locks can then expire while still in use, which silently breaks mutual exclusion.Note that the existing
parts.isEmpty()guard does not help here: this code path always sends at least one key, sopartsis never empty. Thekeys.isEmpty()guard added in #6506 does not help either, becausekeysis non-empty when the command is fired — it is only shrunk afterwards.Fix
Snapshot the incoming collection in the constructor:
This makes the decoder immune to later caller-side mutations, preserves decoding semantics for every existing caller (only
LockTask,ReadLockTaskandRedissonSetconstruct it, none of which relies on the aliasing), and removes theinstanceof Listspecial case.Reproduction
The failure is deterministic once the ordering is forced. With the unpatched
ContainsDecoderfrom 3.52.0:argscleared,parts=[1]Index 0 out of bounds for length 0argsshrunk to 1,partshas 3 elementsIndex 1 out of bounds for length 1Index 488 out of bounds for length 488All three throw exactly the signatures reported from production. With the patch, all pass.
Tests
Added
ContainsDecoderTest(JUnit 5, 6 cases) covering the plain decode paths, empty reply, full argument removal, partial argument removal, and decoding concurrent with caller-side mutation.Against the unpatched 3.52.0 class the same suite reports 3 failures — the three mutation cases above.
On the remaining bounds behaviour
Worth being explicit about scope: this patch removes the only reachable cause of the exception, it does not turn
decode()into code that cannot throw. The loop still readsargs.get(index)without checkingindex < args.size(), so the following remain true after the patch:parts.size() == args.size()(normal path)parts.size() < args.size()parts.size() > args.size()IndexOutOfBoundsExceptionargs, non-emptypartsIndexOutOfBoundsExceptionpartsparts.isEmpty()guard)Neither of the two remaining throwing cases is reachable from the renewal path:
argscannot occur, becauseLockTask.renew()andReadLockTask.renew()both return early whenkeys.isEmpty()(the guard added in IndexOutOfBoundsException from RenewalTask #6506), so the command is never dispatched with zero keys.parts.size() > args.size()cannot occur, because the returnedpartssize is fixed by the Lua script embedded inLockTask.renew()/ReadLockTask.renew(): it appends exactly one element perKEYSentry, andKEYSis built from the sameargslist. The two lengths are equal by construction, and after this patchargscan no longer be shortened behind the decoder's back.I deliberately kept the change to the constructor rather than adding a bounds check inside the loop. Silently skipping mismatched indices would mask a genuine protocol anomaly — if a future change to the Lua script or a decode ordering issue ever produced more elements than keys, the current behaviour surfaces it loudly instead of quietly renewing a subset of locks. If you would prefer defence in depth here anyway, the minimal version is:
Happy to add that as a follow-up if you consider it preferable.
Related
LockTask.renew()dereferencingkeys.get(0)whenkeysis empty); fixed in 3.46.1 and already present in 3.52.0597d604), which is what made the shared-list aliasing reachable