MQTT: Make the decodeProperties early-REPLAY check actually fire#16813
Conversation
Motivation: The CVE-2026-44248 fix added a guard at the top of decodeProperties to trigger an early REPLAY when the cumulation buffer did not yet have the full properties block: if (buffer.readableBytes() < totalPropertiesLength) { buffer.readSlice(totalPropertiesLength); } Because MqttDecoder extends ReplayingDecoder, the buffer passed to decodeProperties is a ReplayingDecoderByteBuf whose readableBytes() returns Integer.MAX_VALUE - readerIndex rather than the actual number of bytes available. The if-condition is false in practice and the readSlice() call never executes, so the early-REPLAY optimization is effectively dead code. When the properties block arrives in chunks the decoder still falls back to partial parsing followed by a mid-loop REPLAY, defeating the optimization's intent on slow streams. Modification: Replace the broken readableBytes() check with a getByte() probe at buffer.readerIndex() + totalPropertiesLength - 1. The ReplayingDecoderByteBuf.checkIndex implementation throws Signal.REPLAY when index + 1 > writerIndex - i.e. exactly when the cumulation buffer does not yet hold the full properties block. The call has no side effect on readerIndex, so subsequent parsing is unchanged once the data arrives. A totalPropertiesLength > 0 guard skips the probe when there are no properties. Result: The early-REPLAY guard now actually fires when properties data is incomplete, restoring the CVE-2026-44248 fix's intent of avoiding repeated partial-properties parsing on slow streams. No semantic change on the happy path; all existing codec-mqtt tests pass.
|
@daguimu We also use |
910f019 to
a28ff79
Compare
|
Good catch — you're right that line 100 has the same I've opened #16916 to re-apply the #16787 fix on 4.1 (drop the On the "protected method to bypass |
### Motivation #16787 fixed the `READ_VARIABLE_HEADER` too-long check on the **4.2** branch by replacing a `ReplayingDecoderByteBuf.readableBytes()` probe with the message size declared by the fixed header. That fix was auto-ported to 4.1 in #16838, **but a later CVE-2026-44248 security merge re-introduced the broken code on 4.1**, so the 4.1 branch still carries the regression: ```java int initialAvailableBytes = buffer.readableBytes(); ... if (initialAvailableBytes < maxBytesInMessage) { throw signal; // REPLAY } else { bailOut = true; // too long } ``` Because `MqttDecoder extends ReplayingDecoder`, `buffer` is a `ReplayingDecoderByteBuf` whose `readableBytes()` returns `Integer.MAX_VALUE - readerIndex` rather than the bytes actually buffered. With the default `maxBytesInMessage` of `8092`, the `initialAvailableBytes < maxBytesInMessage` check is therefore effectively always false. So when `decodeVariableHeader` asks for a `REPLAY` because the variable header has not fully arrived yet, the decoder takes the `bailOut` branch and rejects the message with a `TooLongFrameException` instead of waiting for the rest — a valid message whose variable header is split across reads is dropped. ### Modification Re-apply the #16787 fix on 4.1: drop the `initialAvailableBytes` / `readableBytes()` probe and decide based on `bytesRemainingBeforeVariableHeader` (the remaining length declared by the fixed header). A genuinely oversized message is still rejected by the existing `bytesRemainingBeforeVariableHeader > maxBytesInMessage` check; an incomplete one now correctly `REPLAY`s. ### Result A message whose variable header arrives in chunks is decoded once complete, instead of being rejected as too long, while declared-oversize messages still fail with `TooLongFrameException`. The two regression tests from #16787 are ported here. All `codec-mqtt` tests pass locally. Note: this targets **4.1 only** and intentionally carries no cherry-pick label — 4.2 already has the fix via #16787. Found while addressing @chrisvest's review comment on #16813 about the same `readableBytes()` antipattern.
|
Auto-port PR for 4.2: #16919 |
|
Thanks for finding these. Good to have this straightened out again. |
…ually fire (#16919) Auto-port of #16813 to 4.2 Cherry-picked commit: 2d781e2 --- Motivation: The CVE-2026-44248 fix in 82f47fa added a guard at the top of `decodeProperties` to trigger an early REPLAY when the cumulation buffer did not yet have the full properties block: ```java if (buffer.readableBytes() < totalPropertiesLength) { buffer.readSlice(totalPropertiesLength); } ``` Because `MqttDecoder` extends `ReplayingDecoder`, the buffer passed to `decodeProperties` is a `ReplayingDecoderByteBuf` whose `readableBytes()` returns `Integer.MAX_VALUE - readerIndex` rather than the actual number of bytes available (see `ReplayingDecoderByteBuf.readableBytes()`). The `if`-condition is therefore false in practice and the `readSlice()` call never executes, so the early-REPLAY optimization is effectively dead code. When the properties block arrives in chunks the decoder still falls back to partial parsing followed by a mid-loop REPLAY, which defeats the optimization's intent on slow streams. Modification: Replace the broken `readableBytes()` check with a `getByte()` probe at `buffer.readerIndex() + totalPropertiesLength - 1`. `ReplayingDecoderByteBuf.checkIndex` throws `Signal.REPLAY` when `index + 1 > writerIndex` — i.e. exactly when the cumulation buffer does not yet hold the full properties block. The call has no side effect on `readerIndex`, so subsequent parsing is unchanged once the data arrives. A `totalPropertiesLength > 0` guard skips the probe when there are no properties. Result: The early-REPLAY guard now actually fires when properties data is incomplete, restoring the CVE-2026-44248 fix's intent of avoiding repeated partial-properties parsing on slow streams. No semantic change on the happy path; all 107 existing `codec-mqtt` tests pass locally. Note: this is complementary to #16787, which addresses the *outer* variable-header REPLAY handling broken by the same CVE patch (same `buffer.readableBytes()` antipattern in a ReplayingDecoder context). Both fixes are independent and can land in either order; together they restore the optimization the CVE-2026-44248 fix originally aimed for. Co-authored-by: Guimu <[email protected]>
Motivation:
The CVE-2026-44248 fix in 82f47fa added a guard at the top of
decodePropertiesto trigger an early REPLAY when the cumulation buffer did not yet have the full properties block:Because
MqttDecoderextendsReplayingDecoder, the buffer passed todecodePropertiesis aReplayingDecoderByteBufwhosereadableBytes()returnsInteger.MAX_VALUE - readerIndexrather than the actual number of bytes available (seeReplayingDecoderByteBuf.readableBytes()). Theif-condition is therefore false in practice and thereadSlice()call never executes, so the early-REPLAY optimization is effectively dead code. When the properties block arrives in chunks the decoder still falls back to partial parsing followed by a mid-loop REPLAY, which defeats the optimization's intent on slow streams.Modification:
Replace the broken
readableBytes()check with agetByte()probe atbuffer.readerIndex() + totalPropertiesLength - 1.ReplayingDecoderByteBuf.checkIndexthrowsSignal.REPLAYwhenindex + 1 > writerIndex— i.e. exactly when the cumulation buffer does not yet hold the full properties block. The call has no side effect onreaderIndex, so subsequent parsing is unchanged once the data arrives. AtotalPropertiesLength > 0guard skips the probe when there are no properties.Result:
The early-REPLAY guard now actually fires when properties data is incomplete, restoring the CVE-2026-44248 fix's intent of avoiding repeated partial-properties parsing on slow streams. No semantic change on the happy path; all 107 existing
codec-mqtttests pass locally.Note: this is complementary to #16787, which addresses the outer variable-header REPLAY handling broken by the same CVE patch (same
buffer.readableBytes()antipattern in a ReplayingDecoder context). Both fixes are independent and can land in either order; together they restore the optimization the CVE-2026-44248 fix originally aimed for.