Skip to content

Fix Resampler heap corruption when input block exceeds r8b MaxInLen#2114

Merged
ten9876 merged 1 commit intoten9876:mainfrom
NF0T:fix/resampler-max-block-overflow
Apr 28, 2026
Merged

Fix Resampler heap corruption when input block exceeds r8b MaxInLen#2114
ten9876 merged 1 commit intoten9876:mainfrom
NF0T:fix/resampler-max-block-overflow

Conversation

@NF0T
Copy link
Copy Markdown
Contributor

@NF0T NF0T commented Apr 28, 2026

Problem

r8b::CDSPResampler24 pre-allocates its internal filter buffers (TmpBufs) to
aMaxInLen at construction time. The library does not bounds-check the
l parameter on subsequent process() calls — if l > aMaxInLen, it silently
overflows those pre-allocated buffers and corrupts adjacent heap memory.

Resampler was constructed with the default maxBlockSamples = 4096, which
equals approximately 170 ms of audio at 24 kHz (the RADE TX sample rate). The
overflow guard was never stored, so there was no way to enforce it at call sites.

How the overflow was triggered in practice

The TX audio path reads mic data via QAudioSource::readAll() inside
onTxAudioReady(). readAll() returns all data accumulated in the OS audio
buffer since the last call
— not just one fixed-size block.

When the Qt event loop is stalled for longer than ~170 ms (for example, while
processing a burst of DX cluster or POTA spot messages, or during heavy UI
redraws), mic data accumulates in the PulseAudio/PipeWire server-side buffer.
On resume, the single readAll() call returns a block that can be several
hundred milliseconds deep — 2–3× the r8b MaxInLen limit.

When RADE mode is active on the TX slice, onTxAudioReady() runs continuously
regardless of MOX state (by design, for zero-latency key-up). This kept the TX
resampler active and accumulating data even while not transmitting, making it
reachable on any event loop stall.

Observed crash signature

malloc(): invalid size (unsorted)
Aborted (core dumped)

This is the glibc allocator detecting corrupted free-chunk metadata in the
unsorted bin. The corruption itself happened inside r8b's TmpBufs overflow;
the detection was triggered by the next QByteArray allocation in
processStereoToMono(). Three instances were recorded on 2026-04-26 at
07:59, 08:46, and 19:23 CDT on a FLEX-8400 running firmware v4.1.5.39794.

Fix

Store maxBlockSamples as m_maxBlockSamples in Resampler so it is
available at call time. Add an overflow guard at the top of all four
process*() methods:

if (numSamples > m_maxBlockSamples) {
    QByteArray result;
    for (int offset = 0; offset < numSamples; offset += m_maxBlockSamples)
        result.append(process(in + offset, std::min(numSamples - offset, m_maxBlockSamples)));
    return result;
}

Each method recurses into itself with chunks ≤ m_maxBlockSamples. Because
r8b::CDSPResampler24 maintains its filter state across calls on the same
instance, chunking is transparent to output quality — the resampled stream is
continuous across chunk boundaries.

Stereo methods use offset * 2 for pointer arithmetic to account for
interleaved L/R samples.

Changes

  • src/core/Resampler.h — add int m_maxBlockSamples to private section
  • src/core/Resampler.cpp — store maxBlockSamples in constructor init list;
    add overflow guard to process(), processStereoToMono(),
    processMonoToStereo(), and processStereoToStereo()

Testing

Tested on Linux (Arch) with a FLEX-8400, firmware v4.1.5.39794, RADE mode
active on the TX slice (no transmission — MOX was not keyed).

To reliably reproduce a block larger than the 170 ms / 4096-frame threshold,
the process was frozen with SIGSTOP for 500 ms then resumed with SIGCONT:

pid=$(pgrep -x AetherSDR)
kill -SIGSTOP $pid
sleep 0.5
kill -SIGCONT $pid

This forces PulseAudio/PipeWire to accumulate ~500 ms of mic data in the
server-side buffer. On resume, readAll() returns a single ~12 000-frame
block — approximately 3× the old MaxInLen limit. Under the same conditions
that produced three hard crashes on the unpatched binary, the patched binary
resumed cleanly with no crash and no audio artifacts. The test was repeated
multiple times consecutively with consistent results.

Copy link
Copy Markdown
Contributor

@aethersdr-agent aethersdr-agent Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice fix, @NF0T. The root cause analysis and reproduction method in the PR description are excellent.

Code review — looks good. A few observations:

  1. Correctness: The chunking logic is sound across all four methods. Pointer arithmetic correctly accounts for interleaved stereo (offset * 2) in processStereoToMono and processStereoToStereo, and uses plain offset for the mono paths. std::min handles the final partial chunk correctly.

  2. Recursion terminates: Each recursive call passes a chunk size <= m_maxBlockSamples, so the guard can never trigger again — no risk of unbounded recursion.

  3. Member init order: m_maxBlockSamples is declared between m_dstRate and m_resampler in the header, and the constructor init list matches that order. Good — no -Wreorder warning.

  4. Scope: Changes are confined to Resampler.h and Resampler.cpp, exactly matching the stated scope. No unrelated changes.

  5. r8b filter state continuity: As noted in the PR description, CDSPResampler24::process() maintains internal filter state across calls, so chunking doesn't introduce discontinuities. This is the key property that makes the fix transparent to audio quality.

One minor note (not blocking): The repeated QByteArray::append() in the chunking loop will trigger a reallocation on each iteration. For the typical case of 2–3 chunks this is negligible, but if you wanted to tighten it up, you could result.reserve(estimatedOutputSize) before the loop. Not necessary for this use case — just mentioning it.

Thanks for tracking this down and for the thorough testing with the SIGSTOP/SIGCONT approach. 73.

r8b::CDSPResampler24 pre-allocates internal filter buffers sized to
aMaxInLen at construction time and documents that process() must not
be called with more samples than that limit.  No bounds check is
performed — exceeding it silently overflows the internal TmpBufs,
corrupting adjacent heap memory.

The default maxBlockSamples is 4096, which equals ~170 ms of audio at
24 kHz.  AudioEngine::onTxAudioReady() drains the mic source with
readAll(), so any event-loop hiccup longer than 170 ms (e.g. a burst
of POTA spot messages being parsed on the main thread) produces a
super-block that exceeds the limit.  The resulting heap corruption is
detected on the next malloc call inside processStereoToMono(),
producing malloc(): invalid size (unsorted) and SIGABRT.

Fix: add an overflow guard at the top of all four process* methods.
When the input exceeds m_maxBlockSamples the method chunks itself
recursively, each chunk ≤ maxBlockSamples.  r8b is stateful between
calls on the same instance, so filter continuity is preserved across
chunk boundaries at no extra cost.

Root-caused from three SIGABRT core dumps (07:59, 08:46, 19:23 CDT
2026-04-26), all crashing in Resampler::processStereoToMono via
RADEEngine::feedTxAudio.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
@NF0T NF0T force-pushed the fix/resampler-max-block-overflow branch from 0bbecff to f926cb1 Compare April 28, 2026 13:11
@ten9876 ten9876 merged commit 0d8b0eb into ten9876:main Apr 28, 2026
5 checks passed
@NF0T NF0T deleted the fix/resampler-max-block-overflow branch April 28, 2026 23:39
ten9876 added a commit that referenced this pull request Apr 29, 2026
Community-driven release.  WAVE Phase 2 visualization (#2124),
DAX-aware TCI multi-stream routing for FlexRadio firmware 4.2.18
(#2140), TCXO frequency-offset calibration (#2119), VFO marker
tri-state UX (#2141), v4.2.18 discovery beacon parsing (#2138).
Bug fixes from the community: r8b heap corruption (#2114, NF0T),
serial PTT triple-fix (#2125, chibondking), slice-audio mute on
band change (#2128, jensenpat), CWX Live toggle (#2122, jensenpat),
connect-radio dialog polish (#2121, jensenpat).

Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants