FirmwareStager: support v4.2+ MSI (native libmspack) + user-selected installer#2169
Merged
FirmwareStager: support v4.2+ MSI (native libmspack) + user-selected installer#2169
Conversation
FlexRadio switched the SmartSDR installer format in v4.2 from a self-extracting InnoSetup .exe to a WiX 6 MSI (OLE Compound File with embedded LZX-compressed CABs). The old byte-pattern Salted__ scanner fails on the new format — only one Salted__ marker is visible in the raw MSI, and that marker is incidental MSI structure, not a firmware boundary. Changes: - Add `versionUsesMsi(version)` — returns true for v4.2+ versions. Picks the right URL pattern (`SmartSDR_v<v>_x64.msi`) and saves with the right extension. Older versions still use the `SmartSDR_v<v>_Installer.exe` URL and InnoSetup extraction. - Detect installer format at extraction time by reading the first 8 bytes (OLE magic D0 CF 11 E0 A1 B1 1A E1 vs InnoSetup PE/COFF "MZ"). Dispatch to the matching extractor. - Refactor the existing byte-scan logic into `extractFromInnoSetup()` unchanged. - New `extractFromMsi()`: shells out to 7-Zip via QProcess to extract cab streams from the MSI, decompress each cab's LZX-packed payload, scan the decompressed files for Salted__-prefixed blobs, sort by size descending, and pick by family (consumer 6x00 multi-platform firmware is much larger than the 9600 government build, giving us a stable size-based mapping without needing to parse the MSI File table). - New `findExtractionTool()` helper probes for `7z` / `7zz` / `7za` in PATH, plus common Homebrew locations on macOS. Returns empty string if none found, in which case extraction emits a clear error with installation instructions per platform. Tested manually against `SmartSDR_v4.2.18_x64.msi`: the algorithm extracts both firmware blobs (FLEX-6x00 386 MB, FLEX-9600 64 MB) with intact Salted__ headers. Follow-up work to vendor a CAB+LZX decoder in-tree (libmspack or similar) is tracked separately so we can drop the 7-Zip runtime dependency. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Replaces the auto-download flow with user-driven file selection: the
user downloads the SmartSDR installer themselves from flexradio.com,
then picks it via "Select Installer..." in Radio Setup.
Why:
- No URL guesswork (we don't know what FlexRadio's v4.2 download paths
are without testing each one).
- No CDN cache concerns or stale MD5-hash-file URLs.
- User already trusts what they downloaded; we just unpack it.
- Works for any installer the user can obtain — including beta/test
builds FlexRadio hasn't published broadly.
Changes:
- FirmwareStager: new stageFromLocalFile(installerPath, modelFamily).
Detects format from filename suffix (.msi / .exe / .ssdr), parses
version from filename, and dispatches:
- .ssdr → validate header, copy into staging dir, stageComplete.
- .msi → verifyAndExtract() with MSI extractor (already added in
this branch).
- .exe → verifyAndExtract() with InnoSetup byte-scanner.
- RadioSetupDialog: "Browse .ssdr..." button renamed to
"Select Installer...". File dialog accepts *.msi *.exe *.ssdr (with
per-type filters). Successful selection kicks off the stager — the
existing stageProgress/stageComplete/stageFailed slots already wire
the UI, so progress and success messaging just work.
- "Check for Update" no longer re-wires itself to a download button on
v-mismatch. It now reports "Update available: vX.Y.Z — download
from flexradio.com, then click 'Select Installer...' to stage it."
The downloadAndStage() public method stays in the API (no current UI
callers, but the logic still works for any future opt-in flow).
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Firmware flashing is irreversible. If something goes wrong we want maximum forensic information from the single attempt. Adds at QtInfoMsg level (visible without needing debug-level filter): - Pre-upload state dump: file path, filename, size, MD5, magic bytes, radio model, serial, address, current firmware version. All in one delimited block so it's easy to grep. - TX command echoes: "file filename" and "file upload <size> update". - RX response capture: the full body returned by the radio for the file-upload command, including non-zero error codes (we previously only logged the code as hex; the body may contain diagnostic info the radio attaches). - Socket state transitions: every QTcpSocket state change goes to the log so we can reconstruct exactly when/where any disconnect or error occurred. - Socket error details: error enum value, error string, bytes transferred at error time, port in use. No behavioral changes — all additions are log statements. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
…egex The flash worked. The diagnostic spam I added in 9507002 was scaffolding for the one-shot-and-pray run; with end-to-end success confirmed in the log we don't need to keep all of that in production. Reverts the QtInfoMsg pre-upload dump, the per-state socket logging, and the inline TX/RX command echoes — the existing protocol-channel logger already covers the actually-useful parts. Also widens the PII redactor's IPv4 regex to handle a case the post-flash log surfaced: protocol-side `software_ver=4.2.18.41174` was being redacted to `*.*.*. 41174` because the regex matched the first three digits of `41174` as an "octet" with the trailing `74` left dangling. Two extensions: (?<!ver=) — negative lookbehind for `ver=` catches `software_ver=`, `firmware_ver=`, `version=`, etc. without enumerating each. (?![\d"]) — negative lookahead rejects matches where the 4th "octet" is a prefix of a longer number (e.g. 41174). Quoted (`"0.9.2.1"`) and v-prefixed (`v0.9.2.1`) cases stay handled. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Replaces the QProcess-to-7z shell-out in extractFromMsi() with an
in-tree pipeline. No external runtime dependencies, no temp files,
works in any sandboxing model that disallows process spawning.
Pipeline:
OleCompoundFile → cab*.cab streams (in memory, MSI-decoded names)
CabExtractor → files inside each cab (libmspack, in memory)
Filter Salted__ → firmware blobs only
Sort by size → pick by family (consumer vs 9600)
Write to staging
Components:
third_party/libmspack/ — vendored subset of libmspack 0.11alpha
(LGPL-2.1, GPL-3 compatible). 11 source files (~6k lines C);
only CAB+LZX+MSZIP+Quantum decompression, no encoders, no CHM/
HLP/KWAJ/LIT/OAB/SZDD. Built as a static lib with -w to keep
upstream warning style out of our build output. README in the
directory documents the version + license + update procedure.
OleCompoundFile — read-only [MS-CFB] reader. Walks the FAT chain,
parses the directory, locates streams by name. MSI-aware
accessor decodes Microsoft's 5-bit-packed name encoding (each
Unicode code point in 0x3800-0x4840 represents two ASCII chars
from a 64-entry alphabet). ~290 lines.
CabExtractor — in-memory mspack_system adapter for libmspack.
Reads cab bytes from a QByteArray and writes decompressed
payloads to QByteArrays — no temp files. extractAll() returns
every file in the cab; extractFirstMatchingMagic() finds the
one that starts with a given magic (used to grab the .ssdr
blob and ignore sibling driver/installer files in the same
cab). ~190 lines.
FirmwareStager::extractFromMsi() — new body, ~80 lines (was 130).
No QProcess, no findExtractionTool(), no temp directory.
findExtractionTool() and the QProcess include are removed.
Verified end-to-end against SmartSDR_v4.2.18_x64.msi:
• Extracted FLEX-6x00 firmware MD5 = 9e8888dc0558ee420ed82f370f805025
matches the bytes that successfully flashed a real radio earlier
in this branch.
• All 6 cab streams extracted byte-identical to standalone 7z.
• cab1 has 14 files (drivers + FLEX-9600 firmware); the magic-match
filter correctly skips the drivers and picks the firmware blob.
New test: tests/ole_compound_file_test.cpp covers OLE CFB stream
extraction, MSI name decoding, libmspack CAB decompression, and the
end-to-end firmware-blob match against the known-good MD5. Skips
with code 77 if the licensed MSI fixture isn't on disk (looks at
$AETHERSDR_TEST_MSI then ~/build/reference/SmartSDR_v4.2.18_x64.msi).
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
system.h gates its inline memcmp/memset/strlen reimplementations behind #ifndef MSPACK_NO_DEFAULT_SYSTEM, so even MSPACK_NO_DEFAULT_SYSTEM=0 makes the compiler take the inline branch. On MSVC those collide with the compiler's built-in intrinsics: error C2169: 'memcmp': intrinsic function, cannot be defined error C2169: 'memset': intrinsic function, cannot be defined error C2169: 'strlen': intrinsic function, cannot be defined Removing the define entirely lets system.h take the default #include <string.h> branch, which is what we wanted in the first place. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Three coupled changes that make the firmware-update path work for SmartSDR v4.2+ without runtime dependencies:
7z, noQProcess, no temp files, no external runtime deps. Works in any sandboxing model that disallows process spawning.software_ver=4.2.18.41174was being mangled to*.*.*. 41174in logs) and removes one-shot diagnostic instrumentation that was added for the first flash attempt.Why native instead of
7zEarlier prototypes shelled out to
7zfor MSI extraction. That approach worked on Linux but would have created silent regressions for Windows users (overwhelmingly don't have 7z on PATH) and broken in sandboxed environments. The native pipeline:Pipeline
.exe(v4.1.x and earlier)MZ(PE/COFF) headerD0 CF 11 E0 A1 B1 1A E1.ssdrWhat landed
Vendored library
third_party/libmspack/— LGPL-2.1, GPL-3 compatiblethird_party/libmspack/CMakeLists.txtwith-wto keep upstream warning style out of our build output.third_party/libmspack/README.mddocuments version, license, file inventory, update procedure.New AetherSDR sources
src/core/OleCompoundFile.{h,cpp}— read-only [MS-CFB] reader. Parses header, FAT chain, directory; reads streams by name. MSI-aware accessor decodes Microsoft's 5-bit-packed name encoding. ~290 lines total.src/core/CabExtractor.{h,cpp}— in-memorymspack_systemadapter. Reads cab bytes from aQByteArrayand writes decompressed payloads toQByteArrays. ~190 lines.Changed AetherSDR sources
src/core/FirmwareStager.{h,cpp}—extractFromMsi()body fully rewritten (~130 → ~80 lines);findExtractionTool()deleted; newstageFromLocalFile()for the file-selection entry point; URL handling switches between_x64.msi(v4.2+) and_Installer.exe(older).src/gui/RadioSetupDialog.cpp— "Browse .ssdr..." renamed to "Select Installer..."; file dialog accepts.msi/.exe/.ssdr; "Check for Update" no longer auto-downloads, just reports the latest version.src/main.cpp—redactPiiregex extended to skip_ver=prefixed and trailing-digit cases.src/core/FirmwareUploader.cpp— diagnostic logging from the one-shot-flash session removed (the actual flash succeeded; the scaffolding isn't needed anymore).Tests
tests/ole_compound_file_test.cpp— 17 assertions covering OLE CFB stream extraction, MSI name decoding, libmspack CAB decompression, and the end-to-end firmware-blob match.$AETHERSDR_TEST_MSIthen~/build/reference/SmartSDR_v4.2.18_x64.msi). CI doesn't have the file, so the test no-ops there.Verification
End-to-end test against
SmartSDR_v4.2.18_x64.msi:4D 53 43 46Salted__header9e8888dc0558ee420ed82f370f805025— same bytes that successfully flashed a real FLEX-8600 earlier in this branchSalted__headerThe tracking issue is #2172 — can be closed on merge.
Acceptance criteria (all met)
OleCompoundFilereads streams from real MSIextractFromMsi()no longer callsQProcessor references 7-ZipfindExtractionTool()deleted.ssdrmatching the known-good MD5Test plan
ole_compound_file_testpasses (17 assertions)🤖 Generated with Claude Code