2.0.11-rc2: Bugfixes - #1674
Merged
Merged
Conversation
|
Any idea when we can expect this to be pushed? |
Collaborator
Author
There's one further fix that needs to land - a release process change that should make releasing future Raspberry Pi Imagers much simpler compared to the previous flows. I'm actively testing this, so once I've gained confidence, you should expect a -rc2 build. |
elibosley
added a commit
to unraid/usb-creator-next
that referenced
this pull request
Aug 10, 2026
2569eb3 previewed raspberrypi#1674. Reviewing that merge surfaced defects in the imported commits; raspberrypi#1674 is still open at the SHA we imported (8357122, untouched since 2026-08-05) and upstream/main has not fixed any of them, so they are carried here rather than waited on. Two are regressions against upstream/main, not pre-existing bugs: - imagewriter.cpp: 65269fa swapped ZSTD_getFrameContentSize for ZSTD_findDecompressedSize and kept only a `fcs == 0` guard. The sentinels are 0ULL-1 and 0ULL-2, so ZSTD_CONTENTSIZE_UNKNOWN set _extrLen to ULLONG_MAX and startWrite() rejected a good local .img.zst as too large. main still checks both by name. - file_operations_{linux,windows}.cpp: b70f908 made the direct-I/O reopen preserve O_EXCL. The handle is closed first, so losing that race now fails every attempt and aborts an in-flight write. OpenDevice() degrades to a shared open; the reopen path now does too. main's fallback was plain O_RDWR and degraded implicitly. Pre-existing, also present in main: - devicewrapper.cpp: ~DeviceWrapper() called a throwing sync(), and a destructor is noexcept — a failed flush called std::terminate(). 473bf48 made this likelier by adding a Flush() throw on the pre-MBR path. - file_operations_linux.cpp: OpenInternal() opens with Close(), which clears using_direct_io_, so the O_DIRECT buffered fallback tested a flag that was always false and never ran. The correct flag, direct_io_attempted_, was already there. The same reset also left IsDirectIOEnabled() false after a successful O_DIRECT open, which the new customisation read-back consults. New in raspberrypi#1674: - downloadthread.cpp: _verifyCustomisation() returned true on cancel, so _writeComplete() reported a cancelled write as success. Its failure path also emitted error() directly, skipping the _onDownloadError() route that refreshes the disk view after the partition table was rewritten. - fastbootflashthread.cpp: performErase() checked _cancelled only after erase:, leaving partinit and partapp unguarded. - fastboot: isRpiFastboot() collapsed a transport failure and a protocol refusal into false, and the poll thread cached that as a permanent storage-less entry — stranding a Pi whose gadget was still starting. Response gains TransportError to separate "said no" from "said nothing", and identifyRpiFastboot() returns ConfirmedPi/ConfirmedNotPi/Inconclusive. Writes require ConfirmedPi; only ConfirmedNotPi is cached. Not carried: the 18d1:4e40 udev rule (narrowing it needs a Pi-specific attribute or a dedicated VID/PID — upstream's call, and untestable here) and the stale-mountpoint race after cleanDiskFast() (unproven, and the remedy partly undoes the deliberate rescan removal for raspberrypi#1665). Both are in the report going to raspberrypi#1674.
Before a write, unmountVolumes() dismounted each volume on the target disk and then called DeleteVolumeMountPointW to stop Windows from re-mounting/re-grabbing it during the clean + raw write. Nothing ever restored that mapping, and there is no SetVolumeMountPoint anywhere in the tree. On card readers that Windows treats as fixed disks (RMB=0), the Mount Manager binding is persistent, so the reader came back with no drive letter — reinserted cards no longer appeared in Explorer and the letter had to be reassigned by hand after every write. Replace the letter deletion with a lock-and-hold approach that matches what the removed mountutils dependency effectively did: lock (FSCTL_LOCK_VOLUME) and dismount each volume, then keep the handles open via a new LockedVolumes RAII holder for the duration of prepare (unmount -> clean -> open). Holding the lock keeps the volume unmounted across the critical window without touching the persistent Mount Manager binding. Once the physical drive is open (which suppresses partition re-scan), the locks are released; when the write finishes and the physical-drive handle closes, Windows re-scans and reassigns the drive letter normally. Fixes #1665.
87b62f8 opens the target physical drive without FILE_SHARE_WRITE so the OS volume manager can't grab it mid-write. But if a transient holder (Explorer re-scanning the just-cleaned removable disk, an AV, or the search indexer) owns the drive at open time, the previous code dropped straight to a shared open, which lets Windows mount the partition mid-write. Retry the exclusive open with geometric backoff (matching Rufus's DRIVE_ACCESS wait loop) and only fall back to shared write if exclusivity genuinely cannot be obtained. Also stop issuing FSCTL_ALLOW_EXTENDED_DASD_IO on physical-drive handles: it is only meaningful for volume handles and returned ERROR_INVALID_FUNCTION on a physical drive, logging a spurious "failed" warning on every write. Note: exclusive access reduces but does not fully eliminate the removable-reader "insert a disk" / "format the disk" shell dialogs. Those stem from Windows auto-managing the drive letter while the disk is partition-less mid-write and are only fully avoidable via mount-manager AutoMount suppression, which is deliberately not used here because of its global, crash-persistent state.
…table When writing a customised image, DeviceWrapper::sync() wrote the FAT blocks and then the MBR, and the raw-write path wrote the deferred first block (containing the MBR) after the image body — but with no device-cache flush in between. On USB card readers the bridge/card write-cache can reorder writes, so the MBR can reach the media before the filesystem is complete. Windows then briefly sees a partition with an incomplete filesystem and prompts to format it. This never reproduces on a virtual disk, which has no reordering write-cache. Flush the device cache before writing the MBR in both the customised (DeviceWrapper::sync) and non-customised (_writeComplete) paths, so the partition only becomes visible once its contents are durably on media, and flush again afterwards so the MBR itself is durable. Crash-safe: touches no global OS state. A device that does not support flush returns success from Flush(), so this is a no-op there. This is a write-ordering correctness fix; on its own it does not eliminate the clean-phase removable "insert a disk" dialog.
The fastboot gadget reuses Google's USB VID/PID (18d1:4e40), so matching
VID/PID alone cannot prove a device is a Raspberry Pi. A non-Pi device in a
colliding fastboot mode (e.g. an Android phone) could otherwise be listed as
a writable target and flashed.
Add positive identification in FastbootProtocol::isRpiFastboot():
- primary: the USB interface string descriptor the rpi-fastbootd gadget
advertises ("fastbootd-provisioner"), read at open time via the new
IUsbTransport::interfaceString() (overridden by LibusbTransport);
- fallback: the RPi-specific "block-devices" getvar, which stock Android
fastboot does not implement.
Enforce the check at discovery (drive-list poll) and again before any
destructive command in the flash thread (erase / partition / flash), so a
non-Pi 18d1:4e40 device is never enumerated or written to.
stage() has always sent the "download:" verb, not "stage:", but two tests and the header comment assumed "stage:", leaving the fastboot suite with two long-standing failures. This is correct behaviour, not a bug: rpi-fastbootd maps both "download" and "stage" to the same DownloadHandler and keeps "stage" only for backward compat with older imagers, while its restricted TCP data-plane command map accepts only download/flash/getvar. "download" is therefore the canonical, portable verb and must not be changed back to "stage". Update the two tests to expect "download:", correct the stage() documentation, and add a note at the call site so the verb is not "fixed" back. The fastboot test suite is now fully green.
_verify() runs before _customizeImage(), so the customisation files are the one part of the card nothing ever reads back. A device that acknowledges writes it never commits therefore gives a green write whose settings are simply absent on first boot, with no indication why. Record a digest and size for each file written, re-read them after the final sync, and fail on a mismatch. The read uses a fresh DeviceWrapper, as the existing block cache does not evict on sync() and would otherwise compare our own buffer against itself, and re-asserts direct I/O to bypass the page cache. The device's own cache cannot be defeated, so this yields false passes but never false failures; hence failing hard on a mismatch is safe. boot.img is content-checked only when verification is enabled, size always. Hashing goes through AcceleratedCryptographicHash, as the rest of the write path does. Reported as a new customisation_verify event. This also catches the swallowed writeFile() failure at the end of _createSecureBootFiles(), which needs no faulty hardware. Not yet exercised on real media.
tdewey-rpi
force-pushed
the
dev/tdewey/2.0.11-rc2-fixes
branch
from
August 10, 2026 16:29
8357122 to
dbe9034
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.