Skip to content

2.0.11-rc2: Bugfixes - #1674

Merged
tdewey-rpi merged 18 commits into
mainfrom
dev/tdewey/2.0.11-rc2-fixes
Aug 10, 2026
Merged

2.0.11-rc2: Bugfixes#1674
tdewey-rpi merged 18 commits into
mainfrom
dev/tdewey/2.0.11-rc2-fixes

Conversation

@tdewey-rpi

Copy link
Copy Markdown
Collaborator

No description provided.

@dutypro

dutypro commented Aug 5, 2026

Copy link
Copy Markdown

Any idea when we can expect this to be pushed?

@tdewey-rpi

Copy link
Copy Markdown
Collaborator Author

Any idea when we can expect this to be pushed?

There's one further fix that needs to land - a release process change that should make releasing future Raspberry Pi Imagers much simpler compared to the previous flows. I'm actively testing this, so once I've gained confidence, you should expect a -rc2 build.

elibosley added a commit to unraid/usb-creator-next that referenced this pull request Aug 10, 2026
2569eb3 previewed raspberrypi#1674. Reviewing that merge
surfaced defects in the imported commits; raspberrypi#1674 is still open at the SHA
we imported (8357122, untouched since 2026-08-05) and upstream/main has
not fixed any of them, so they are carried here rather than waited on.

Two are regressions against upstream/main, not pre-existing bugs:

- imagewriter.cpp: 65269fa swapped ZSTD_getFrameContentSize for
  ZSTD_findDecompressedSize and kept only a `fcs == 0` guard. The
  sentinels are 0ULL-1 and 0ULL-2, so ZSTD_CONTENTSIZE_UNKNOWN set
  _extrLen to ULLONG_MAX and startWrite() rejected a good local .img.zst
  as too large. main still checks both by name.
- file_operations_{linux,windows}.cpp: b70f908 made the direct-I/O
  reopen preserve O_EXCL. The handle is closed first, so losing that race
  now fails every attempt and aborts an in-flight write. OpenDevice()
  degrades to a shared open; the reopen path now does too. main's
  fallback was plain O_RDWR and degraded implicitly.

Pre-existing, also present in main:

- devicewrapper.cpp: ~DeviceWrapper() called a throwing sync(), and a
  destructor is noexcept — a failed flush called std::terminate(). 473bf48
  made this likelier by adding a Flush() throw on the pre-MBR path.
- file_operations_linux.cpp: OpenInternal() opens with Close(), which
  clears using_direct_io_, so the O_DIRECT buffered fallback tested a flag
  that was always false and never ran. The correct flag,
  direct_io_attempted_, was already there. The same reset also left
  IsDirectIOEnabled() false after a successful O_DIRECT open, which the
  new customisation read-back consults.

New in raspberrypi#1674:

- downloadthread.cpp: _verifyCustomisation() returned true on cancel, so
  _writeComplete() reported a cancelled write as success. Its failure path
  also emitted error() directly, skipping the _onDownloadError() route that
  refreshes the disk view after the partition table was rewritten.
- fastbootflashthread.cpp: performErase() checked _cancelled only after
  erase:, leaving partinit and partapp unguarded.
- fastboot: isRpiFastboot() collapsed a transport failure and a protocol
  refusal into false, and the poll thread cached that as a permanent
  storage-less entry — stranding a Pi whose gadget was still starting.
  Response gains TransportError to separate "said no" from "said nothing",
  and identifyRpiFastboot() returns ConfirmedPi/ConfirmedNotPi/Inconclusive.
  Writes require ConfirmedPi; only ConfirmedNotPi is cached.

Not carried: the 18d1:4e40 udev rule (narrowing it needs a Pi-specific
attribute or a dedicated VID/PID — upstream's call, and untestable here)
and the stale-mountpoint race after cleanDiskFast() (unproven, and the
remedy partly undoes the deliberate rescan removal for raspberrypi#1665). Both are in
the report going to raspberrypi#1674.
Before a write, unmountVolumes() dismounted each volume on the target
disk and then called DeleteVolumeMountPointW to stop Windows from
re-mounting/re-grabbing it during the clean + raw write. Nothing ever
restored that mapping, and there is no SetVolumeMountPoint anywhere in
the tree. On card readers that Windows treats as fixed disks (RMB=0),
the Mount Manager binding is persistent, so the reader came back with no
drive letter — reinserted cards no longer appeared in Explorer and the
letter had to be reassigned by hand after every write.

Replace the letter deletion with a lock-and-hold approach that matches
what the removed mountutils dependency effectively did: lock
(FSCTL_LOCK_VOLUME) and dismount each volume, then keep the handles open
via a new LockedVolumes RAII holder for the duration of prepare (unmount
-> clean -> open). Holding the lock keeps the volume unmounted across the
critical window without touching the persistent Mount Manager binding.
Once the physical drive is open (which suppresses partition re-scan), the
locks are released; when the write finishes and the physical-drive handle
closes, Windows re-scans and reassigns the drive letter normally.

Fixes #1665.
87b62f8 opens the target physical drive without FILE_SHARE_WRITE so the OS
volume manager can't grab it mid-write. But if a transient holder (Explorer
re-scanning the just-cleaned removable disk, an AV, or the search indexer) owns
the drive at open time, the previous code dropped straight to a shared open,
which lets Windows mount the partition mid-write. Retry the exclusive open with
geometric backoff (matching Rufus's DRIVE_ACCESS wait loop) and only fall back
to shared write if exclusivity genuinely cannot be obtained.

Also stop issuing FSCTL_ALLOW_EXTENDED_DASD_IO on physical-drive handles: it is
only meaningful for volume handles and returned ERROR_INVALID_FUNCTION on a
physical drive, logging a spurious "failed" warning on every write.

Note: exclusive access reduces but does not fully eliminate the removable-reader
"insert a disk" / "format the disk" shell dialogs. Those stem from Windows
auto-managing the drive letter while the disk is partition-less mid-write and
are only fully avoidable via mount-manager AutoMount suppression, which is
deliberately not used here because of its global, crash-persistent state.
…table

When writing a customised image, DeviceWrapper::sync() wrote the FAT blocks and
then the MBR, and the raw-write path wrote the deferred first block (containing
the MBR) after the image body — but with no device-cache flush in between. On
USB card readers the bridge/card write-cache can reorder writes, so the MBR can
reach the media before the filesystem is complete. Windows then briefly sees a
partition with an incomplete filesystem and prompts to format it. This never
reproduces on a virtual disk, which has no reordering write-cache.

Flush the device cache before writing the MBR in both the customised
(DeviceWrapper::sync) and non-customised (_writeComplete) paths, so the
partition only becomes visible once its contents are durably on media, and
flush again afterwards so the MBR itself is durable.

Crash-safe: touches no global OS state. A device that does not support flush
returns success from Flush(), so this is a no-op there. This is a write-ordering
correctness fix; on its own it does not eliminate the clean-phase removable
"insert a disk" dialog.
The fastboot gadget reuses Google's USB VID/PID (18d1:4e40), so matching
VID/PID alone cannot prove a device is a Raspberry Pi. A non-Pi device in a
colliding fastboot mode (e.g. an Android phone) could otherwise be listed as
a writable target and flashed.

Add positive identification in FastbootProtocol::isRpiFastboot():
  - primary: the USB interface string descriptor the rpi-fastbootd gadget
    advertises ("fastbootd-provisioner"), read at open time via the new
    IUsbTransport::interfaceString() (overridden by LibusbTransport);
  - fallback: the RPi-specific "block-devices" getvar, which stock Android
    fastboot does not implement.

Enforce the check at discovery (drive-list poll) and again before any
destructive command in the flash thread (erase / partition / flash), so a
non-Pi 18d1:4e40 device is never enumerated or written to.
stage() has always sent the "download:" verb, not "stage:", but two tests
and the header comment assumed "stage:", leaving the fastboot suite with two
long-standing failures.

This is correct behaviour, not a bug: rpi-fastbootd maps both "download" and
"stage" to the same DownloadHandler and keeps "stage" only for backward compat
with older imagers, while its restricted TCP data-plane command map accepts
only download/flash/getvar. "download" is therefore the canonical, portable
verb and must not be changed back to "stage".

Update the two tests to expect "download:", correct the stage() documentation,
and add a note at the call site so the verb is not "fixed" back. The fastboot
test suite is now fully green.
_verify() runs before _customizeImage(), so the customisation files are the one
part of the card nothing ever reads back. A device that acknowledges writes it
never commits therefore gives a green write whose settings are simply absent on
first boot, with no indication why.

Record a digest and size for each file written, re-read them after the final
sync, and fail on a mismatch. The read uses a fresh DeviceWrapper, as the
existing block cache does not evict on sync() and would otherwise compare our
own buffer against itself, and re-asserts direct I/O to bypass the page cache.
The device's own cache cannot be defeated, so this yields false passes but never
false failures; hence failing hard on a mismatch is safe.

boot.img is content-checked only when verification is enabled, size always.
Hashing goes through AcceleratedCryptographicHash, as the rest of the write path
does. Reported as a new customisation_verify event.

This also catches the swallowed writeFile() failure at the end of
_createSecureBootFiles(), which needs no faulty hardware.

Not yet exercised on real media.
@tdewey-rpi
tdewey-rpi force-pushed the dev/tdewey/2.0.11-rc2-fixes branch from 8357122 to dbe9034 Compare August 10, 2026 16:29
@tdewey-rpi
tdewey-rpi merged commit 14336ec into main Aug 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants