Repository navigation
fix(server): never unlink an untracked-file path that an asset now references - #31074
danieldietzler merged 5 commits into
Conversation
…ferences An untracked_file report only records a path. If an asset starts referencing that path afterwards, which a large upload racing an integrity scan can produce, nothing invalidates the report: handleUntrackedRefresh only checks that the file still exists on disk, so Refresh and Check both leave it listed. Deleting it then unlinks the original of a live asset, and the next scan reports the same path as missing. Reference checking moves into getTrackedPaths, and the refresh job and both delete paths now use it. A stale report is dropped instead of acted on, and a path an asset, asset file or person thumbnail references is never unlinked.
getTrackedPaths passed an empty list straight into the `in` clauses, which Postgres rejects with a syntax error. handleUntrackedRefresh is called with no items when there is nothing to refresh, so the job failed instead of succeeding.
danieldietzler
left a comment
There was a problem hiding this comment.
I would prefer a simple DB lookup in handleUntrackedRefresh, line 316, over what you have done here
Replaces the getTrackedPaths service helper and its use in the two delete paths with a single repository lookup inside handleUntrackedRefresh, per review. handleUntrackedFiles, deleteIntegrityReport and handleDeleteIntegrityReports are back to what main had. IntegrityRepository.getTrackedPaths unions the asset original path, the asset file path and the person thumbnail path in one statement, so the refresh costs one query rather than three.
|
Done in
One thing worth your call rather than mine: dropping the guard in 28 checks pass, including SQL Schema Checks and the medium tests; |
danieldietzler
left a comment
There was a problem hiding this comment.
This seems good to me, thanks!
|
Backport-action backported this pull request in workflow run 33515801315.
|
…ferences (#31194) fix(server): never unlink an untracked-file path that an asset now references (#31074) (cherry picked from commit c1f2756) Co-authored-by: Aditya Raj Singh <[email protected]>
* fix: run codeql and weblate checks on release branches (immich-app#31066) fix: run codeql and weblate checks on release branches (immich-app#31065) (cherry picked from commit bbf201d) Co-authored-by: bo0tzz <[email protected]> * fix: build release branch images instead of re-tagging main (immich-app#31064) fix: build release branch images instead of re-tagging main (immich-app#31063) (cherry picked from commit 5baf181) Co-authored-by: bo0tzz <[email protected]> * fix: avoid a concurrency deadlock when drafting a release (immich-app#31070) fix: avoid a concurrency deadlock when drafting a release (immich-app#31069) (cherry picked from commit b912dcf) Co-authored-by: bo0tzz <[email protected]> * chore: version v3.2.0-rc.1 * fix(web): face editor coordinates on a not-yet-loaded video (immich-app#31085) fix(web): face editor coordinates on a not-yet-loaded video (immich-app#31083) (cherry picked from commit 1a8fcf1) Co-authored-by: bo0tzz <[email protected]> * fix: generate release notes from the previous release on the line (immich-app#31094) fix: generate release notes from the previous release on the line (immich-app#31080) (cherry picked from commit 469a870) Co-authored-by: bo0tzz <[email protected]> * chore(deps): update base-image to v202608300913 (immich-app#31129) chore(deps): update base-image to v202608300913 (immich-app#31128) (cherry picked from commit 1fd3d87) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * fix: do not move faces of users other than the current owner (immich-app#31149) fix: do not move faces of users other than the current owner (immich-app#31145) (cherry picked from commit 0628f8b) Co-authored-by: Daniel Dietzler <[email protected]> * fix(server): allow an empty assetIds array when creating an album shared link (immich-app#31151) fix(server): allow an empty assetIds array when creating an album shared link (immich-app#31098) (cherry picked from commit 9e5dcc5) Co-authored-by: Ufuk <[email protected]> * chore: tune map icon size (immich-app#31150) chore: tune map icon size (immich-app#31108) (cherry picked from commit 015e5f2) Co-authored-by: Alex <[email protected]> * fix(mobile): keep the original filename when sharing downloaded assets (immich-app#31154) fix(mobile): keep the original filename when sharing downloaded assets (immich-app#30267) * fix(mobile): keep the original filename when sharing downloaded assets * use ordinal suffixes for duplicate share filenames * rename display name maps to totals and duplicates seen * drop the display name totals map and keep the first copy name clean * fold the ordinal suffix into the share display name derivation * resolve share names after download and test through the public flow * use the renamed store repository in the share tests * add docstrings to the share file helpers * simplify the share name resolver to a single set * note the share_plus cache folder behavior on the resolver * drop the version pin from the resolver comment * pin the share_plus version in the resolver comment * undo the accidental reformat, keep the version pin * explain the test harness and cut the teardown and wait noise * say why the storage is stubbed, drop the platform override and busy wait for the share cleanup * spy on the cleanup in the share tests instead of waiting for it (cherry picked from commit 006e7fd) Co-authored-by: Santo Shakil <[email protected]> * fix(mobile): handle transient loading states for map timelines (immich-app#31153) fix(mobile): handle transient loading states for map timelines (immich-app#29735) * fix(mobile): handle transient loading states for map timelines * fix(mobile): handle transient loading states for map timelines * Add support for new method * Rewrote map options notifier to be more Riverpody (cherry picked from commit dc813e2) Co-authored-by: Adam Gastineau <[email protected]> * fix(web): interaction with some filter elements closes the search panel (immich-app#31160) fix(web): interaction with some filter elements closes the search panel (immich-app#31107) (cherry picked from commit 9dead78) Co-authored-by: Alex <[email protected]> * fix(ml): read CLIP model configs as UTF-8 (immich-app#31152) fix(ml): read CLIP model configs as UTF-8 (immich-app#31075) (cherry picked from commit c1a88dd) Co-authored-by: Aditya Raj Singh <[email protected]> * fix(mobile): refresh server info when the websocket connects (immich-app#31155) fix(mobile): refresh server info when the websocket connects (immich-app#31144) refresh the server features and config when the websocket connects (cherry picked from commit c17a119) Co-authored-by: Santo Shakil <[email protected]> * fix: open an announcement discussion for each release line (immich-app#31163) fix: open an announcement discussion for each release line (immich-app#31101) (cherry picked from commit 5666d57) Co-authored-by: bo0tzz <[email protected]> * chore: version v3.2.0-rc.2 * chore(deps): update github-actions (immich-app#31192) chore(deps): update github-actions (immich-app#31182) (cherry picked from commit 40e6abc) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * fix: add discussions perm to release publish token (immich-app#31193) fix: add discussions perm to release publish token (immich-app#31167) (cherry picked from commit 9e1e595) Co-authored-by: bo0tzz <[email protected]> * fix(server): never unlink an untracked-file path that an asset now references (immich-app#31194) fix(server): never unlink an untracked-file path that an asset now references (immich-app#31074) (cherry picked from commit c1f2756) Co-authored-by: Aditya Raj Singh <[email protected]> * fix: fdroid link (immich-app#31221) fix: fdroid link (immich-app#31219) (cherry picked from commit a68937c) Co-authored-by: Jason Rasmussen <[email protected]> * chore: require number format (immich-app#31226) chore: require number format (immich-app#31222) (cherry picked from commit 2c53b76) Co-authored-by: Jason Rasmussen <[email protected]> * fix: don't force builds on release-base PRs (immich-app#31231) fix: don't force builds on release-base PRs (immich-app#31225) (cherry picked from commit b48266c) Co-authored-by: bo0tzz <[email protected]> * fix: incorrect edit's openapi type (immich-app#31233) fix: incorrect edit's openapi type (immich-app#31218) * fix: incorrect edit's openapi type * patch * remove type changes (cherry picked from commit fa8a191) Co-authored-by: Alex <[email protected]> * fix(web): album date range formatting (immich-app#31238) fix(web): album date range formatting (immich-app#28564) (cherry picked from commit da8131d) Co-authored-by: Mees Frensel <[email protected]> * fix(mobile): switch to new OpenAPI int in RotateParameters (immich-app#31248) fix(mobile): switch to new OpenAPI int in RotateParameters (immich-app#31246) (cherry picked from commit 6d85f20) Co-authored-by: Adam Gastineau <[email protected]> * fix: face detection of edited assets (immich-app#31243) fix: face detection of edited assets (immich-app#31240) (cherry picked from commit 399217b) Co-authored-by: Daniel Dietzler <[email protected]> * chore(mobile): log app resume and pause (immich-app#31224) chore(mobile): log app resume and pause (immich-app#31207) (cherry picked from commit 7211efa) Co-authored-by: Santo Shakil <[email protected]> * fix(mobile): prevent inner mutability on Freezed classes (immich-app#31251) fix(mobile): prevent inner mutability on Freezed classes (immich-app#31229) * fix(mobile): prevent inner mutability on Freezed classes * Minor fixes (cherry picked from commit 8e04e6e) Co-authored-by: Adam Gastineau <[email protected]> * fix(web): partner sharing timeline (immich-app#31252) fix(web): partner sharing timeline (immich-app#31241) * fix(web): prevent websocket assets from leaking into scoped timelines * test: add 1 more test case to make sure the changes didnt break the logic (cherry picked from commit 040ae6b) Co-authored-by: Yohanes Kirana <[email protected]> * fix(mobile): refresh the memory lane after resume (immich-app#31260) fix(mobile): refresh the memory lane after resume (immich-app#31239) (cherry picked from commit b1a9368) Co-authored-by: Santo Shakil <[email protected]> * chore: version v3.2.0-rc.3 * fix: memory page navigation (immich-app#31368) fix: memory page navigation (immich-app#31334) (cherry picked from commit d2245c7) Co-authored-by: Daniel Dietzler <[email protected]> * fix(server): live photo transcode visibility (immich-app#31374) fix(server): live photo transcode visibility (immich-app#31319) * allow live photos to be transcoded on queue trigger * new gen sql * add test to check hidden assets are returned (cherry picked from commit 58fb1ed) Co-authored-by: Jake Bybee <[email protected]> * fix(deps): update dependency maplibre-gl to v6 [security] (immich-app#31375) * BACKPORT-CONFLICT * fix: pnpm lockfile conflicts --------- Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: Daniel Dietzler <[email protected]> * fix(web): use RTL-friendly layout for people panel buttons (immich-app#31378) fix(web): use RTL-friendly layout for people panel buttons (immich-app#31266) (cherry picked from commit 95bfed1) Co-authored-by: Mees Frensel <[email protected]> * fix(web): word-wrap long album names (immich-app#31379) fix(web): word-wrap long album names (immich-app#31265) (cherry picked from commit d7ae4ae) Co-authored-by: Mees Frensel <[email protected]> * fix(mobile): hide negative age (immich-app#31381) fix(mobile): hide negative age (immich-app#31341) (cherry picked from commit 9563309) Co-authored-by: Yaros <[email protected]> * fix(mobile): make shared link download toggle depend on metadata toggle (immich-app#31382) fix(mobile): make shared link download toggle depend on metadata toggle (immich-app#31264) (cherry picked from commit 31943fb) Co-authored-by: Arthur <[email protected]> Co-authored-by: Arthur <[email protected]> * chore: version v3.2.0 --------- Co-authored-by: immich-push-o-matic[bot] <179150890+immich-push-o-matic[bot]@users.noreply.github.com> Co-authored-by: bo0tzz <[email protected]> Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: Daniel Dietzler <[email protected]> Co-authored-by: Ufuk <[email protected]> Co-authored-by: Alex <[email protected]> Co-authored-by: Santo Shakil <[email protected]> Co-authored-by: Adam Gastineau <[email protected]> Co-authored-by: Aditya Raj Singh <[email protected]> Co-authored-by: Jason Rasmussen <[email protected]> Co-authored-by: Mees Frensel <[email protected]> Co-authored-by: Yohanes Kirana <[email protected]> Co-authored-by: Jake Bybee <[email protected]> Co-authored-by: Daniel Dietzler <[email protected]> Co-authored-by: Yaros <[email protected]> Co-authored-by: Arthur <[email protected]> Co-authored-by: Arthur <[email protected]> Co-authored-by: Claude Opus 5 <[email protected]>
Description
Fixes #30933
Problem
An
untracked_filereport stores only a path. If an asset starts referencing that path after the report is created, Refresh previously kept the stale report because it checked only whether the file existed. Deleting the stale report then unlinked the original of a live asset, and the next integrity scan reported that same path as missing.Changes
The normal untracked scan keeps its existing query shape.
User impact
Refreshing or deleting a stale untracked report no longer removes a file that Immich currently references. Genuinely untracked files are still deleted as before.
Verification
pnpm --filter immich test src/services/integrity.service.spec.ts --run— 7/7 passed.pnpm run check— passed.--max-warnings 0— passed.git diff --check upstream/main...HEAD— passed.The focused regression coverage exercises tracked-path Refresh, single deletion, and mixed batch deletion. The existing medium tests continue to cover unlinking a genuinely untracked path.
Checklist:
src/services/uses repositories implementations for database calls, filesystem operations, etc.src/repositories/is pretty basic/simple and does not have any immich specific logic (that belongs insrc/services/)Please describe to which degree, if any, an LLM was used in creating this pull request.
An LLM assisted with tracing the job and delete paths through the service and with drafting the tests. The diagnosis, final query shape, delete-time guards, and verification were reviewed by the author.