Skip to content

serve restic: return 500 not 404 when delete object lookup fails - fixes #10100 - #10109

Merged
ncw merged 1 commit into
rclone:masterfrom
bunnysayzz:fix/serve-restic-delete-500-10100
Oct 18, 2026
Merged

ncw merged 1 commit into
rclone:masterfrom
bunnysayzz:fix/serve-restic-delete-500-10100

Conversation

@bunnysayzz

Copy link
Copy Markdown
Contributor

What does this change do?

rclone serve restic replied 404 Not Found when a DELETE request's object lookup failed with any error other than not-found, while a GET with the same lookup failure replied 500 Internal Server Error (serveObject() in the same file only maps fs.ErrorObjectNotFound to 404).

restic treats a 404 on delete as final and never retries. So a transient lookup failure while restic removed its lock (seen in production with a Backblaze B2 remote: a temporary B2 error during lock removal) left the lock in the repository, and the next exclusive operation failed with "repository is already locked".

This change makes deleteObject() use the same 404/500 split as serveObject(): 404 only for fs.ErrorObjectNotFound, 500 for everything else, so restic retries the delete.

Linked issue

Fixes #10100

Checklist

  • This change is trivial OR it has been discussed and agreed in the linked issue.
  • I have read the contribution guidelines.
  • (If I used AI tools to help write this code) I have read and understood the AI-assisted contributions guidance, and I have tested and take ownership of this change myself.
  • I have added tests for all changes in this PR if appropriate.

deleteObject() replied 404 for any error from newObject(), while
serveObject() in the same file replies 404 only for
fs.ErrorObjectNotFound and 500 otherwise. restic treats a 404 on
DELETE as final and never retries, so a transient lookup failure
while removing a lock left the lock behind and broke the next
exclusive operation (eg forget --prune failing with "repository is
already locked"). Mirror the serveObject() 404/500 split in
deleteObject().

Fixes rclone#10100
@hafizahmedzia

Copy link
Copy Markdown

Tested your existing change at 09a66319 with the actual restic 0.19.1 client against an empty disposable local repository. The fixture compiles Rclone's server code, disables object caching as in #10100, and injects one transient NewObject failure when restic deletes its lock.

  • Base 4c52ae05: one DELETE 404, no retry, one lock left. The first restic check exits 0 with an unlock warning; the next exclusive check fails with exit 11 (“repository is already locked”).
  • PR head: DELETE 500 → 200, restic logs one retry and recovery, no lock left, and the next exclusive check succeeds.
  • True not-found control, with the disposable lock physically removed before lookup: both revisions return one 404 without retry, leave no lock, and allow the next check to succeed. Restic still prints an unlock warning for that 404.

Results were consistent across four runs per case. This verifies client retry and lock cleanup beyond the handler-status test. It uses the local backend and a synthetic one-shot lookup fault, not B2; I haven't run the full Rclone suite. No backup data was needed.

@ncw ncw left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the fix @bunnysayzz - very nice :-)

@ncw
ncw merged commit 0677b0c into rclone:master Oct 18, 2026
7 of 9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

serve restic: DELETE replies 404 when the object lookup fails, so restic leaves its lock behind

3 participants