Skip to content

[Enhancement]: Retention can delete an incremental chain's FULL base if its metadata sidecar is unreadable #206

Open

Description

@X1pheR

Problem

Incremental backup retention can delete the FULL archive of a chain while retaining newer incremental archives from that same chain. Those retained increments are unusable without the base. This was observed on a live DBackup backup destination after changing Smart GFS from 2/2/1/1 to 2/1/1/0. An independent destination still contains the complete chain.

Evidence / conditions

  • The destination's retention log reported 15 of 17 backup metadata sidecars supplied valid creation timestamps.
  • The affected FULL archive fell back to its storage modification time; later incremental members correctly had a chain.id and metadata timestamps.
  • Retention reported deletion of the old FULL while preserving multiple incremental archives referring to it.
  • src/lib/runner/steps/retention-sidecars.ts currently catches sidecar read/parse failures and leaves a backup unlocked and without chainId, while 05-retention.ts applies retention regardless of incomplete metadata.
  • The existing chain-preservation logic can only protect files sharing an observed chainId; it cannot correlate a chainless FULL with incremental children.

Expected behavior

For an incremental job, fail closed for that destination when any listed backup's sidecar cannot establish a valid chain ID and creation time, or when a chain lacks its FULL archive. Do not delete any backup/sidecar under uncertainty. Surface a clear warning/error in the execution logs while keeping the newly uploaded backup usable.

Reproduction

  1. Create an incremental chain (one FULL, several incrementals), each with a .meta.json sidecar.
  2. Make only the FULL sidecar absent or unreadable; retain valid incremental sidecars.
  3. Use a short GFS policy causing the older FULL to fall outside selected buckets, with an incremental snapshot still selected.
  4. Observe that the FULL is deleted and its increments remain; then attempt to restore an increment.

A regression test should prove neither the FULL nor its increments can be deleted when even one metadata sidecar is invalid/missing; also cover pre-existing orphaned incrementals. Production workaround is retaining a more conservative default and not using short retention until the guard is deployed. A targeted upstream PR is being prepared.

Activity

  1. self-assigned this
    on Nov 8, 2026
  2. Skyfay commented on Nov 8, 2026

    @Skyfay
    Owner

    Hey @X1pheR

    Thanks for the report. Agree, should be improved. More in #208

  3. added a commit that references this issue on Nov 10, 2026
    152b291
  4. changed the title [-]Retention can delete an incremental chain's FULL base if its metadata sidecar is unreadable[/-] [+][Enhancement]: Retention can delete an incremental chain's FULL base if its metadata sidecar is unreadable[/+] on Nov 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

enhancementAn improvement to an existing feature

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions