Skip to content

*: lower default GC TTL #89233

@irfansharif

Description

@irfansharif

This is tracking issue to lower the default GC TTL from 25h. There's an internal document that more fully spells out some of the motivations and requisite steps, the text for which was copied over below. There's also this internal slack thread.

Is your feature request related to a problem? Please describe.

The default GC TTL today is 25h, which means overwritten values in the last 25h window are retained. This translates to higher storage use and for outbox-like workloads where rows are deleted frequently, can make for costlier reads CPU wise since they have to scan over a larger number of overwritten values to get to the one of interest (this is not fundamental). Since all versions of key are stored within a single range (we split along key boundaries, not within different values of a key), we’ve seen incidents where we have large (>> 512 MB) unsplittable ranges causing cluster instability: snapshot timeouts all get triggered, other snapshots get queued behind it, range starts backpressuring writes (can be indistinguishable from an outage).

We chose a default of 25h originally to accommodate daily incremental backups with revision history. They fail (informatively) if the data you are trying to backup was GC-ed, which happens when incremental backups are taken less frequently than the GC periods for any of the objects in the base backup. In 22.2 however scheduled backups “chain together” protected timestamp records, which lets your scheduled backup protect only what’s needed and ensure coverage of revision history across each incremental backup. The short of it is we no longer need a 25h default, hence this issue.

Describe the solution you'd like

Lower it to something like 4h or 1.5h.

Jira issue: CRDB-20144

Metadata

Metadata

Assignees

Labels

C-enhancementSolution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)T-kvKV Team

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions