Remove incremental snapshots [dev]#1747
Merged
Merged
Conversation
badrishc
force-pushed
the
badrishc/no-incr-snapshot
branch
3 times, most recently
from
e51a6bc to
b2c0042
Compare
Contributor
There was a problem hiding this comment.
Pull request overview
This PR removes the incremental snapshot checkpoint feature across Tsavorite, Garnet server/cluster layers, tests, benchmarks, and docs, simplifying checkpoint/recovery paths by eliminating delta-log infrastructure and dirty-bit tracking.
Changes:
- Remove incremental snapshot/delta-log infrastructure (DeltaLog, incremental SM task, dirty-bit/page marking) and associated options/config/docs.
- Simplify Tsavorite checkpoint/recovery APIs and checkpoint metadata format (bump checkpoint version; remove
tryIncremental/recoverTo). - Update cluster replication checkpoint file handling and remove delta-log transmission; adjust tests/benchmarks accordingly.
Reviewed changes
Copilot reviewed 53 out of 53 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| website/docs/getting-started/configuration.md | Removes incremental snapshot CLI option from configuration docs. |
| website/docs/dev/range-index-resp-api.md | Updates RangeIndex documentation to reflect full-only checkpoints and removes delta-log references. |
| test/Garnet.test/TestUtils.cs | Removes incremental snapshot option wiring from test server helpers. |
| test/Garnet.test.cluster/VectorSets/ClusterVectorSetTests.cs | Drops incremental snapshot parameter from cluster setup helper usage. |
| test/Garnet.test.cluster/ReplicationTests/ClusterReplicationBaseTests.cs | Removes incremental snapshot parameterization from replication tests and related helpers. |
| test/Garnet.test.cluster/ClusterTestContext.cs | Removes incremental snapshot-related params/logic from cluster test context helpers. |
| test/Garnet.fuzz/Targets/GarnetEndToEnd.cs | Removes enabling incremental snapshots from fuzz target server configuration. |
| libs/storage/Tsavorite/cs/test/RecoveryCheckTests.cs | Deletes ignored incremental snapshot recovery test and removes related usings. |
| libs/storage/Tsavorite/cs/test/DeltaLogTests.cs | Deletes delta-log unit tests (feature removal). |
| libs/storage/Tsavorite/cs/src/devices/AzureStorageDevice/AzureCheckpointNamingScheme.cs | Removes delta-log naming for Azure checkpoint paths. |
| libs/storage/Tsavorite/cs/src/core/Index/Tsavorite/Tsavorite.cs | Removes tryIncremental and recoverTo from public Tsavorite APIs and related logic. |
| libs/storage/Tsavorite/cs/src/core/Index/Tsavorite/Implementation/InternalUpsert.cs | Removes page-dirty marking call sites for in-place updates. |
| libs/storage/Tsavorite/cs/src/core/Index/Tsavorite/Implementation/InternalRMW.cs | Removes page-dirty marking and replaces dirty+modified bit with modified-only. |
| libs/storage/Tsavorite/cs/src/core/Index/Tsavorite/Implementation/InternalDelete.cs | Removes page-dirty marking and replaces dirty+modified bit with modified-only for tombstoning. |
| libs/storage/Tsavorite/cs/src/core/Index/Tsavorite/Implementation/Helpers.cs | Removes MarkPage helper used for incremental snapshot dirty tracking. |
| libs/storage/Tsavorite/cs/src/core/Index/Recovery/Recovery.cs | Removes delta-log-aware recovery/metadata scanning and simplifies recovery helpers. |
| libs/storage/Tsavorite/cs/src/core/Index/Recovery/ICheckpointManager.cs | Removes incremental checkpoint APIs and simplifies log checkpoint metadata retrieval signature. |
| libs/storage/Tsavorite/cs/src/core/Index/Recovery/DeltaLog.cs | Deletes delta-log implementation (feature removal). |
| libs/storage/Tsavorite/cs/src/core/Index/Recovery/Checkpoint.cs | Removes last-snapshot tracking and incremental metadata commit/cleanup hooks. |
| libs/storage/Tsavorite/cs/src/core/Index/Common/RecordInfo.cs | Removes dirty bit and exposes SetModified() instead of dirty+modified mutations. |
| libs/storage/Tsavorite/cs/src/core/Index/Checkpointing/TsavoriteStateMachineProperties.cs | Removes incremental snapshot tail address property. |
| libs/storage/Tsavorite/cs/src/core/Index/Checkpointing/StreamingSnapshotCheckpointSMTask.cs | Removes last-snapshot disposal step tied to incremental snapshots. |
| libs/storage/Tsavorite/cs/src/core/Index/Checkpointing/StateMachineTaskType.cs | Removes incremental snapshot SM task type. |
| libs/storage/Tsavorite/cs/src/core/Index/Checkpointing/SnapshotCheckpointSMTask.cs | Disposes snapshot devices immediately instead of storing for incremental reuse. |
| libs/storage/Tsavorite/cs/src/core/Index/Checkpointing/SemaphoreWaiterMonitor.cs | Removes incremental snapshot SM semaphore type. |
| libs/storage/Tsavorite/cs/src/core/Index/Checkpointing/IncrementalSnapshotCheckpointSMTask.cs | Deletes incremental snapshot checkpoint SM task implementation. |
| libs/storage/Tsavorite/cs/src/core/Index/Checkpointing/FoldOverSMTask.cs | Removes last-snapshot disposal step tied to incremental snapshots. |
| libs/storage/Tsavorite/cs/src/core/Index/Checkpointing/Checkpoint.cs | Removes incremental checkpoint factory methods. |
| libs/storage/Tsavorite/cs/src/core/Index/CheckpointManagement/RecoveryInfo.cs | Bumps checkpoint version and removes delta-tail metadata from checkpoint format. |
| libs/storage/Tsavorite/cs/src/core/Index/CheckpointManagement/ICheckpointNamingScheme.cs | Removes delta-log naming scheme contract. |
| libs/storage/Tsavorite/cs/src/core/Index/CheckpointManagement/DeviceLogCommitCheckpointManager.cs | Removes delta-log metadata scanning/write paths and simplifies metadata retrieval. |
| libs/storage/Tsavorite/cs/src/core/Index/CheckpointManagement/DefaultCheckpointNamingScheme.cs | Removes delta-log naming from default scheme. |
| libs/storage/Tsavorite/cs/src/core/ClientSession/SessionFunctionsWrapper.cs | Replaces dirty+modified bit updates with modified-only in post-writer/updater hooks. |
| libs/storage/Tsavorite/cs/src/core/Allocator/TsavoriteLogAllocator.cs | Removes allocator MarkPage* forwarding APIs. |
| libs/storage/Tsavorite/cs/src/core/Allocator/SpanByteAllocator.cs | Removes allocator MarkPage* forwarding APIs. |
| libs/storage/Tsavorite/cs/src/core/Allocator/PageUnit.cs | Removes page-status dirty tracking field and updates ToString. |
| libs/storage/Tsavorite/cs/src/core/Allocator/ObjectAllocatorImpl.cs | Removes unsupported delta-flush override (no longer needed). |
| libs/storage/Tsavorite/cs/src/core/Allocator/ObjectAllocator.cs | Removes allocator MarkPage* forwarding APIs. |
| libs/storage/Tsavorite/cs/src/core/Allocator/IAllocator.cs | Removes MarkPage* methods from allocator interface. |
| libs/storage/Tsavorite/cs/src/core/Allocator/AllocatorBase.cs | Removes delta flush/apply-delta and dirty-page tracking logic. |
| libs/storage/Tsavorite/cs/benchmark/YCSB.benchmark/YcsbConstants.cs | Removes incremental checkpoint benchmark constant. |
| libs/storage/Tsavorite/cs/benchmark/YCSB.benchmark/SpanByteYcsbBenchmark.cs | Removes try-incremental argument from periodic checkpointing. |
| libs/storage/Tsavorite/cs/benchmark/YCSB.benchmark/Options.cs | Removes CLI option for incremental periodic checkpoints and updates option string formatting. |
| libs/storage/Tsavorite/cs/benchmark/YCSB.benchmark/ObjectYcsbBenchmark.cs | Removes try-incremental argument from periodic checkpointing. |
| libs/storage/Tsavorite/cs/benchmark/YCSB.benchmark/FixedLenYcsbBenchmark.cs | Removes try-incremental argument from periodic checkpointing. |
| libs/server/Servers/GarnetServerOptions.cs | Removes server options controlling incremental snapshots and delta log size limit. |
| libs/server/Databases/DatabaseManagerBase.cs | Removes incremental checkpoint selection logic and always uses full checkpoint state machine selection. |
| libs/host/defaults.conf | Removes incremental snapshot option from default config. |
| libs/host/Configuration/Options.cs | Removes incremental snapshot CLI option and server option wiring. |
| libs/cluster/Server/Replication/PrimaryOps/ReplicaSyncSession.cs | Removes delta-log segment sending and simplifies snapshot metadata retrieval. |
| libs/cluster/Server/Replication/GarnetClusterCheckpointManager.cs | Simplifies cookie metadata retrieval and removes delta-log metadata scanning override. |
| libs/cluster/Server/Replication/CheckpointStore.cs | Removes delta-log device/scan path when extracting checkpoint cookie metadata. |
| libs/cluster/Server/Replication/CheckpointFileType.cs | Removes STORE_DLOG and adjusts enum values/reservations for checkpoint file type identifiers. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
badrishc
force-pushed
the
badrishc/no-incr-snapshot
branch
4 times, most recently
from
23dd796 to
0448927
Compare
badrishc
force-pushed
the
badrishc/no-incr-snapshot
branch
6 times, most recently
from
19f692e to
08660aa
Compare
vazois
approved these changes
Apr 30, 2026
badrishc
force-pushed
the
badrishc/no-incr-snapshot
branch
from
08660aa to
b121227
Compare
Remove the incremental snapshot checkpoint feature which does not work with the object allocator (unified single-store design). This is a full removal including write paths, recovery paths, delta log infrastructure, dirty-bit tracking, and all related configuration/tests. Key changes: - Delete IncrementalSnapshotCheckpointSMTask.cs, DeltaLog.cs, DeltaLogTests.cs - Remove tryIncremental/recoverTo from public Tsavorite APIs - Remove _lastSnapshotCheckpoint field; dispose snapshot devices immediately - Bump CheckpointVersion 7->8 (deltaTailAddress removed from format) - Remove dirty-bit tracking (SetDirtyAndModified->SetModified, MarkPage removed) - Remove EnableIncrementalSnapshots/IncrementalSnapshotLogSizeLimit server options - Remove STORE_DLOG from cluster replication CheckpointFileType - Clean up ICheckpointManager interface (remove delta methods) - Remove incremental test parameterization and YCSB benchmark option 53 files changed, ~1455 net lines removed. Co-authored-by: Copilot <[email protected]>
badrishc
force-pushed
the
badrishc/no-incr-snapshot
branch
from
b121227 to
c28af9f
Compare
TedHartMS
approved these changes
Apr 30, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Remove the incremental snapshot checkpoint feature. This feature is complex, works only for raw strings, and is no longer as interesting with the unified object allocator (unified single-store design) coupled with the availability of the AOF for capturing incremental updates. This is a full removal including write paths, recovery paths, delta log infrastructure, dirty-bit tracking, and all related configuration/tests.
Key changes:
53 files changed, ~1455 net lines removed.