Skip to content

Min 4k TKV log page size; revise device sector size setting; DefaultInitialIORecordSize overridable by operation, session, or store#1853

Merged
badrishc merged 24 commits into
mainfrom
tedhar/io-4k
Jun 11, 2026
Merged

Min 4k TKV log page size; revise device sector size setting; DefaultInitialIORecordSize overridable by operation, session, or store#1853
badrishc merged 24 commits into
mainfrom
tedhar/io-4k

Conversation

@TedHartMS

@TedHartMS TedHartMS commented Jun 4, 2026

Copy link
Copy Markdown
Contributor

This pull request introduces a configurable option to override the default initial IO record size used when reading records from disk, allowing users to tune disk read behavior for different workloads. The change propagates this setting from configuration files and server options down to the storage engine and replaces the previous hardcoded constant with a default value that can be overridden at the store, session, or operation level. Additional refactoring improves naming and documentation for clarity.

Configuration and Settings Changes:

  • Added a new InitialIORecordSize option to Options, GarnetServerOptions, and the default configuration file, allowing users to specify the initial read size for records on disk. This setting accepts memory size strings (e.g., "4k", "8k") and defaults to 32 bytes if not set. [1] [2] [3]
  • Updated the settings propagation so that InitialIORecordSize is passed through from configuration to internal KVSettings, including parsing and validation logic. [1] [2] [3]

Storage Engine and API Changes:

  • Replaced the old IStreamBuffer.InitialIOSize constant with IStreamBuffer.DefaultInitialIORecordSize, and updated all usages and documentation for clarity and flexibility. [1] [2] [3] [4] [5]
  • Extended the client session API to allow specifying InitialIORecordSize at the session and per-operation level, ensuring that the IO record size can be fine-tuned for different access patterns. [1] [2] [3] [4] [5]
  • Increased the minimum log page size from 512 bytes to 4kb and revised tests accordingly, which in many cases required changing expected log addresses, object counts, etc.
  • Modified the device sector size setting to be the max of 512 or the device's reported logical sector size.

Garnet test fixes:

  • Fixed intermittent hang (due to LASTSAVE having only one-second resolution) when waiting for checkpoint completion.

Internal Refactoring and Documentation:

  • Improved naming and documentation throughout the storage engine to clarify the purpose and usage of the initial IO record size, and added a utility method to compute expected record sizes for IO. [1] [2] [3] [4]

These changes make the system more flexible and performant by allowing better control over disk IO behavior, which can be critical for optimizing latency and throughput under different workloads.

TedHartMS added 11 commits May 22, 2026 18:56
…ified otherwise, and clamp to end of page. Reduce IStreamBuffer.DefaultInitialIORecordSize to 32 and add ReadOptions and RMWOptions to add InitialIORecordSize to override this, and also add store-level and session-level overrides. Update local storage device variants to look for the max of logical and physical sector size.
…ceSectorSize to 512b (which required fixing up some tests that were using MinDeviceSectorSize instead of MinKVPageSize)

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR makes Tsavorite’s initial on-disk record read size configurable (store/session/per-operation), updates device-sector sizing to clamp to a minimum of 512 bytes, and raises the minimum Tsavorite log page size to 4KB—then adjusts Garnet/Tsavorite test expectations accordingly.

Changes:

  • Add InitialIORecordSize configuration plumbed from Garnet options/defaults into KVSettings, and into Tsavorite session/operation options used by disk reads/RMW.
  • Increase Tsavorite minimum log page size to 4KB (and minimum log memory size accordingly), updating many tests that depend on page/address math.
  • Refactor sector-size determination to use a minimum sector size constant and updated device implementations/tests.

Reviewed changes

Copilot reviewed 82 out of 82 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
test/standalone/Garnet.test/TestUtils.cs Introduces shared 4KB min page-size constants for tests; adjusts low-memory defaults; increases flush filler default.
test/standalone/Garnet.test/RespTests.cs Adjusts async test payload sizing to exercise disk behavior with larger pages.
test/standalone/Garnet.test/RespConfigTests.cs Updates config utilization tests for 4KB min page size and revised thresholds/expectations.
test/standalone/Garnet.test/RespAdminCommandsTests.cs Updates save/recover tests to use new min page size and revised memory sizing.
test/standalone/Garnet.test/GarnetServerConfigTests.cs Adds parsing tests for InitialIORecordSize CLI/JSON and default behavior.
test/standalone/Garnet.test.scripting/RespAofTests.cs Updates AOF oversized-entry test for new page-size constraints.
test/standalone/Garnet.test.rangeindex/RespRangeIndexTests.cs Increases filler writes to ensure eviction with larger pages.
test/standalone/Garnet.test.extensions/ReadCacheTests.cs Adjusts read-cache tests for 4KB pages and new tail-address expectations.
test/standalone/Garnet.test.extensions/CacheSizeTrackerTests.cs Recomputes eviction math/expectations for 4KB pages and larger targets.
test/standalone/Garnet.test.complexstring/HyperLogLogTests.cs Updates low-memory test sizes/page sizes to match new minimums.
test/standalone/Garnet.test.complexstring/GarnetBitmapTests.cs Forces bitmap LTM tests to use minimum 4KB page size.
test/standalone/Garnet.test.collections/RespSortedSetTests.cs Increases key counts to ensure LTM behavior with larger pages.
test/standalone/Garnet.test.collections/RespHashTests.cs Increases key counts to ensure LTM behavior with larger pages.
test/cluster/Garnet.test.cluster/ClusterTestUtils.cs Bumps flush filler default to match updated standalone test helper.
libs/storage/Tsavorite/cs/test/TestUtils.cs Adds shared 4KB min-page constants; updates LocalMemoryDevice construction signature usage.
libs/storage/Tsavorite/cs/test/test.session/ReadCacheChainTests.cs Updates page/read-cache page sizes to new minimum.
libs/storage/Tsavorite/cs/test/test.session/NativeReadCacheTests.cs Updates page/read-cache sizes and improves assertion diagnostics.
libs/storage/Tsavorite/cs/test/test.session.context/UnsafeContextTests.cs Updates smoke tests to use 4KB pages and larger in-memory sizes.
libs/storage/Tsavorite/cs/test/test.recovery/RecoveryTests.cs Updates LocalMemoryDevice usage and sector-size mismatch test setup/message.
libs/storage/Tsavorite/cs/test/test.recovery/RecoveryCheckTests.cs Updates page sizes and related log/segment sizes for new minimums.
libs/storage/Tsavorite/cs/test/test.recovery/RecoverReadOnlyTest.cs Updates TsavoriteLog settings to use new min page/memory sizes.
libs/storage/Tsavorite/cs/test/test.recovery/ObjectTests.cs Adjusts disk-write/read test sizes and assertions for larger pages.
libs/storage/Tsavorite/cs/test/test.recovery/ObjectRecoveryTest3.cs Updates log memory/page sizes to new minimum.
libs/storage/Tsavorite/cs/test/test.recovery/ObjectRecoveryTest2.cs Updates log memory/page sizes to new minimum.
libs/storage/Tsavorite/cs/test/test.recovery/ObjectReadCacheTests.cs Updates page and read-cache page sizes.
libs/storage/Tsavorite/cs/test/test.recovery/ObjectLogScanTests.cs Updates page size for scan tests.
libs/storage/Tsavorite/cs/test/test.recovery/ObjectLogCompactionTests.cs Updates memory/page sizing constants for compaction tests.
libs/storage/Tsavorite/cs/test/test.recovery/ObjectIterationTests.cs Updates memory/page sizes and LocalMemoryDevice ctor usage.
libs/storage/Tsavorite/cs/test/test.recovery/ObjectInlineTests.cs Updates page size to new minimum.
libs/storage/Tsavorite/cs/test/test.recovery/ObjectDiskDeleteTests.cs Updates page size to new minimum.
libs/storage/Tsavorite/cs/test/test.recovery/CheckpointManagerTests.cs Updates page size to new minimum.
libs/storage/Tsavorite/cs/test/test.recordops/RecordLifecycleTests.cs Updates page/read-cache page sizes to new minimum.
libs/storage/Tsavorite/cs/test/test.recordops/DeleteDisposeTests.cs Updates page size to new minimum.
libs/storage/Tsavorite/cs/test/test.hlog/SpanByteTests.cs Updates page-size assumptions and record-fill math for scanning tests.
libs/storage/Tsavorite/cs/test/test.hlog/SpanByteLogCompactionTests.cs Updates page size to new minimum.
libs/storage/Tsavorite/cs/test/test.hlog/SpanByteIterationTests.cs Updates page size to new minimum.
libs/storage/Tsavorite/cs/test/test.hlog/MoreLogCompactionTests.cs Updates page size to new minimum.
libs/storage/Tsavorite/cs/test/test.hlog/LogShiftTailStressTest.cs Updates LocalMemoryDevice ctor usage.
libs/storage/Tsavorite/cs/test/test.hlog/LogRecoverReadOnlyTests.cs Updates TsavoriteLog settings for new min page/memory sizes.
libs/storage/Tsavorite/cs/test/test.hlog/DeviceTests.cs Updates sector-size-dependent constants and device construction sites.
libs/storage/Tsavorite/cs/test/test.hlog/DeviceLogTests.cs Updates LocalMemoryDevice ctor usage.
libs/storage/Tsavorite/cs/test/test.hlog/BasicStorageTests.cs Updates sharded-device sector size and page size.
libs/storage/Tsavorite/cs/test/StateMachineDriverTests.cs Updates page size to new minimum.
libs/storage/Tsavorite/cs/test/RecordTriggersExtTests.cs Updates page size to new minimum.
libs/storage/Tsavorite/cs/test/PostOperationsTests.cs Updates page size to new minimum.
libs/storage/Tsavorite/cs/test/NeedCopyUpdateTests.cs Updates page size to new minimum.
libs/storage/Tsavorite/cs/test/LowMemoryTests.cs Updates LocalMemoryDevice ctor usage and log sizing for new minimums.
libs/storage/Tsavorite/cs/test/InputOutputParameterTests.cs Updates page size to new minimum.
libs/storage/Tsavorite/cs/test/InitialIORecordSizeTests.cs New tests covering default/sentinel values and store/session/operation overrides.
libs/storage/Tsavorite/cs/test/BasicTests.cs Updates page size to new minimum.
libs/storage/Tsavorite/cs/src/core/Utilities/Native32.cs Adds GetDeviceSectorSize helper and related structs/docs (Windows).
libs/storage/Tsavorite/cs/src/core/Index/Tsavorite/Tsavorite.cs Stores InitialIORecordSize from KVSettings; adds per-operation RMW overload with IO-size override.
libs/storage/Tsavorite/cs/src/core/Index/Tsavorite/Implementation/InternalRMW.cs Resolves initial IO record size before returning RECORD_ON_DISK.
libs/storage/Tsavorite/cs/src/core/Index/Tsavorite/Implementation/InternalRead.cs Resolves initial IO record size before returning RECORD_ON_DISK (read & read-at-address).
libs/storage/Tsavorite/cs/src/core/Index/Tsavorite/Implementation/HandleOperationStatus.cs Uses resolved IO record size when issuing AsyncGetFromDisk; adds resolver helper.
libs/storage/Tsavorite/cs/src/core/Index/Common/PendingContext.cs Adds initialIORecordSize to pending context and plumbs through constructors/debug string.
libs/storage/Tsavorite/cs/src/core/Index/Common/OperationOptions.cs Adds InitialIORecordSize to ReadOptions/RMWOptions.
libs/storage/Tsavorite/cs/src/core/Index/Common/LogSettings.cs Raises minimum page size to 4KB; adjusts min memory bits; preserves 64B min overflow inline size bits.
libs/storage/Tsavorite/cs/src/core/Index/Common/KVSettings.cs Adds store-level InitialIORecordSize and sentinel constant.
libs/storage/Tsavorite/cs/src/core/Index/Common/ExecutionContext.cs Adds session-level InitialIORecordSize.
libs/storage/Tsavorite/cs/src/core/Device/StorageDeviceBase.cs Minor cleanup.
libs/storage/Tsavorite/cs/src/core/Device/RandomAccessLocalStorageDevice.cs Refactors sector-size lookup and adds ToString diagnostics.
libs/storage/Tsavorite/cs/src/core/Device/NullDevice.cs Uses IDevice.MinDeviceSectorSize for sector size.
libs/storage/Tsavorite/cs/src/core/Device/NativeStorageDevice.cs Uses IDevice.MinDeviceSectorSize as fallback sector floor.
libs/storage/Tsavorite/cs/src/core/Device/ManagedLocalStorageDevice.cs Refactors sector-size lookup and adds ToString diagnostics.
libs/storage/Tsavorite/cs/src/core/Device/LocalStorageDevice.cs Keeps sector-size caching; adds ToString diagnostics.
libs/storage/Tsavorite/cs/src/core/Device/LocalMemoryDevice.cs Renames ctor parameter and defaults sector size to MinDeviceSectorSize.
libs/storage/Tsavorite/cs/src/core/Device/IDevice.cs Introduces MinDeviceSectorSize constant (512 bytes).
libs/storage/Tsavorite/cs/src/core/ClientSession/UnsafeContext.cs Plumbs RMW per-operation InitialIORecordSize override.
libs/storage/Tsavorite/cs/src/core/ClientSession/TransactionalUnsafeContext.cs Plumbs RMW per-operation InitialIORecordSize override.
libs/storage/Tsavorite/cs/src/core/ClientSession/TransactionalContext.cs Plumbs RMW per-operation InitialIORecordSize override.
libs/storage/Tsavorite/cs/src/core/ClientSession/ManageClientSessions.cs Adds session-level initialIORecordSize parameter to NewSession and stores into session context.
libs/storage/Tsavorite/cs/src/core/ClientSession/BasicContext.cs Plumbs RMW per-operation InitialIORecordSize override.
libs/storage/Tsavorite/cs/src/core/Allocator/ObjectSerialization/ObjectLogReader.cs Updates docs to reference new default IO record size constant name.
libs/storage/Tsavorite/cs/src/core/Allocator/ObjectSerialization/IStreamBuffer.cs Renames constant to DefaultInitialIORecordSize and updates doc wording.
libs/storage/Tsavorite/cs/src/core/Allocator/MallocFixedPageSize.cs Clarifies sector-size comment for flush/restore path.
libs/storage/Tsavorite/cs/src/core/Allocator/LogRecord.cs Adds helper to compute expected record size for IO/speculation.
libs/storage/Tsavorite/cs/src/core/Allocator/AllocatorScan.cs Uses new default IO record size constant.
libs/storage/Tsavorite/cs/src/core/Allocator/AllocatorBase.cs Cleans up aligned read window math; uses new default IO record size constant.
libs/server/Servers/GarnetServerOptions.cs Adds InitialIORecordSize option and parsing into KVSettings.
libs/host/defaults.conf Adds InitialIORecordSize default (null) and documentation comment.
libs/host/Configuration/Options.cs Adds CLI option --initial-io-record-size with validation and help text.

Comment thread libs/storage/Tsavorite/cs/src/core/Index/Common/KVSettings.cs
Comment thread libs/storage/Tsavorite/cs/src/core/Utilities/Native32.cs Outdated
Comment thread libs/host/defaults.conf
Comment thread libs/storage/Tsavorite/cs/src/core/Utilities/Native32.cs
Comment thread libs/server/Servers/GarnetServerOptions.cs
@TedHartMS TedHartMS requested a review from vazois June 11, 2026 00:04
@badrishc badrishc merged commit 5fe6e37 into main Jun 11, 2026
384 of 385 checks passed
@badrishc badrishc deleted the tedhar/io-4k branch June 11, 2026 06:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants