perf(wal): add cairo.wal.writer.madvise.mode configuration option#6841
perf(wal): add cairo.wal.writer.madvise.mode configuration option#6841bluestreak01 merged 1 commit intomasterfrom
Conversation
WalWriter previously hardcoded POSIX_MADV_RANDOM for memory-mapped column files. This hurts most workloads, but some benefit from it. Make the madvise hint opt-in via a new config property with valid values: none (default, no hint), sequential and random. RANDOM is beneficial when ingesting into many tables with many columns, as it prevents the OS from speculatively reading adjacent pages under memory pressure. Example configuration: cairo.wal.writer.madvise.mode=random
|
Important Review skippedAuto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Tip Try Coding Plans. Let us write the prompt for your AI agent so you can ship faster (with fewer bugs). Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
🧹 Nitpick comments (2)
core/src/test/java/io/questdb/test/PropServerConfigurationTest.java (1)
2102-2102: Consider adding coverage forsequentialand invalid values.This change validates
random, but adding assertions forsequentialand invalid-token fallback would lock all parser branches for the new option.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@core/src/test/java/io/questdb/test/PropServerConfigurationTest.java` at line 2102, Add two more assertions in PropServerConfigurationTest to cover the remaining parser branches for wal writer madvise: assert that configuration.getWalWriterMadviseMode() equals Files.POSIX_MADV_SEQUENTIAL when the property is set to "sequential", and assert the fallback behavior (the expected default constant) when an invalid token is provided; update the test setup that feeds the property (the same place where "random" is exercised) to supply the "sequential" and an invalid value and verify the returned constants to lock the parser branches for Files.POSIX_MADV_SEQUENTIAL, Files.POSIX_MADV_RANDOM and the invalid-token fallback.core/src/main/java/io/questdb/PropServerConfiguration.java (1)
2260-2279: Use enhanced switch to decode madvise mode and extract the sentinel constant.This method is a strong fit for an enhanced
switchexpression, consistent with patterns already used elsewhere in this class (e.g.,getLineTimestampUnit). Extract the sentinel value-1into a named constant for clarity.♻️ Proposed refactor
+ private static final int WAL_WRITER_MADVISE_NONE = -1; + private int getWalWriterMadviseMode(Properties properties, `@Nullable` Map<String, String> env, ConfigPropertyKey key) throws ServerConfigurationException { final String mode = getString(properties, env, key, "none"); - - // must not be null because we provided non-null default value - assert mode != null; - - if (Chars.equalsLowerCaseAscii(mode, "none")) { - return -1; - } - - if (Chars.equalsLowerCaseAscii(mode, "sequential")) { - return Files.POSIX_MADV_SEQUENTIAL; - } - - if (Chars.equalsLowerCaseAscii(mode, "random")) { - return Files.POSIX_MADV_RANDOM; - } - - throw ServerConfigurationException.forInvalidKey(key.getPropertyPath(), mode); + assert mode != null; + return switch (mode.toLowerCase()) { + case "none" -> WAL_WRITER_MADVISE_NONE; + case "sequential" -> Files.POSIX_MADV_SEQUENTIAL; + case "random" -> Files.POSIX_MADV_RANDOM; + default -> throw ServerConfigurationException.forInvalidKey(key.getPropertyPath(), mode); + }; }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@core/src/main/java/io/questdb/PropServerConfiguration.java` around lines 2260 - 2279, Refactor getWalWriterMadviseMode to use an enhanced switch on the lowercased mode string and extract the sentinel -1 into a named constant (e.g., POSIX_MADV_NONE) for readability; replace the chain of Chars.equalsLowerCaseAscii checks with something like switch (Chars.toLowerCaseAscii(mode)) { case "none" -> POSIX_MADV_NONE; case "sequential" -> Files.POSIX_MADV_SEQUENTIAL; case "random" -> Files.POSIX_MADV_RANDOM; default -> throw ServerConfigurationException.forInvalidKey(key.getPropertyPath(), mode); } while keeping the method signature and null-safety assertions unchanged.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Nitpick comments:
In `@core/src/main/java/io/questdb/PropServerConfiguration.java`:
- Around line 2260-2279: Refactor getWalWriterMadviseMode to use an enhanced
switch on the lowercased mode string and extract the sentinel -1 into a named
constant (e.g., POSIX_MADV_NONE) for readability; replace the chain of
Chars.equalsLowerCaseAscii checks with something like switch
(Chars.toLowerCaseAscii(mode)) { case "none" -> POSIX_MADV_NONE; case
"sequential" -> Files.POSIX_MADV_SEQUENTIAL; case "random" ->
Files.POSIX_MADV_RANDOM; default -> throw
ServerConfigurationException.forInvalidKey(key.getPropertyPath(), mode); } while
keeping the method signature and null-safety assertions unchanged.
In `@core/src/test/java/io/questdb/test/PropServerConfigurationTest.java`:
- Line 2102: Add two more assertions in PropServerConfigurationTest to cover the
remaining parser branches for wal writer madvise: assert that
configuration.getWalWriterMadviseMode() equals Files.POSIX_MADV_SEQUENTIAL when
the property is set to "sequential", and assert the fallback behavior (the
expected default constant) when an invalid token is provided; update the test
setup that feeds the property (the same place where "random" is exercised) to
supply the "sequential" and an invalid value and verify the returned constants
to lock the parser branches for Files.POSIX_MADV_SEQUENTIAL,
Files.POSIX_MADV_RANDOM and the invalid-token fallback.
ℹ️ Review info
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (11)
core/src/main/java/io/questdb/PropServerConfiguration.javacore/src/main/java/io/questdb/PropertyKey.javacore/src/main/java/io/questdb/cairo/CairoConfiguration.javacore/src/main/java/io/questdb/cairo/CairoConfigurationWrapper.javacore/src/main/java/io/questdb/cairo/DefaultCairoConfiguration.javacore/src/main/java/io/questdb/cairo/wal/WalWriter.javacore/src/main/resources/io/questdb/site/conf/server.confcore/src/test/java/io/questdb/test/PropServerConfigurationTest.javacore/src/test/java/io/questdb/test/ServerMainTest.javacore/src/test/resources/server.confpkg/ami/marketplace/assets/server.conf
[PR Coverage check]😍 pass : 28 / 28 (100.00%) file detail
|
WalWriter previously hardcoded
POSIX_MADV_RANDOMfor memory-mapped column files. This hurts most workloads, but some benefit from it. Make the madvise hint opt-in via a new config property with valid values: none (default, no hint), sequential and random.RANDOM is beneficial when ingesting into many tables with many columns, as it prevents the OS from speculatively reading adjacent pages under memory pressure.
Example configuration: