Skip to content

perf(wal): add cairo.wal.writer.madvise.mode configuration option#6841

Merged
bluestreak01 merged 1 commit intomasterfrom
jh_walwriter_configurable_madvise
Mar 3, 2026
Merged

perf(wal): add cairo.wal.writer.madvise.mode configuration option#6841
bluestreak01 merged 1 commit intomasterfrom
jh_walwriter_configurable_madvise

Conversation

@jerrinot
Copy link
Copy Markdown
Contributor

@jerrinot jerrinot commented Mar 3, 2026

WalWriter previously hardcoded POSIX_MADV_RANDOM for memory-mapped column files. This hurts most workloads, but some benefit from it. Make the madvise hint opt-in via a new config property with valid values: none (default, no hint), sequential and random.

RANDOM is beneficial when ingesting into many tables with many columns, as it prevents the OS from speculatively reading adjacent pages under memory pressure.

Example configuration:

cairo.wal.writer.madvise.mode=random

WalWriter previously hardcoded POSIX_MADV_RANDOM for memory-mapped
column files. This hurts most workloads, but some benefit from it.
Make the madvise hint opt-in via a new config property with valid
values: none (default, no hint), sequential and random.

RANDOM is beneficial when ingesting into many tables with many columns,
as it prevents the OS from speculatively reading adjacent pages under
memory pressure.

Example configuration:
cairo.wal.writer.madvise.mode=random
@jerrinot jerrinot added Performance Performance improvements WAL labels Mar 3, 2026
@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Mar 3, 2026

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • ✅ Review completed - (🔄 Check again to review again)
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch jh_walwriter_configurable_madvise

Tip

Try Coding Plans. Let us write the prompt for your AI agent so you can ship faster (with fewer bugs).
Share your feedback on Discord.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (2)
core/src/test/java/io/questdb/test/PropServerConfigurationTest.java (1)

2102-2102: Consider adding coverage for sequential and invalid values.

This change validates random, but adding assertions for sequential and invalid-token fallback would lock all parser branches for the new option.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@core/src/test/java/io/questdb/test/PropServerConfigurationTest.java` at line
2102, Add two more assertions in PropServerConfigurationTest to cover the
remaining parser branches for wal writer madvise: assert that
configuration.getWalWriterMadviseMode() equals Files.POSIX_MADV_SEQUENTIAL when
the property is set to "sequential", and assert the fallback behavior (the
expected default constant) when an invalid token is provided; update the test
setup that feeds the property (the same place where "random" is exercised) to
supply the "sequential" and an invalid value and verify the returned constants
to lock the parser branches for Files.POSIX_MADV_SEQUENTIAL,
Files.POSIX_MADV_RANDOM and the invalid-token fallback.
core/src/main/java/io/questdb/PropServerConfiguration.java (1)

2260-2279: Use enhanced switch to decode madvise mode and extract the sentinel constant.

This method is a strong fit for an enhanced switch expression, consistent with patterns already used elsewhere in this class (e.g., getLineTimestampUnit). Extract the sentinel value -1 into a named constant for clarity.

♻️ Proposed refactor
+    private static final int WAL_WRITER_MADVISE_NONE = -1;
+
     private int getWalWriterMadviseMode(Properties properties, `@Nullable` Map<String, String> env, ConfigPropertyKey key) throws ServerConfigurationException {
         final String mode = getString(properties, env, key, "none");
-
-        // must not be null because we provided non-null default value
-        assert mode != null;
-
-        if (Chars.equalsLowerCaseAscii(mode, "none")) {
-            return -1;
-        }
-
-        if (Chars.equalsLowerCaseAscii(mode, "sequential")) {
-            return Files.POSIX_MADV_SEQUENTIAL;
-        }
-
-        if (Chars.equalsLowerCaseAscii(mode, "random")) {
-            return Files.POSIX_MADV_RANDOM;
-        }
-
-        throw ServerConfigurationException.forInvalidKey(key.getPropertyPath(), mode);
+        assert mode != null;
+        return switch (mode.toLowerCase()) {
+            case "none" -> WAL_WRITER_MADVISE_NONE;
+            case "sequential" -> Files.POSIX_MADV_SEQUENTIAL;
+            case "random" -> Files.POSIX_MADV_RANDOM;
+            default -> throw ServerConfigurationException.forInvalidKey(key.getPropertyPath(), mode);
+        };
     }
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@core/src/main/java/io/questdb/PropServerConfiguration.java` around lines 2260
- 2279, Refactor getWalWriterMadviseMode to use an enhanced switch on the
lowercased mode string and extract the sentinel -1 into a named constant (e.g.,
POSIX_MADV_NONE) for readability; replace the chain of
Chars.equalsLowerCaseAscii checks with something like switch
(Chars.toLowerCaseAscii(mode)) { case "none" -> POSIX_MADV_NONE; case
"sequential" -> Files.POSIX_MADV_SEQUENTIAL; case "random" ->
Files.POSIX_MADV_RANDOM; default -> throw
ServerConfigurationException.forInvalidKey(key.getPropertyPath(), mode); } while
keeping the method signature and null-safety assertions unchanged.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@core/src/main/java/io/questdb/PropServerConfiguration.java`:
- Around line 2260-2279: Refactor getWalWriterMadviseMode to use an enhanced
switch on the lowercased mode string and extract the sentinel -1 into a named
constant (e.g., POSIX_MADV_NONE) for readability; replace the chain of
Chars.equalsLowerCaseAscii checks with something like switch
(Chars.toLowerCaseAscii(mode)) { case "none" -> POSIX_MADV_NONE; case
"sequential" -> Files.POSIX_MADV_SEQUENTIAL; case "random" ->
Files.POSIX_MADV_RANDOM; default -> throw
ServerConfigurationException.forInvalidKey(key.getPropertyPath(), mode); } while
keeping the method signature and null-safety assertions unchanged.

In `@core/src/test/java/io/questdb/test/PropServerConfigurationTest.java`:
- Line 2102: Add two more assertions in PropServerConfigurationTest to cover the
remaining parser branches for wal writer madvise: assert that
configuration.getWalWriterMadviseMode() equals Files.POSIX_MADV_SEQUENTIAL when
the property is set to "sequential", and assert the fallback behavior (the
expected default constant) when an invalid token is provided; update the test
setup that feeds the property (the same place where "random" is exercised) to
supply the "sequential" and an invalid value and verify the returned constants
to lock the parser branches for Files.POSIX_MADV_SEQUENTIAL,
Files.POSIX_MADV_RANDOM and the invalid-token fallback.

ℹ️ Review info

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between fc24a7f and 1919f85.

📒 Files selected for processing (11)
  • core/src/main/java/io/questdb/PropServerConfiguration.java
  • core/src/main/java/io/questdb/PropertyKey.java
  • core/src/main/java/io/questdb/cairo/CairoConfiguration.java
  • core/src/main/java/io/questdb/cairo/CairoConfigurationWrapper.java
  • core/src/main/java/io/questdb/cairo/DefaultCairoConfiguration.java
  • core/src/main/java/io/questdb/cairo/wal/WalWriter.java
  • core/src/main/resources/io/questdb/site/conf/server.conf
  • core/src/test/java/io/questdb/test/PropServerConfigurationTest.java
  • core/src/test/java/io/questdb/test/ServerMainTest.java
  • core/src/test/resources/server.conf
  • pkg/ami/marketplace/assets/server.conf

@bluestreak01 bluestreak01 merged commit ea4ef52 into master Mar 3, 2026
41 of 48 checks passed
@bluestreak01 bluestreak01 deleted the jh_walwriter_configurable_madvise branch March 3, 2026 13:49
@glasstiger
Copy link
Copy Markdown
Contributor

[PR Coverage check]

😍 pass : 28 / 28 (100.00%)

file detail

path covered line new line coverage
🔵 io/questdb/cairo/O3PartitionJob.java 6 6 100.00%
🔵 io/questdb/cairo/ParquetTimestampFinder.java 7 7 100.00%
🔵 io/questdb/cairo/TableWriter.java 15 15 100.00%

maciulis pushed a commit to maciulis/questdb that referenced this pull request Mar 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Performance Performance improvements WAL

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants