Skip to content

perf(core): run mat view refresh on a separate thread pool#6111

Merged
bluestreak01 merged 10 commits intomasterfrom
vi_mv_pool
Sep 9, 2025
Merged

perf(core): run mat view refresh on a separate thread pool#6111
bluestreak01 merged 10 commits intomasterfrom
vi_mv_pool

Conversation

@ideoma
Copy link
Copy Markdown
Collaborator

@ideoma ideoma commented Sep 8, 2025

Summary

Reintroduce dedicated thread pool for Materialized View refresh operations to prevent blocking of WAL apply jobs

Problem

In PR #5805, the dedicated MV refresh pool was removed and MV refresh jobs were moved to run on the shared writer pool alongside Apply WAL jobs. This architectural change has created a resource contention issue where:

  • Heavy Materialized View workloads can monopolize the shared pool
  • Apply WAL jobs may experience significant delays or starvation
  • Database write performance can degrade when MV operations are intensive

Solution

This PR restores the dedicated MV refresh pool as the default configuration. When the dedicated MV pool is disabled or has 0 workers assigned, the MV refresh job will not run.

@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Sep 8, 2025

Important

Review skipped

Auto reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

✨ Finishing Touches
🧪 Generate unit tests
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch vi_mv_pool

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@ideoma
Copy link
Copy Markdown
Collaborator Author

ideoma commented Sep 8, 2025

@coderabbitai review

@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Sep 8, 2025

✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Copy link
Copy Markdown

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🧹 Nitpick comments (18)
core/src/main/java/io/questdb/cairo/wal/seq/TableSequencerImpl.java (1)

215-217: Guard against long→int overflow for WAL IDs.

IDGenerator.getNextId() returns long, casting to int risks overflow if IDs ever exceed Integer.MAX_VALUE. If the interface must return int, add a defensive check to fail fast.

-    public int getNextWalId() {
-        return (int) walIdGenerator.getNextId();
-    }
+    public int getNextWalId() {
+        final long id = walIdGenerator.getNextId();
+        if (id > Integer.MAX_VALUE) {
+            throw CairoException.critical(0).put("wal id overflow: ").put(id);
+        }
+        return (int) id;
+    }
core/src/main/java/io/questdb/cutlass/line/tcp/LineTcpMeasurementScheduler.java (2)

110-118: Nit: variable name consistency

Consider renaming networkSharedPoolSize → sharedNetworkPoolSize for consistency with the new terminology.


125-131: Guard against zero writer threads

If sharedPoolWrite.getWorkerCount() can ever be 0 (misconfiguration), downstream arrays will be zero-sized and later indexing will throw. Add a fail-fast check.

-            int nWriterThreads = sharedPoolWrite.getWorkerCount();
+            int nWriterThreads = sharedPoolWrite.getWorkerCount();
+            if (nWriterThreads <= 0) {
+                throw new IllegalArgumentException("sharedPoolWrite must have at least one worker");
+            }
core/src/test/java/io/questdb/test/TestServerConfiguration.java (1)

256-259: Optional: expose shared query/write configs for symmetry

If tests need them, also override getSharedWorkerPoolQueryConfiguration/getSharedWorkerPoolWriteConfiguration to return confSharedPool for consistency.

core/src/main/java/io/questdb/cutlass/line/tcp/LineTcpReceiver.java (1)

58-60: Initialize scheduler before registering dispatcher (minor ordering hardening)

Currently safe because pools aren’t started yet, but initializing scheduler first avoids future race potential if ordering changes.

-            this.dispatcher = IODispatchers.create(configuration, contextFactory);
-            sharedPoolNetwork.assign(dispatcher);
-            this.scheduler = new LineTcpMeasurementScheduler(configuration, engine, sharedPoolNetwork, dispatcher, sharedPoolWrite);
+            this.dispatcher = IODispatchers.create(configuration, contextFactory);
+            this.scheduler = new LineTcpMeasurementScheduler(configuration, engine, sharedPoolNetwork, dispatcher, sharedPoolWrite);
+            sharedPoolNetwork.assign(dispatcher);

Also applies to: 61-65

core/src/test/java/io/questdb/test/WorkerPoolManagerTest.java (1)

81-95: Nit: local variable naming

Consider renaming variables like networkSharedPool → sharedPoolNetwork to mirror the API and reduce cognitive load.

core/src/main/java/io/questdb/WorkerPoolManager.java (3)

54-56: Minor: avoid duplicate getter and clarify intent.

Cache the query configuration locally to avoid the double call and improve readability.

-        sharedPoolNetwork = new WorkerPool(config.getSharedWorkerPoolNetworkConfiguration());
-        sharedPoolQuery = config.getSharedWorkerPoolQueryConfiguration().getWorkerCount() > 0 ? new WorkerPool(config.getSharedWorkerPoolQueryConfiguration()) : null;
+        sharedPoolNetwork = new WorkerPool(config.getSharedWorkerPoolNetworkConfiguration());
+        final WorkerPoolConfiguration queryConf = config.getSharedWorkerPoolQueryConfiguration();
+        sharedPoolQuery = queryConf.getWorkerCount() > 0 ? new WorkerPool(queryConf) : null;
         sharedPoolWrite = new WorkerPool(config.getSharedWorkerPoolWriteConfiguration());

71-73: Add no-arg write accessor for symmetry and clarity.

Like network’s no-arg getter, provide an explicit shared write accessor to avoid accidental dedicated-pool selection when a config is passed.

     public WorkerPool getSharedPoolWrite(@NotNull WorkerPoolConfiguration config, @NotNull Requester requester) {
         return getWorkerPool(config, requester, sharedPoolWrite);
     }
+
+    public WorkerPool getSharedPoolWrite() {
+        return sharedPoolWrite;
+    }

178-181: Annotate sharedPoolQuery as nullable in the hook.

This parameter can be null when parallel querying is disabled; make it explicit.

-    protected abstract void configureWorkerPools(
-            final WorkerPool sharedPoolQuery,
-            final WorkerPool sharedPoolWrite
-    );
+    protected abstract void configureWorkerPools(
+            @Nullable final WorkerPool sharedPoolQuery,
+            final WorkerPool sharedPoolWrite
+    );
core/src/main/java/io/questdb/ServerConfiguration.java (1)

87-92: Bridge old pool getters to preserve binary compatibility.

Add deprecated default methods mapping old names to the new “shared” accessors to ease downstream upgrades.

     WorkerPoolConfiguration getSharedWorkerPoolWriteConfiguration();
 
+    /** @deprecated use {@link #getSharedWorkerPoolNetworkConfiguration()} */
+    @Deprecated
+    default WorkerPoolConfiguration getNetworkWorkerPoolConfiguration() {
+        return getSharedWorkerPoolNetworkConfiguration();
+    }
+    /** @deprecated use {@link #getSharedWorkerPoolQueryConfiguration()} */
+    @Deprecated
+    default WorkerPoolConfiguration getQueryWorkerPoolConfiguration() {
+        return getSharedWorkerPoolQueryConfiguration();
+    }
+    /** @deprecated use {@link #getSharedWorkerPoolWriteConfiguration()} */
+    @Deprecated
+    default WorkerPoolConfiguration getWriteWorkerPoolConfiguration() {
+        return getSharedWorkerPoolWriteConfiguration();
+    }

Also consider a brief Javadoc note that “dedicated vs shared” is selected by WorkerPoolConfiguration.getWorkerCount() (0 → shared).

core/src/main/java/io/questdb/DynamicPropServerConfiguration.java (2)

359-361: Consistency: add non-reloadable comment like network.

Keep comments uniform across the three shared pool getters.

     @Override
     public WorkerPoolConfiguration getSharedWorkerPoolQueryConfiguration() {
+        // nested object is kept non-reloadable
         return serverConfig.get().getSharedWorkerPoolQueryConfiguration();
     }

364-366: Consistency: add non-reloadable comment like network.

     @Override
     public WorkerPoolConfiguration getSharedWorkerPoolWriteConfiguration() {
+        // nested object is kept non-reloadable
         return serverConfig.get().getSharedWorkerPoolWriteConfiguration();
     }
core/src/main/java/io/questdb/ServerMain.java (2)

399-404: Clarify the comment vs API name

The comment says “dedicated pool”, but the API used is getSharedPoolWrite. If this method conditionally creates a dedicated pool, please reword the comment to reflect that to avoid confusion during maintenance.

-                // This starts mat view refresh jobs only when there is a dedicated pool for mat view refresh
-                // this will not use shared pool write because getWorkerCount() > 0
+                // Starts MV refresh jobs using a dedicated write pool (via manager), since workerCount > 0

405-409: Include env var hint in advisory

Surface both the property path and its env var to make the message actionable in containerized setups.

-                log.advisory().$("mat view refresh is disabled; set ")
-                        .$(MAT_VIEW_REFRESH_WORKER_COUNT.getPropertyPath())
-                        .$(" to a positive value or keep default to enable mat view refresh.")
-                        .$();
+                log.advisory().$("mat view refresh is disabled; set ")
+                        .$(MAT_VIEW_REFRESH_WORKER_COUNT.getPropertyPath())
+                        .$(" (env: ")
+                        .$(propertyPathToEnvVarName(MAT_VIEW_REFRESH_WORKER_COUNT.getPropertyPath()))
+                        .$(") to a positive value or keep default to enable mat view refresh.")
+                        .$();
core/src/test/java/io/questdb/test/PropServerConfigurationTest.java (1)

1082-1090: Use hyphenated pool names in DefaultServerConfiguration
Replace underscore-separated names with hyphens to match PropServerConfiguration:
• In DefaultServerConfiguration (core/src/main/java/io/questdb/DefaultServerConfiguration.java), change
"shared_network""shared-network"
"shared_query""shared-query"
"shared_write""shared-write"

core/src/main/java/io/questdb/DefaultServerConfiguration.java (1)

61-66: Use hyphenated pool names for consistency

Elsewhere (tests, logs) pools use “shared-network/query/write”, “mat-view-refresh”, “wal-apply”. Recommend aligning defaults here.

-        this.sharedPoolNetworkConfiguration = new DefaultWorkerPoolConfiguration("shared_network");
-        this.sharedPoolQueryConfiguration = new DefaultWorkerPoolConfiguration("shared_query");
-        this.sharedPoolWriteConfiguration = new DefaultWorkerPoolConfiguration("shared_write");
-        this.matViewRefreshPoolConfiguration = new DefaultWorkerPoolConfiguration("mat_view_refresh");
-        this.walApplyPoolConfiguration = new DefaultWorkerPoolConfiguration("wal_apply");
+        this.sharedPoolNetworkConfiguration = new DefaultWorkerPoolConfiguration("shared-network");
+        this.sharedPoolQueryConfiguration = new DefaultWorkerPoolConfiguration("shared-query");
+        this.sharedPoolWriteConfiguration = new DefaultWorkerPoolConfiguration("shared-write");
+        this.matViewRefreshPoolConfiguration = new DefaultWorkerPoolConfiguration("mat-view-refresh");
+        this.walApplyPoolConfiguration = new DefaultWorkerPoolConfiguration("wal-apply");
core/src/main/java/io/questdb/PropServerConfiguration.java (2)

1738-1742: Keep query pool affinity consistent when disabling the pool

Directly mutating sharedWorkerCount to 0 can leave a stale non-empty affinity array. Also clear affinity.

-                sharedWorkerPoolQueryConfiguration.sharedWorkerCount = 0;
+                sharedWorkerPoolQueryConfiguration.sharedWorkerCount = 0;
+                sharedWorkerPoolQueryConfiguration.sharedWorkerAffinity = new int[0];

5183-5228: Optional: make mat-view pool priority and enablement explicit

For parity with other pools and easier ops tuning, consider setting an explicit priority and an isEnabled() guard.

         @Override
         public boolean haltOnError() {
             return matViewRefreshWorkerHaltOnError;
         }
+
+        @Override
+        public int workerPoolPriority() {
+            // Keep below network to avoid starving IO; align with write pool bias.
+            return Thread.NORM_PRIORITY - 1;
+        }
+
+        @Override
+        public boolean isEnabled() {
+            return matViewRefreshWorkerCount > 0;
+        }
📜 Review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 4aeca59 and 9d8eee8.

📒 Files selected for processing (16)
  • core/src/main/java/io/questdb/DefaultServerConfiguration.java (2 hunks)
  • core/src/main/java/io/questdb/DynamicPropServerConfiguration.java (1 hunks)
  • core/src/main/java/io/questdb/PropServerConfiguration.java (10 hunks)
  • core/src/main/java/io/questdb/ServerConfiguration.java (1 hunks)
  • core/src/main/java/io/questdb/ServerMain.java (6 hunks)
  • core/src/main/java/io/questdb/WorkerPoolManager.java (1 hunks)
  • core/src/main/java/io/questdb/cairo/wal/seq/TableSequencerImpl.java (1 hunks)
  • core/src/main/java/io/questdb/cutlass/Services.java (4 hunks)
  • core/src/main/java/io/questdb/cutlass/line/tcp/LineTcpMeasurementScheduler.java (3 hunks)
  • core/src/main/java/io/questdb/cutlass/line/tcp/LineTcpReceiver.java (2 hunks)
  • core/src/test/java/io/questdb/test/PropServerConfigurationTest.java (6 hunks)
  • core/src/test/java/io/questdb/test/ServerMainVectorGroupByTest.java (2 hunks)
  • core/src/test/java/io/questdb/test/TestServerConfiguration.java (1 hunks)
  • core/src/test/java/io/questdb/test/WorkerPoolManagerTest.java (6 hunks)
  • core/src/test/java/io/questdb/test/cairo/mv/MatViewReloadOnRestartTest.java (1 hunks)
  • core/src/test/java/io/questdb/test/cutlass/line/tcp/LineTcpO3Test.java (1 hunks)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (27)
  • GitHub Check: New pull request (Coverage Report Coverage Report)
  • GitHub Check: New pull request (SelfHosted Griffin And Fuzz tests on linux-x64-zfs)
  • GitHub Check: New pull request (SelfHosted Griffin And Fuzz tests on linux-arm64)
  • GitHub Check: New pull request (SelfHosted Other tests on linux-x64-zfs)
  • GitHub Check: New pull request (SelfHosted Other tests on linux-arm64)
  • GitHub Check: New pull request (Hosted Running tests on windows-other)
  • GitHub Check: New pull request (Hosted Running tests on windows-pgwire)
  • GitHub Check: New pull request (Hosted Running tests on windows-cairo)
  • GitHub Check: New pull request (Hosted Running tests on windows-fuzz2)
  • GitHub Check: New pull request (Hosted Running tests on windows-fuzz1)
  • GitHub Check: New pull request (Hosted Running tests on windows-griffin)
  • GitHub Check: New pull request (Hosted Running tests on mac-other)
  • GitHub Check: New pull request (Hosted Running tests on mac-pgwire)
  • GitHub Check: New pull request (Hosted Running tests on mac-cairo-fuzz)
  • GitHub Check: New pull request (Hosted Running tests on mac-cairo)
  • GitHub Check: New pull request (SelfHosted Griffin And Fuzz tests Start ARM Agent)
  • GitHub Check: New pull request (SelfHosted Other tests Start X64Zfs Agent)
  • GitHub Check: New pull request (SelfHosted Griffin And Fuzz tests Start X64Zfs Agent)
  • GitHub Check: New pull request (Hosted Running tests on mac-griffin)
  • GitHub Check: New pull request (Rust Test and Lint on linux-jdk17)
  • GitHub Check: New pull request (Hosted Running tests with cover on linux-other)
  • GitHub Check: New pull request (Hosted Running tests with cover on linux-pgwire)
  • GitHub Check: New pull request (Hosted Running tests with cover on linux-cairo)
  • GitHub Check: New pull request (Hosted Running tests with cover on linux-fuzz)
  • GitHub Check: New pull request (Hosted Running tests with cover on linux-griffin)
  • GitHub Check: New pull request (SelfHosted Other tests Start ARM Agent)
  • GitHub Check: New pull request (Check Changes Check changes)
🔇 Additional comments (29)
core/src/main/java/io/questdb/cairo/wal/seq/TableSequencerImpl.java (1)

97-101: Formatting-only change looks good.

Argument reflow is clean and preserves behavior. No concerns.

core/src/test/java/io/questdb/test/cutlass/line/tcp/LineTcpO3Test.java (1)

128-128: Switch to shared network worker-pool config — LGTM

Accessor aligns with the new shared pool API; no issues spotted.

core/src/main/java/io/questdb/cutlass/line/tcp/LineTcpMeasurementScheduler.java (2)

97-100: Constructor param rename to sharedPoolNetwork/sharedPoolWrite — LGTM

Names now match Services/LineTcpReceiver usage.


169-171: Writer jobs bound to sharedPoolWrite — LGTM

assign/freeOnExit usage is correct.

core/src/test/java/io/questdb/test/TestServerConfiguration.java (1)

256-259: Adds getSharedWorkerPoolNetworkConfiguration — LGTM

Matches the shared-pool API.

core/src/test/java/io/questdb/test/cairo/mv/MatViewReloadOnRestartTest.java (1)

1004-1009: Param rename to mvWorkerPool — LGTM

Signature matches ServerMain changes; no behavior impact.

core/src/test/java/io/questdb/test/ServerMainVectorGroupByTest.java (1)

96-97: Assert shared network pool size via new accessor — LGTM

Correctly targets the shared network pool configuration.

Also applies to: 121-122

core/src/main/java/io/questdb/cutlass/Services.java (1)

83-85: Rename to getSharedPoolNetwork/getSharedPoolWrite — LGTM

Call sites and comments align with the new API; behavior unchanged.

Also applies to: 154-163, 195-199, 259-261

core/src/test/java/io/questdb/test/WorkerPoolManagerTest.java (1)

81-95: Tests updated to getSharedPoolNetwork and shared config accessors — LGTM

Renames are consistent and assertions still validate intended semantics.

Also applies to: 113-121, 127-141, 149-159, 181-189, 285-297

core/src/main/java/io/questdb/WorkerPoolManager.java (3)

63-65: All getSharedNetworkPool references migrated. No occurrences of the old API were found in the codebase.


71-73: getInstanceWrite removal confirmed safe. No occurrences of getInstanceWrite( found in the codebase.


63-65: Mat-View Refresh Pool Wiring
Confirmed mat-view refresh uses

workerPoolManager.getSharedPoolWrite(
    config.getMatViewRefreshPoolConfiguration(),
    Requester.MAT_VIEW_REFRESH
);

which delegates to getWorkerPool(..., sharedPoolWrite) and would fallback to sharedPoolWrite if workerCount<1. However, in ServerMain (lines 391–398), when getMatViewRefreshPoolConfiguration().getWorkerCount()==0 the code logs “mat view refresh is disabled” instead of invoking getSharedPoolWrite. Confirm whether disabling is intentional or if it should fall back to the shared write pool.

core/src/main/java/io/questdb/DynamicPropServerConfiguration.java (2)

353-356: LGTM: delegation updated to the shared-network config.

Behavior preserved; non-reloadable note retained.


353-366: Verified: PropServerConfiguration already implements getSharedWorkerPoolNetworkConfiguration, getSharedWorkerPoolQueryConfiguration, and getSharedWorkerPoolWriteConfiguration.

core/src/main/java/io/questdb/ServerMain.java (3)

288-291: Pass logger into initialize() — good change

This enables advisory logging during init without global lookups.


412-418: WAL apply on dedicated/shared pool — looks correct

Using getSharedPoolWrite with WAL_APPLY requester aligns with the new pool model.


390-409: No action needed: getWorkerCount()>0 is the sole effective guard
PropMatViewsRefreshPoolConfiguration does not override isEnabled() (it inherits the default true), so gating on getWorkerCount()>0 already covers enablement; adding isEnabled() would be redundant.

Likely an incorrect or invalid review comment.

core/src/test/java/io/questdb/test/PropServerConfigurationTest.java (5)

139-145: HTTP concurrent cache rows now tied to shared network workers — OK

Assertions reflect the new shared-network pool contract and defaults.


448-449: PGWire cache rows bound to shared-network workers — OK


506-508: MV refresh pool defaults — verify coupling

You assert isEnabled() and workerCount > 0. Given ServerMain gates on workerCount only, please confirm isEnabled() mirrors that to avoid false positives if implementations diverge.


751-753: Env overrides for shared-network workers/affinity — OK

Covers both count and affinity mapping via QDB_SHARED_NETWORK_WORKER_AFFINITY.


1234-1238: Shared-network thresholds from file — OK

core/src/main/java/io/questdb/DefaultServerConfiguration.java (1)

133-150: New shared-pool getters — OK

API surface matches the migration to shared pools.

core/src/main/java/io/questdb/PropServerConfiguration.java (6)

308-309: LGTM: dedicated mat-view refresh pool instance

Separate pool object makes the intent explicit.


376-378: LGTM: split shared pools (network/query/write) with clear names

Good separation and naming for observability.


1662-1673: LGTM: shared pools configured with sensible priorities

Network > Query > Write priority hierarchy is clear; defaults sourced once.

Also applies to: 1676-1687, 1691-1701


1882-1894: LGTM: new shared-pool getters

Public API exposure of the split shared pools looks good.


1896-1899: LGTM: wal-apply pool getter unchanged

No issues spotted.


1303-1310: Gating on zero worker count is correct – no changes needed
ServerMain.java correctly checks config.getMatViewRefreshPoolConfiguration().getWorkerCount() > 0 (lines 390–398) before starting the refresh pool; if set to 0 it logs an advisory and skips mat‐view refresh.

@ideoma ideoma changed the title chore(core): run mat view refresh on a separate thread pool perf(core): run mat view refresh on a separate thread pool Sep 8, 2025
@puzpuzpuz puzpuzpuz self-requested a review September 9, 2025 10:43
@puzpuzpuz puzpuzpuz added Bug Incorrect or unexpected behavior Materialized View labels Sep 9, 2025
@glasstiger
Copy link
Copy Markdown
Contributor

[PR Coverage check]

😍 pass : 60 / 72 (83.33%)

file detail

path covered line new line coverage
🔵 io/questdb/DefaultServerConfiguration.java 3 7 42.86%
🔵 io/questdb/ServerMain.java 13 17 76.47%
🔵 io/questdb/PropServerConfiguration.java 18 22 81.82%
🔵 io/questdb/cutlass/line/tcp/LineTcpMeasurementScheduler.java 7 7 100.00%
🔵 io/questdb/WorkerPoolManager.java 4 4 100.00%
🔵 io/questdb/cairo/wal/seq/TableSequencerImpl.java 2 2 100.00%
🔵 io/questdb/cutlass/line/tcp/LineTcpReceiver.java 4 4 100.00%
🔵 io/questdb/DynamicPropServerConfiguration.java 3 3 100.00%
🔵 io/questdb/cutlass/Services.java 6 6 100.00%

@ideoma
Copy link
Copy Markdown
Collaborator Author

ideoma commented Sep 9, 2025

Thanks for the review @puzpuzpuz

@bluestreak01 bluestreak01 merged commit 568a930 into master Sep 9, 2025
35 checks passed
@bluestreak01 bluestreak01 deleted the vi_mv_pool branch September 9, 2025 21:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Bug Incorrect or unexpected behavior Materialized View

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants