Skip to content

Conversation

@naive924
Copy link
Contributor

@naive924 naive924 commented Sep 2, 2025

Description

This PR adds an option to run map copy in parallel during MVStore.compact().

  • Controlled by system property h2.compactThreads (default = 1).
  • Uses ExecutorService and CompletableFuture to copy multiple maps concurrently.
  • Falls back to sequential behavior if h2.compactThreads=1.
  • Significantly reduces compaction time on large databases. For example, compacting a 80 GB store down to 20 GB took about 50 min → 20 min with 8 threads.

The change preserves existing logic and metadata handling, only parallelizing the map copy loop.

targetMeta.put(MVMap.getMapKey(targetMap.getId()), sourceMeta.get(MVMap.getMapKey(sourceMap.getId())));
}

int poolSize = Integer.getInteger("h2.compactThreads", 1);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why would we not want to use half or quarter of the available cores as default instead?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You’re right — defaulting to ¼ of available cores is a good balance.
int poolSize = Integer.getInteger( "h2.compactThreads", Math.max(1, Runtime.getRuntime().availableProcessors() / 4) );

@andreitokar
Copy link
Contributor

@naive924, Thank You for your contribution!
Please send a license statement as described here
https://h2database.com/html/build.html#providing_patches
to the mailing list (Google group):
https://groups.google.com/g/h2-database

@naive924 naive924 changed the title Add parallel map copy option for MVStore.compact() SHUTDOWN COMPACT: parallel map copy (¼ cores default, override with h2.compactThreads) Sep 4, 2025
@naive924 naive924 force-pushed the feat/compactThreads branch from 4673d31 to 6672123 Compare September 4, 2025 02:26
@andreitokar andreitokar merged commit a448d91 into h2database:master Sep 5, 2025
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants