Repository navigation
fix: disable SQLite mmap to prevent memory growing with DB size - #22428
jiangliang79 wants to merge 1 commit into
Conversation
Bun's bundled SQLite enables memory-mapped I/O by default, which maps the entire DB file into the process address space. This causes the process footprint to grow proportionally with the DB file size regardless of actual data access patterns. Observed on macOS ARM64: a 1.1 GB opencode.db results in ~1.8 GB of IOAccelerator memory in the process footprint snapshot. Setting PRAGMA mmap_size = 0 restores SQLite's official default behavior, bounding SQLite's memory contribution to the cache_size limit (~64 MB). The performance impact is negligible — sub-millisecond extra latency on cache misses, dwarfed by LLM API call latency.
|
Thanks for your contribution! This PR doesn't have a linked issue. All PRs must reference an existing issue. Please:
See CONTRIBUTING.md for details. |
github-actions
Bot
added
needs:compliance
github-actions
Bot
removed
the
needs:compliance
|
Thanks for updating your PR! It now meets our contributing guidelines. 👍 |
Bun enables SQLite memory-mapped I/O by default, mapping the entire DB file into the process address space. A 1.1 GB database adds ~1.8 GB to RSS regardless of how much data is actually accessed. Add PRAGMA mmap_size = 0 to the existing PRAGMA initialization block. Ref anomalyco#22428 Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
|
Hit the same issue initially and opened a duplicate PR (#22553, now closed). Before deferring, we ran a quick verification on Linux (Ubuntu 24.04 LTS, kernel 6.17, Bun 1.3.11): |
8.5GB 主 DB 通过 mmap 每进程贡献 134-260MB（IOAccelerator）， 关联上游 PR anomalyco#22428、anomalyco#18136 等已知问题。
|
This branch has been upgraded to V2. We're no longer taking PRs for V1. If you think this is still relevant, please port it over and open it for V2. — from 𝕺𝖕𝖊𝖓𝕮𝖔𝖉𝖊 |
Issue for this PR
Closes #22429
Type of change
What does this PR do?
Bun's bundled SQLite enables memory-mapped I/O by default, which maps the entire DB file into the process address space. This causes the process footprint to grow proportionally with the DB file size regardless of actual data access patterns.
Observed on macOS ARM64: a 1.1 GB opencode.db results in ~1.8 GB of IOAccelerator memory in the process footprint snapshot.
Setting PRAGMA mmap_size = 0 restores SQLite's official default behavior, bounding SQLite's memory contribution to the cache_size limit (~64 MB). The performance impact is negligible — sub-millisecond extra latency on cache misses, dwarfed by LLM API call latency.
How did you verify your code works?
Took a footprint snapshot of the opencode process on macOS ARM64 before and after the change with a 1.1 GB DB. The IOAccelerator region (SQLite mmap pages) dropped from ~1.8 GB to negligible. Functional behavior is unchanged — mmap only affects how pages are loaded from disk, not what data SQLite returns.
Checklist