Skip to content

Commit d2ec388

Browse files
committed
test(repro): memory-budget #363 + bounded RSS-soak #581
- #363: the cgroup CPU axis was fixed in v0.8.0 (detect_system_linux reads cpu.max/cfs_quota), but there is no user-controllable memory ceiling — cbm_mem_init derives the budget from host/cgroup RAM with no override. Test sets CBM_MEM_BUDGET_MB and asserts cbm_mem_budget honors it; RED because that env knob is not read (mirrors the open ask from the issue thread; note this is the memory-override remainder, adjacent to enhancement #580). - #581: bounded RSS-growth reproduction for the long-running leak — repeats a query op ~150x and asserts current RSS (cbm_mem_rss / /proc/self/statm) stays within 3x of the warmup baseline. Documents flakiness + that a slow per-op leak may not trip the bounded threshold (then a true multi-hour soak is needed). #513 (Windows stdio handshake hang) is NOT reproducible in the unit-test runner: it is Windows-only WaitForSingleObject code in cbm_mcp_server_run. It needs the prod cbm binary built on windows-latest + a stdio-driver test that sends an initialize request, keeps stdin open, and asserts a response before EOF. Deferred as a board extension (documented in private/BUG_REPRO_PLAN.md). Signed-off-by: Martin Vogel <[email protected]>
1 parent 895069a commit d2ec388

4 files changed

Lines changed: 421 additions & 1 deletion

File tree

‎Makefile.cbm‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -421,7 +421,9 @@ TEST_REPRO_SRCS = \
421421
tests/repro/repro_issue548.c \
422422
tests/repro/repro_new_ts_class_field_arrow.c \
423423
tests/repro/repro_new_py_tuple_unpack.c \
424-
tests/repro/repro_new_cypher_limit_zero.c
424+
tests/repro/repro_new_cypher_limit_zero.c \
425+
tests/repro/repro_issue363.c \
426+
tests/repro/repro_issue581.c
425427

426428
ALL_TEST_SRCS =$(TEST_FOUNDATION_SRCS) $(TEST_EXTRACTION_SRCS) $(TEST_STORE_SRCS) $(TEST_CYPHER_SRCS) $(TEST_MCP_SRCS) $(TEST_DISCOVER_SRCS) $(TEST_GRAPH_BUFFER_SRCS) $(TEST_PIPELINE_SRCS) $(TEST_WATCHER_SRCS) $(TEST_LZ4_SRCS) $(TEST_ZSTD_SRCS) $(TEST_ARTIFACT_SRCS) $(TEST_SQLITE_WRITER_SRCS) $(TEST_GO_LSP_SRCS) $(TEST_C_LSP_SRCS) $(TEST_PHP_LSP_SRCS) $(TEST_CS_LSP_SRCS) $(TEST_CS_LSP_BENCH_SRCS) $(TEST_SCOPE_SRCS) $(TEST_TYPE_REP_SRCS) $(TEST_PY_LSP_SRCS) $(TEST_PY_LSP_BENCH_SRCS) $(TEST_PY_LSP_STRESS_SRCS) $(TEST_PY_LSP_SCALE_SRCS) $(TEST_TS_LSP_SRCS) $(TEST_JAVA_LSP_SRCS) $(TEST_KOTLIN_LSP_SRCS) $(TEST_RUST_LSP_SRCS) $(TEST_TRACES_SRCS) $(TEST_CLI_SRCS) $(TEST_MEM_SRCS) $(TEST_UI_SRCS) $(TEST_HTTPD_SRCS) $(TEST_SECURITY_SRCS) $(TEST_YAML_SRCS) $(TEST_SIMHASH_SRCS) $(TEST_STACK_OVERFLOW_SRCS) $(TEST_INTEGRATION_SRCS)
427429

‎tests/repro/repro_issue363.c‎

Lines changed: 120 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,120 @@
1+
/*
2+
* repro_issue363.c — Reproduce-first case for OPEN bug #363.
3+
*
4+
* Issue: #363 — "Linux: cbm_system_info / cbm_default_worker_count don't
5+
* respect cgroup CPU/memory limits"
6+
*
7+
* ROOT CAUSE (two distinct axes):
8+
*
9+
* CPU axis — FIXED in v0.8.0 (commit a5a3d1d).
10+
* cbm_detect_cgroup_cpus() reads /sys/fs/cgroup/cpu.max (v2) or
11+
* .../cpu/cpu.cfs_quota_us + .../cpu/cpu.cfs_period_us (v1) and the
12+
* result is used by detect_system_linux() in system_info.c:226.
13+
* cbm_default_worker_count() also honours the CBM_WORKERS env override
14+
* (commit d952238). Both are thoroughly tested in test_platform.c.
15+
*
16+
* Memory axis — STILL OPEN (confirmed by reporter @mayurpise in the last
17+
* open comment on #363, 2026-06-25).
18+
* cbm_detect_cgroup_mem() similarly reads /sys/fs/cgroup/memory.max (v2)
19+
* or .../memory/memory.limit_in_bytes (v1), and detect_system_linux()
20+
* uses it (system_info.c:229). BUT: there is NO env-override knob on
21+
* the memory axis. The CPU axis has CBM_WORKERS; the memory side has
22+
* nothing. On a bare-metal host with no enclosing cgroup, users cannot
23+
* cap cbm_mem_init's budget without wrapping the process in a cgroup
24+
* scope (as @mayurpise's workaround shows).
25+
*
26+
* EXACT OPEN GAP:
27+
* A CBM_MEM_BUDGET_MB environment variable (analogous to CBM_WORKERS) that
28+
* cbm_mem_init() checks before computing g_budget from info.total_ram.
29+
* If set to a valid integer N, cbm_mem_init() should set
30+
* g_budget = N * 1024 * 1024, honouring it regardless of cgroup or host RAM.
31+
*
32+
* WHY THIS TEST IS RED:
33+
* cbm_mem_init() (src/foundation/mem.c) reads cbm_system_info().total_ram
34+
* and multiplies by ram_fraction. It does NOT call cbm_safe_getenv for
35+
* CBM_MEM_BUDGET_MB — the override path does not exist. Setting
36+
* CBM_MEM_BUDGET_MB=4096 has no effect; cbm_mem_budget() returns a value
37+
* derived from host RAM (or cgroup RAM when inside a container), not from
38+
* the env var. The assertion ASSERT_EQ(cbm_mem_budget(), 4096*1024*1024)
39+
* therefore fails on any host whose cgroup or physical RAM != exactly 4 GiB.
40+
*
41+
* ROOT CAUSE LOCATION:
42+
* src/foundation/mem.c, cbm_mem_init(), after the mimalloc option block
43+
* (currently around line 126):
44+
* cbm_system_info_t info = cbm_system_info();
45+
* g_budget = (size_t)((double)info.total_ram * ram_fraction);
46+
* The fix is to insert a cbm_safe_getenv("CBM_MEM_BUDGET_MB", ...) lookup
47+
* BEFORE this line and, if valid, set g_budget directly without involving
48+
* info.total_ram — mirroring the CBM_WORKERS pattern in
49+
* cbm_default_worker_count() (system_info.c:290).
50+
*
51+
* INTENDED FIX:
52+
* 1. In cbm_mem_init(): read CBM_MEM_BUDGET_MB; if set to a valid positive
53+
* integer, use that value (in bytes) as g_budget and log it.
54+
* 2. Test: set CBM_MEM_BUDGET_MB=4096, call cbm_mem_init(0.5), assert
55+
* cbm_mem_budget() == 4096 * 1024 * 1024. This test goes GREEN when
56+
* the override is wired.
57+
* 3. Complementary: on Linux, confirm cbm_system_info().total_ram is capped
58+
* by the cgroup memory limit when present — already covered in
59+
* test_platform.c via cbm_detect_cgroup_mem() unit tests, but an
60+
* integration path via cbm_system_info() is untestable without a seam
61+
* that lets callers override the hardcoded "/sys/fs/cgroup" root in
62+
* detect_system_linux() (system_info.c:229).
63+
*
64+
* NOTE on cbm_mem_init() caching:
65+
* g_budget is initialised once via atomic_compare_exchange_strong.
66+
* The test must run in a process where cbm_mem_init() has NOT been called
67+
* yet, OR the test must reset g_initialized — neither is supported today.
68+
* The repro works as written because the repro runner does not call
69+
* cbm_mem_init() before this suite. If the initialisation guard is an
70+
* issue, the fix also needs a cbm_mem_reset_for_test() hook (test-only,
71+
* guarded by CBM_TEST_HOOKS or similar).
72+
*/
73+
74+
#include "test_framework.h"
75+
#include <foundation/mem.h>
76+
#include <foundation/compat.h>
77+
#include <stdint.h>
78+
#include <stdlib.h>
79+
80+
#define REPRO363_BUDGET_MB 4096UL
81+
#define REPRO363_BUDGET_BYTES (REPRO363_BUDGET_MB * 1024UL * 1024UL)
82+
83+
/*
84+
* repro_issue363_mem_budget_env_override
85+
*
86+
* Precondition: CBM_MEM_BUDGET_MB=4096 is set before cbm_mem_init() is
87+
* called. The budget should be 4096 MiB regardless of host RAM or cgroup.
88+
*
89+
* RED condition (current code):
90+
* cbm_mem_init() ignores CBM_MEM_BUDGET_MB entirely; cbm_mem_budget()
91+
* returns host-RAM * fraction, not 4 GiB. The assertion fires unless the
92+
* test runner happens to be on a machine whose effective RAM is exactly
93+
* 8 GiB with fraction=0.5 — essentially never.
94+
*
95+
* GREEN condition (after fix):
96+
* cbm_mem_init() reads CBM_MEM_BUDGET_MB, finds "4096", sets
97+
* g_budget = 4096 * 1024 * 1024. The assertion passes on any machine.
98+
*/
99+
TEST(repro_issue363_mem_budget_env_override) {
100+
cbm_setenv("CBM_MEM_BUDGET_MB", "4096", 1);
101+
102+
cbm_mem_init(0.5);
103+
104+
size_t budget = cbm_mem_budget();
105+
106+
cbm_unsetenv("CBM_MEM_BUDGET_MB");
107+
108+
/*
109+
* RED on current code: budget derives from host/cgroup RAM, not the env
110+
* var. On any machine where effective RAM != 8192 MiB this fails.
111+
* GREEN once CBM_MEM_BUDGET_MB is wired in cbm_mem_init().
112+
*/
113+
ASSERT_EQ((long long)budget, (long long)REPRO363_BUDGET_BYTES);
114+
115+
PASS();
116+
}
117+
118+
SUITE(repro_issue363) {
119+
RUN_TEST(repro_issue363_mem_budget_env_override);
120+
}

0 commit comments

Comments
 (0)