Skip to content

Commit de16152

Browse files
committed
feat(index): add worker resource watchdogs
Discovery limits bound what indexing accepts, not what it then costs. A repository well inside those bounds can still exhaust the host through parser memory, or simply never finish, and a supervised worker that hangs leaves the parent waiting with nothing to report. Add index_max_rss_mb and index_max_duration_seconds, enforced by the parent against the worker process tree rather than the worker process alone, so a runaway child cannot hide behind a small parent. Resident memory is sampled through the platform interface on macOS, Linux and Windows. Crossing a limit terminates the tree and yields one trusted, structured terminal result that attributes the failure to the resource that caused it. Both limits default to off. A measurement that cannot be taken fails the attempt instead of passing it: a watchdog that quietly stops watching is worse than no watchdog at all. The shell fixture that stands in for the supervisor names the two new keys. The worker accepts only a policy that spells out every key it knows, which is what keeps a stale supervisor from starting a worker it cannot bound. Signed-off-by: 刘冲 <[email protected]>
1 parent d4980ad commit de16152

21 files changed

Lines changed: 1472 additions & 111 deletions

‎README.md‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -928,10 +928,12 @@ codebase-memory-mcp config set watcher_enabled false # stop the watcher thre
928928
codebase-memory-mcp config set watch_non_git true # also poll non-git project roots (default: false)
929929
codebase-memory-mcp config set index_max_files 250000 # optional per-index source-file limit
930930
codebase-memory-mcp config set index_max_source_mb 16384 # optional per-index source-size limit
931+
codebase-memory-mcp config set index_max_rss_mb 8192 # optional worker-tree current RSS limit
932+
codebase-memory-mcp config set index_max_duration_seconds 3600 # optional total worker duration
931933
codebase-memory-mcp config reset auto_index # reset to default
932934
```
933935

934-
The two `index_max_*` settings default to `off`. Exceeding one fails the complete
936+
The four `index_max_*` settings default to `off`. Exceeding one fails the complete
935937
index attempt rather than publishing a partial graph; an existing serving index
936938
is preserved. See [Index resource limits](docs/INDEX_RESOURCE_LIMITS.md).
937939

‎docs/CONFIGURATION.md‎

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -93,6 +93,8 @@ Current keys:
9393
| `watch_non_git` | `false` | Also poll project roots that are **not git repositories**. By default the watcher only follows git projects, so a project indexed from a plain directory is never refreshed after its first index — reindex it manually with `index_repository`. Set `true` to poll such roots on the same adaptive cadence with a file-tree scan: the indexer's own discovery walk (same skip lists, `.gitignore` and `.cbmignore` rules) hashed over each file's path, size and mtime. Any change reindexes once; paths the indexer skips (including cbm's own `.codebase-memory/` output) never trigger. The first poll after the daemon starts reindexes each such project once, since nothing records which tree state the index holds. The scan walks the whole tree every poll, so it costs more than git polling on very large trees. Read once when the daemon starts, like `watcher_enabled`. |
9494
| `index_max_files` | `off` | Optional maximum number of accepted source files in one discovery run. |
9595
| `index_max_source_mb` | `off` | Optional maximum accepted source size in MiB in one discovery run. |
96+
| `index_max_rss_mb` | `off` | Optional maximum current RSS in MiB for the complete contained index-worker process tree (`64..1048576`). |
97+
| `index_max_duration_seconds` | `off` | Optional maximum total worker duration in seconds (`1..86400`). |
9698

9799
For a watched project, `index_status` names the polling method in `watch.strategy`:
98100
`git` (HEAD and dirty-state polling), `tree` (a non-git root polled by the
@@ -126,11 +128,13 @@ has not run yet).
126128
> `auto_index` still runs, and `index_repository` stays available for manual
127129
> reindexing.
128130
129-
The two `index_max_*` settings are independent and disabled by default. They
131+
The four `index_max_*` settings are independent and disabled by default. They
130132
apply to explicit indexing, automatic indexing, and watcher re-indexing, but not
131133
to `cross-repo-intelligence`, which does not scan repository source files.
132-
Equality is allowed; exceeding either setting fails the complete index request
133-
and preserves any previously serving database. See
134+
Equality is allowed; exceeding any setting fails the complete index request and
135+
preserves any previously serving database. Worker RSS covers descendants and is
136+
not the same as the internal `CBM_MEM_BUDGET_MB` allocation budget. Total
137+
duration is independent of the existing 15-minute no-log-progress timeout. See
134138
[Index resource limits](INDEX_RESOURCE_LIMITS.md) for counting, validation, and
135139
error-response details.
136140

‎docs/INDEX_RESOURCE_LIMITS.md‎

Lines changed: 44 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,8 @@
11
# Index resource limits
22

33
Index resource limits are optional operator controls for repositories whose
4-
discovery breadth is not known in advance. They are disabled by default so
5-
existing large-repository workloads retain their current behavior.
4+
discovery breadth or worker runtime is not known in advance. They are disabled
5+
by default so existing large-repository workloads retain their current behavior.
66

77
## Discovery settings
88

@@ -23,6 +23,39 @@ Values use base-10 integers. MiB means 1,048,576 bytes. Empty values, zero,
2323
negative values, suffixes, trailing characters, and values outside the stated
2424
ranges are rejected without changing the stored value.
2525

26+
## Worker settings
27+
28+
| Key | Default | Accepted value | Protects |
29+
|---|---:|---:|---|
30+
| `index_max_rss_mb` | `off` | `off` or `64..1048576` | Current RSS of the complete worker process tree |
31+
| `index_max_duration_seconds` | `off` | `off` or `1..86400` | Total worker wall-clock duration |
32+
33+
```bash
34+
codebase-memory-mcp config set index_max_rss_mb 8192
35+
codebase-memory-mcp config set index_max_duration_seconds 3600
36+
```
37+
38+
RSS is the current resident memory of the contained worker and every descendant,
39+
not the worker's allocation budget and not peak memory. This hard watchdog is
40+
separate from the internal `CBM_MEM_BUDGET_MB` soft budget. The supervisor
41+
samples RSS at most once every 250 milliseconds so the watchdog does not turn
42+
full process-table enumeration into a busy loop.
43+
44+
Duration uses a monotonic clock from successful spawn. It is independent of the
45+
existing 15-minute quiet timeout: continuous log progress does not reset total
46+
duration, while the quiet timeout continues to identify a worker that stops
47+
making progress.
48+
49+
Equality is allowed. The first RSS or elapsed-duration observation above its
50+
limit starts the existing graceful-to-force process-tree shutdown. CBM reports
51+
terminal only after the tree is quiescent or a bounded containment failure is
52+
explicitly surfaced. Resource termination is not retried and does not
53+
quarantine a source file.
54+
55+
If RSS is enabled and three consecutive probes cannot obtain any trustworthy
56+
tree measurement while the root worker is still running, CBM fails closed with
57+
`code=resource_probe_failed`.
58+
2659
## Counting and failure semantics
2760

2861
`index_max_files` counts a file only after it passes directory pruning, ignore
@@ -55,6 +88,12 @@ The previous database remains available because publication occurs only after a
5588
complete discovery and successful staged build. If no previous database exists,
5689
`serving_index_preserved` is false.
5790

91+
Worker limit failures use the same shape with `stage=worker`,
92+
`resource=rss_bytes` and `unit=bytes`, or `resource=duration_ms` and
93+
`unit=milliseconds`. RSS measurement failures use `code=resource_probe_failed`
94+
and omit `observed`, `limit`, and `unit` because no trustworthy observation was
95+
available.
96+
5897
## Trust and compatibility
5998

6099
Limits are read from the CLI-managed `_config.db`; they are not MCP request
@@ -64,5 +103,6 @@ parent policy.
64103

65104
These settings do not replace or increase `auto_index_limit`, change the 512 MiB
66105
single-file cap, alter workspace-root authorization, or affect
67-
`cross-repo-intelligence`. With both settings `off`, discovery follows the
68-
existing unbounded path.
106+
`cross-repo-intelligence`. With all settings `off`, discovery follows the
107+
existing path and the supervisor performs no periodic RSS probe or total-duration
108+
termination.

‎scripts/test-runtime.sh‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -112,7 +112,8 @@ _cbm_test_runtime_daemon() {
112112
# cbm_mcp_index_policy_add_to_args: every key in cbm_index_policy_key_at, and
113113
# nothing else.
114114
cbm_test_index_worker_policy_json() {
115-
printf '%s' '"_cbm_index_policy":{"index_max_files":"off","index_max_source_mb":"off"}'
115+
printf '%s' '"_cbm_index_policy":{"index_max_files":"off","index_max_source_mb":"off"'
116+
printf '%s' ',"index_max_rss_mb":"off","index_max_duration_seconds":"off"}'
116117
}
117118

118119
cbm_test_runtime_cleanup() {

‎src/cli/cli.c‎

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -7603,6 +7603,8 @@ static const config_key_def_t CONFIG_KEYS[] = {
76037603
{CBM_CONFIG_UI_PORT, "9749", "Port for the graph UI listener when enabled"},
76047604
{CBM_INDEX_CONFIG_MAX_FILES, "off", "Max accepted source files per index, or off"},
76057605
{CBM_INDEX_CONFIG_MAX_SOURCE_MB, "off", "Max accepted source MiB per index, or off"},
7606+
{CBM_INDEX_CONFIG_MAX_RSS_MB, "off", "Max worker process-tree RSS MiB, or off"},
7607+
{CBM_INDEX_CONFIG_MAX_DURATION_SECONDS, "off", "Max worker duration in seconds, or off"},
76067608
};
76077609

76087610
/* #1558: ui_enabled and ui_port were reachable ONLY by hand-editing
@@ -7629,8 +7631,12 @@ static bool config_key_is_ui(const char *key) {
76297631
}
76307632

76317633
static bool config_key_is_index_policy(const char *key) {
7632-
return key && (strcmp(key, CBM_INDEX_CONFIG_MAX_FILES) == 0 ||
7633-
strcmp(key, CBM_INDEX_CONFIG_MAX_SOURCE_MB) == 0);
7634+
for (size_t index = 0; key && index < cbm_index_policy_key_count(); index++) {
7635+
if (strcmp(key, cbm_index_policy_key_at(index)) == 0) {
7636+
return true;
7637+
}
7638+
}
7639+
return false;
76347640
}
76357641

76367642
static int config_index_policy_write(cbm_config_t *config, const char *key, const char *value) {

‎src/daemon/application.c‎

Lines changed: 17 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -363,6 +363,14 @@ static int application_worker_start_default(void *context, const char *args_json
363363
const char *quarantine_file,
364364
cbm_daemon_application_worker_t *worker_out) {
365365
(void)context;
366+
cbm_index_resource_policy_t resource_policy;
367+
char error[CBM_SZ_256] = {0};
368+
if (!cbm_mcp_index_policy_from_internal_args(args_json, &resource_policy, error,
369+
sizeof(error))) {
370+
cbm_log_error("daemon.index.policy", "error", error);
371+
*worker_out = NULL;
372+
return -1;
373+
}
366374
application_default_worker_t *worker = cbm_calloc(CBM_MEM_CLASS_OTHER, sizeof(*worker));
367375
if (!worker) {
368376
return -1;
@@ -372,9 +380,9 @@ static int application_worker_start_default(void *context, const char *args_json
372380
const char *root = yyjson_get_str(yyjson_obj_get(yyjson_doc_get_root(args_doc), "repo_path"));
373381
worker->project = application_index_project_key(root, args_json);
374382
yyjson_doc_free(args_doc);
375-
int result = cbm_index_worker_start_with_log(args_json, memory_budget_bytes, false, marker_file,
376-
quarantine_file, application_worker_diagnostic,
377-
worker, &worker->handle);
383+
int result = cbm_index_worker_start_with_policy_log(
384+
args_json, memory_budget_bytes, &resource_policy, false, marker_file, quarantine_file,
385+
application_worker_diagnostic, worker, &worker->handle);
378386
if (result != 0) {
379387
application_worker_destroy_default(NULL, worker);
380388
*worker_out = NULL;
@@ -1567,6 +1575,12 @@ static application_attempt_decision_t application_consume_attempt(
15671575
application_attempt_free(attempt);
15681576
return APPLICATION_ATTEMPT_DECISION_SUCCESS;
15691577
}
1578+
if (disposition == CBM_MCP_SUPERVISED_RESULT_RESOURCE_FAILURE) {
1579+
execution->response =
1580+
cbm_mcp_index_worker_resource_response(job->args_json, &attempt->result);
1581+
application_attempt_free(attempt);
1582+
return APPLICATION_ATTEMPT_DECISION_STOP;
1583+
}
15701584
if (disposition == CBM_MCP_SUPERVISED_RESULT_UNSAFE_TERMINAL) {
15711585
execution->unsafe_terminal = true;
15721586
execution->supervision_failed =

0 commit comments

Comments
 (0)