Skip to content

Commit 1e4751d

Browse files
committed
fix(index): charge macOS phys_footprint and pin duration across recovery
Watchdogs now sample the OS-charged footprint instead of resident size on Darwin, and index_max_duration_seconds covers the whole request so a crash/hang respawn cannot reset the clock. Signed-off-by: 刘冲 <[email protected]>
1 parent a45858f commit 1e4751d

13 files changed

Lines changed: 166 additions & 64 deletions

‎README.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -928,8 +928,8 @@ codebase-memory-mcp config set watcher_enabled false # stop the watcher thre
928928
codebase-memory-mcp config set watch_non_git true # also poll non-git project roots (default: false)
929929
codebase-memory-mcp config set index_max_files 250000 # optional per-index source-file limit
930930
codebase-memory-mcp config set index_max_source_mb 16384 # optional per-index source-size limit
931-
codebase-memory-mcp config set index_max_rss_mb 8192 # optional worker-tree current RSS limit
932-
codebase-memory-mcp config set index_max_duration_seconds 3600 # optional total worker duration
931+
codebase-memory-mcp config set index_max_rss_mb 8192 # optional worker-tree charged-memory limit
932+
codebase-memory-mcp config set index_max_duration_seconds 3600 # optional per-request duration
933933
codebase-memory-mcp config reset auto_index # reset to default
934934
```
935935

‎docs/CONFIGURATION.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -93,8 +93,8 @@ Current keys:
9393
| `watch_non_git` | `false` | Also poll project roots that are **not git repositories**. By default the watcher only follows git projects, so a project indexed from a plain directory is never refreshed after its first index — reindex it manually with `index_repository`. Set `true` to poll such roots on the same adaptive cadence with a file-tree scan: the indexer's own discovery walk (same skip lists, `.gitignore` and `.cbmignore` rules) hashed over each file's path, size and mtime. Any change reindexes once; paths the indexer skips (including cbm's own `.codebase-memory/` output) never trigger. The first poll after the daemon starts reindexes each such project once, since nothing records which tree state the index holds. The scan walks the whole tree every poll, so it costs more than git polling on very large trees. Read once when the daemon starts, like `watcher_enabled`. |
9494
| `index_max_files` | `off` | Optional maximum number of accepted source files in one discovery run. |
9595
| `index_max_source_mb` | `off` | Optional maximum accepted source size in MiB in one discovery run. |
96-
| `index_max_rss_mb` | `off` | Optional maximum current RSS in MiB for the complete contained index-worker process tree (`64..1048576`). |
97-
| `index_max_duration_seconds` | `off` | Optional maximum total worker duration in seconds (`1..86400`). |
96+
| `index_max_rss_mb` | `off` | Optional maximum charged memory in MiB for the complete contained index-worker process tree (`64..1048576`). On macOS this is phys_footprint, not resident size. |
97+
| `index_max_duration_seconds` | `off` | Optional maximum wall-clock duration in seconds for the whole index request, including crash/hang recovery (`1..86400`). |
9898

9999
For a watched project, `index_status` names the polling method in `watch.strategy`:
100100
`git` (HEAD and dirty-state polling), `tree` (a non-git root polled by the

‎docs/INDEX_RESOURCE_LIMITS.md‎

Lines changed: 17 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -27,24 +27,30 @@ ranges are rejected without changing the stored value.
2727

2828
| Key | Default | Accepted value | Protects |
2929
|---|---:|---:|---|
30-
| `index_max_rss_mb` | `off` | `off` or `64..1048576` | Current RSS of the complete worker process tree |
31-
| `index_max_duration_seconds` | `off` | `off` or `1..86400` | Total worker wall-clock duration |
30+
| `index_max_rss_mb` | `off` | `off` or `64..1048576` | Charged memory of the complete worker process tree |
31+
| `index_max_duration_seconds` | `off` | `off` or `1..86400` | Wall-clock duration of the whole index request |
3232

3333
```bash
3434
codebase-memory-mcp config set index_max_rss_mb 8192
3535
codebase-memory-mcp config set index_max_duration_seconds 3600
3636
```
3737

38-
RSS is the current resident memory of the contained worker and every descendant,
39-
not the worker's allocation budget and not peak memory. This hard watchdog is
38+
RSS is the charged memory of the contained worker and every descendant, not
39+
the worker's allocation budget and not peak memory. On macOS that quantity is
40+
`phys_footprint` (the same number `cbm_mem_charged()` enforces), not
41+
`resident_size`, which still counts pages the allocator has already handed
42+
back. Linux and Windows use RSS / working set. This hard watchdog is
4043
separate from the internal `CBM_MEM_BUDGET_MB` soft budget. The supervisor
41-
samples RSS at most once every 250 milliseconds so the watchdog does not turn
42-
full process-table enumeration into a busy loop.
43-
44-
Duration uses a monotonic clock from successful spawn. It is independent of the
45-
existing 15-minute quiet timeout: continuous log progress does not reset total
46-
duration, while the quiet timeout continues to identify a worker that stops
47-
making progress.
44+
samples the tree at most once every 250 milliseconds so the watchdog does not
45+
turn full process-table enumeration into a busy loop.
46+
47+
Duration is per request: the clock starts at the first worker spawn and is
48+
not reset when crash/hang recovery starts a later attempt. Continuous log
49+
progress does not reset it. It is independent of the existing 15-minute quiet
50+
timeout, which still identifies a worker that stops making progress. A
51+
duration limit shorter than that quiet timeout kills a hung worker before hang
52+
quarantine can name the in-flight file, so the next attempt may hang on the
53+
same file.
4854

4955
Equality is allowed. The first RSS or elapsed-duration observation above its
5056
limit starts the existing graceful-to-force process-tree shutdown. CBM reports

‎src/cli/cli.c‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -7603,8 +7603,9 @@ static const config_key_def_t CONFIG_KEYS[] = {
76037603
{CBM_CONFIG_UI_PORT, "9749", "Port for the graph UI listener when enabled"},
76047604
{CBM_INDEX_CONFIG_MAX_FILES, "off", "Max accepted source files per index, or off"},
76057605
{CBM_INDEX_CONFIG_MAX_SOURCE_MB, "off", "Max accepted source MiB per index, or off"},
7606-
{CBM_INDEX_CONFIG_MAX_RSS_MB, "off", "Max worker process-tree RSS MiB, or off"},
7607-
{CBM_INDEX_CONFIG_MAX_DURATION_SECONDS, "off", "Max worker duration in seconds, or off"},
7606+
{CBM_INDEX_CONFIG_MAX_RSS_MB, "off", "Max worker process-tree charged memory MiB, or off"},
7607+
{CBM_INDEX_CONFIG_MAX_DURATION_SECONDS, "off",
7608+
"Max wall-clock seconds for the whole index request, or off"},
76087609
};
76097610

76107611
/* #1558: ui_enabled and ui_port were reachable ONLY by hand-editing

‎src/daemon/application.c‎

Lines changed: 9 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -159,6 +159,7 @@ struct cbm_daemon_application_job {
159159
* job outlives the request and session that started it. */
160160
bool async_owned;
161161
bool worker_started;
162+
uint64_t request_started_ms;
162163
cbm_daemon_application_job_t *next;
163164
};
164165

@@ -361,6 +362,7 @@ static void application_worker_diagnostic(const char *line, void *context) {
361362
static int application_worker_start_default(void *context, const char *args_json,
362363
size_t memory_budget_bytes, const char *marker_file,
363364
const char *quarantine_file,
365+
uint64_t duration_origin_ms,
364366
cbm_daemon_application_worker_t *worker_out) {
365367
(void)context;
366368
cbm_index_resource_policy_t resource_policy;
@@ -382,7 +384,7 @@ static int application_worker_start_default(void *context, const char *args_json
382384
yyjson_doc_free(args_doc);
383385
int result = cbm_index_worker_start_with_policy_log(
384386
args_json, memory_budget_bytes, &resource_policy, false, marker_file, quarantine_file,
385-
application_worker_diagnostic, worker, &worker->handle);
387+
application_worker_diagnostic, worker, duration_origin_ms, &worker->handle);
386388
if (result != 0) {
387389
application_worker_destroy_default(NULL, worker);
388390
*worker_out = NULL;
@@ -1377,10 +1379,13 @@ static application_attempt_status_t application_job_run_attempt(cbm_daemon_appli
13771379
}
13781380

13791381
cbm_daemon_application_worker_t worker = NULL;
1382+
if (job->request_started_ms == 0) {
1383+
job->request_started_ms = cbm_index_worker_now_ms();
1384+
}
13801385
application_tmp_lock();
1381-
int start_result =
1382-
application->worker_ops.start(application->worker_ops.context, job->args_json,
1383-
memory_budget_bytes, marker_path, quarantine_path, &worker);
1386+
int start_result = application->worker_ops.start(
1387+
application->worker_ops.context, job->args_json, memory_budget_bytes, marker_path,
1388+
quarantine_path, job->request_started_ms, &worker);
13841389
application_tmp_unlock();
13851390
if (start_result != 0 || !worker) {
13861391
return application_job_cancel_requested(job) ? APPLICATION_ATTEMPT_CANCELLED

‎src/daemon/application.h‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -31,7 +31,7 @@ typedef void *cbm_daemon_application_update_worker_t;
3131
typedef struct {
3232
void *context;
3333
int (*start)(void *context, const char *args_json, size_t memory_budget_bytes,
34-
const char *marker_file, const char *quarantine_file,
34+
const char *marker_file, const char *quarantine_file, uint64_t duration_origin_ms,
3535
cbm_daemon_application_worker_t *worker_out);
3636
cbm_index_worker_poll_t (*poll)(void *context, cbm_daemon_application_worker_t worker,
3737
const cbm_index_worker_result_t **result_out);

‎src/foundation/subprocess.c‎

Lines changed: 11 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -513,6 +513,12 @@ static cbm_proc_tree_rss_status_t cbm_subprocess_tree_rss_platform(cbm_subproces
513513
bool root_failed = false;
514514
for (DWORD index = 0; index < processes->NumberOfProcessIdsInList; index++) {
515515
DWORD pid = (DWORD)processes->ProcessIdList[index];
516+
/* GetProcessMemoryInfo documents PROCESS_QUERY_INFORMATION |
517+
* PROCESS_VM_READ. PROCESS_QUERY_LIMITED_INFORMATION is a lesser
518+
* alternative on newer Windows; the Job Object membership list
519+
* already closes most of the pid-reuse window between listing and
520+
* opening. A member we cannot open is a probe failure when it is
521+
* the root. */
516522
HANDLE member = OpenProcess(PROCESS_QUERY_INFORMATION | PROCESS_VM_READ, FALSE, pid);
517523
if (!member) {
518524
root_failed = root_failed || pid == process->process_id;
@@ -592,7 +598,11 @@ static cbm_proc_tree_rss_status_t cbm_subprocess_tree_rss_platform(cbm_subproces
592598
root_failed = root_failed || pids[index] == process->pid;
593599
continue;
594600
}
595-
cbm_rss_add_saturated(&total, usage.ri_resident_size);
601+
/* phys_footprint is what the OS charges, matching cbm_mem_charged().
602+
* ri_resident_size still counts pages mimalloc already marked
603+
* MADV_FREE_REUSABLE, so a Mac worker can look like 17 GB RSS while
604+
* the kernel holds it to 5.5 GB. */
605+
cbm_rss_add_saturated(&total, usage.ri_phys_footprint);
596606
measured++;
597607
}
598608
cbm_free(CBM_MEM_CLASS_OTHER, pids);

‎src/foundation/subprocess.h‎

Lines changed: 6 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -155,10 +155,12 @@ int cbm_subprocess_spawn(const cbm_proc_opts_t *opts, cbm_subprocess_t **out);
155155
* tree when cancel_grace_ms elapses. Callers must keep polling to make progress. */
156156
cbm_proc_poll_t cbm_subprocess_poll(cbm_subprocess_t *process, cbm_proc_result_t *out);
157157

158-
/* Read the current resident-set size of the complete contained process tree.
159-
* OK returns an overflow-safe byte total, EMPTY means the owned tree currently
160-
* has no observable members, and ERROR means no trustworthy measurement could
161-
* be obtained. Individual processes that exit during enumeration are ignored. */
158+
/* Read the current charged memory of the complete contained process tree.
159+
* On macOS this is phys_footprint (what the OS holds against the process);
160+
* elsewhere it is RSS. OK returns an overflow-safe byte total, EMPTY means
161+
* the owned tree currently has no observable members, and ERROR means no
162+
* trustworthy measurement could be obtained. Individual processes that exit
163+
* during enumeration are ignored. */
162164
cbm_proc_tree_rss_status_t cbm_subprocess_tree_rss_bytes(cbm_subprocess_t *process,
163165
uint64_t *rss_bytes);
164166
bool cbm_subprocess_root_running(const cbm_subprocess_t *process);

‎src/mcp/index_supervisor.c‎

Lines changed: 24 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -881,7 +881,8 @@ static int worker_start_internal(const char *args_json, size_t memory_budget_byt
881881
const cbm_index_resource_policy_t *resource_policy,
882882
bool single_thread, const char *marker_file,
883883
const char *quarantine_file, cbm_proc_log_cb log_callback,
884-
void *log_context, cbm_index_worker_handle_t **handle_out) {
884+
void *log_context, uint64_t duration_origin_ms,
885+
cbm_index_worker_handle_t **handle_out) {
885886
if (handle_out) {
886887
*handle_out = NULL;
887888
}
@@ -985,45 +986,54 @@ static int worker_start_internal(const char *args_json, size_t memory_budget_byt
985986
return -1;
986987
}
987988
if (handle->resource_policy.max_duration_ms.enabled) {
988-
handle->started_ms = worker_resource_now_ms();
989+
/* Duration is per request, not per spawn: a crash/hang recovery must
990+
* not reset the clock. Callers pass the request origin; 0 means now. */
991+
uint64_t origin = duration_origin_ms != 0 ? duration_origin_ms : worker_resource_now_ms();
992+
handle->started_ms = origin;
989993
}
990994
*handle_out = handle;
991995
return 0;
992996
}
993997

998+
uint64_t cbm_index_worker_now_ms(void) {
999+
return worker_resource_now_ms();
1000+
}
1001+
9941002
int cbm_index_worker_start_with_log(const char *args_json, size_t memory_budget_bytes,
9951003
bool single_thread, const char *marker_file,
9961004
const char *quarantine_file, cbm_proc_log_cb log_callback,
9971005
void *log_context, cbm_index_worker_handle_t **handle_out) {
9981006
return worker_start_internal(args_json, memory_budget_bytes, NULL, single_thread, marker_file,
999-
quarantine_file, log_callback, log_context, handle_out);
1007+
quarantine_file, log_callback, log_context, 0, handle_out);
10001008
}
10011009

10021010
int cbm_index_worker_start(const char *args_json, size_t memory_budget_bytes, bool single_thread,
10031011
const char *marker_file, const char *quarantine_file,
10041012
cbm_index_worker_handle_t **handle_out) {
10051013
return worker_start_internal(args_json, memory_budget_bytes, NULL, single_thread, marker_file,
1006-
quarantine_file, NULL, NULL, handle_out);
1014+
quarantine_file, NULL, NULL, 0, handle_out);
10071015
}
10081016

10091017
int cbm_index_worker_start_with_policy(const char *args_json, size_t memory_budget_bytes,
10101018
const cbm_index_resource_policy_t *resource_policy,
10111019
bool single_thread, const char *marker_file,
1012-
const char *quarantine_file,
1020+
const char *quarantine_file, uint64_t duration_origin_ms,
10131021
cbm_index_worker_handle_t **handle_out) {
10141022
return worker_start_internal(args_json, memory_budget_bytes, resource_policy, single_thread,
1015-
marker_file, quarantine_file, NULL, NULL, handle_out);
1023+
marker_file, quarantine_file, NULL, NULL, duration_origin_ms,
1024+
handle_out);
10161025
}
10171026

10181027
int cbm_index_worker_start_with_policy_log(const char *args_json, size_t memory_budget_bytes,
10191028
const cbm_index_resource_policy_t *resource_policy,
10201029
bool single_thread, const char *marker_file,
10211030
const char *quarantine_file,
10221031
cbm_proc_log_cb log_callback, void *log_context,
1032+
uint64_t duration_origin_ms,
10231033
cbm_index_worker_handle_t **handle_out) {
10241034
return worker_start_internal(args_json, memory_budget_bytes, resource_policy, single_thread,
10251035
marker_file, quarantine_file, log_callback, log_context,
1026-
handle_out);
1036+
duration_origin_ms, handle_out);
10271037
}
10281038

10291039
cbm_index_worker_poll_t cbm_index_worker_poll(cbm_index_worker_handle_t *handle,
@@ -1167,14 +1177,15 @@ static int worker_spawn_internal(const char *args_json,
11671177
bool single_thread, const char *marker_file,
11681178
const char *quarantine_file, cbm_proc_log_cb log_callback,
11691179
void *log_context, const atomic_int *cancel_requested,
1170-
cbm_index_worker_result_t *result) {
1180+
uint64_t duration_origin_ms, cbm_index_worker_result_t *result) {
11711181
if (!result) {
11721182
return -1;
11731183
}
11741184
worker_result_init(result);
11751185
cbm_index_worker_handle_t *handle = NULL;
11761186
if (worker_start_internal(args_json, 0, resource_policy, single_thread, marker_file,
1177-
quarantine_file, log_callback, log_context, &handle) != 0) {
1187+
quarantine_file, log_callback, log_context, duration_origin_ms,
1188+
&handle) != 0) {
11781189
return -1;
11791190
}
11801191
const cbm_index_worker_result_t *cached = NULL;
@@ -1206,16 +1217,17 @@ int cbm_index_spawn_worker_with_log_cancel(const char *args_json, bool single_th
12061217
const atomic_int *cancel_requested,
12071218
cbm_index_worker_result_t *result) {
12081219
return worker_spawn_internal(args_json, NULL, single_thread, marker_file, quarantine_file,
1209-
log_callback, log_context, cancel_requested, result);
1220+
log_callback, log_context, cancel_requested, 0, result);
12101221
}
12111222

12121223
int cbm_index_spawn_worker_with_policy_log_cancel(
12131224
const char *args_json, const cbm_index_resource_policy_t *resource_policy, bool single_thread,
12141225
const char *marker_file, const char *quarantine_file, cbm_proc_log_cb log_callback,
1215-
void *log_context, const atomic_int *cancel_requested, cbm_index_worker_result_t *result) {
1226+
void *log_context, const atomic_int *cancel_requested, uint64_t duration_origin_ms,
1227+
cbm_index_worker_result_t *result) {
12161228
return worker_spawn_internal(args_json, resource_policy, single_thread, marker_file,
12171229
quarantine_file, log_callback, log_context, cancel_requested,
1218-
result);
1230+
duration_origin_ms, result);
12191231
}
12201232

12211233
int cbm_index_spawn_worker_with_log(const char *args_json, bool single_thread,

0 commit comments

Comments
 (0)