Skip to content

Commit 7187500

Browse files
Flipper1994claude
andcommitted
fix(win): escape spawned command-line args so the index worker gets valid argv
Fixes the Windows-only smoke-windows (standard + ui) failure in the release dry run, where index_repository reported outcome=exit_nonzero and "Indexing worker crashed on a file" on big_templated.hpp. It is not the DeusData#424 allocator class and not big_templated.hpp — the file parses fine; the worker never reaches it. This is the Windows residual of DeusData#838. That issue fixed the CLI parser to accept the supervisor's raw-JSON-positional + --response-out flag layout, verified with a PowerShell repro where the JSON argument arrives intact. But the real supervisor spawns the worker via CreateProcessA, whose command-line re-parse strips the JSON's quotes, so on Windows the now-fixed parser still receives corrupted JSON. On Windows the index supervisor spawns the worker as `<self> cli --index-worker index_repository <args_json> ...`, but cbm_run_win built the CreateProcess command line by wrapping each argv element in bare quotes with no escaping. Windows re-parses that single string back into argv and strips the inner quotes, so the JSON argument {"repo_path":"..."} arrived at the child as {repo_path:...} — invalid JSON. The worker exited non-zero at arg parse ("repo_path is required"); the supervisor classified exit_nonzero and blamed the last-marked file (big_templated.hpp), producing the misleading crash message. POSIX is unaffected: cbm_run_posix passes the argv array straight to execv with no re-parse, which is why only smoke-windows was red while Linux/macOS indexed the same fixture fine. Fix: cbm_build_win_cmdline() applies the Microsoft C runtime quoting rules (quote-wrap + escape embedded quotes and their preceding backslash runs) so the child re-parses byte-identical argv. It is defined unconditionally (pure string logic, no Windows headers) and unit-tested on every platform via a round-trip against a reference CommandLineToArgvW parser. Also in this change (from a full Windows quoting audit of the tree): - ui/http_server.c: the UI index spawn had a byte-for-byte duplicate of the same bug (its own "%s" cmdline). Route it through cbm_build_win_cmdline too. - pipeline/artifact.c: the two `git -C '%s'` shell-outs were single-quoted and unvalidated — broken for spaced repo paths on cmd.exe and the only unguarded git interpolation in the tree. Validate repo_path (cbm_validate_shell_arg + the Windows % ! ^ guard) and use double quotes + the per-platform null device, matching git_context.c (the best-hardened git shell-out). - mcp/index_supervisor.c: log the worker's raw exit_code and KEEP its log on any failure (delete only on a clean run). Previously the log was always deleted and only outcome+signal were logged, so a non-zero worker left nothing to diagnose — the CI blind spot that hid this bug behind a generic message. Verified on a CLANG64 build (the toolchain CI ships): the supervised big_templated.hpp fixture now indexes clean (reap outcome=clean exit_code=0); subprocess suite 11/11, affected suites (artifact, git_context, index_resilience, ui, httpd, mcp, security, str_util) 322/322 green. Refs: DeusData#838, DeusData#423 Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> Signed-off-by: Flipper <[email protected]>
1 parent affa223 commit 7187500

6 files changed

Lines changed: 311 additions & 21 deletions

File tree

‎src/foundation/subprocess.c‎

Lines changed: 93 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -132,25 +132,107 @@ static bool cbm_tail_log(const char *log_file, long *tail_pos, cbm_proc_log_cb c
132132
return progressed;
133133
}
134134

135+
/* ── Windows command-line quoting (pure; unit-tested on every platform) ─────── */
136+
137+
/* Append char `c` to buf[cap], reserving the final byte for a NUL terminator.
138+
* Sets *ovf on overflow and stops writing; pos keeps advancing so the caller
139+
* still detects the overflow after the loop. */
140+
static size_t cbm_cmdline_put(char *buf, size_t cap, size_t pos, char c, bool *ovf) {
141+
if (pos + 1 >= cap) {
142+
*ovf = true;
143+
return pos;
144+
}
145+
buf[pos] = c;
146+
return pos + 1;
147+
}
148+
149+
/* Append one argv element to the command line using the Microsoft C runtime
150+
* quoting rules (see MS "Parsing C Command-Line Arguments"). CreateProcess takes
151+
* a SINGLE string that the child re-parses back into argv, so any element with a
152+
* space, tab or double-quote must be wrapped in quotes and its embedded quotes /
153+
* preceding backslashes escaped. Without this a JSON argument like
154+
* {"repo_path":"C:/r"} loses its inner quotes and the child receives the invalid
155+
* {repo_path:C:/r} — the Windows-only index-worker cmdline-quoting bug (the worker exited
156+
* non-zero at JSON-arg parse, misattributed to the last-marked file). POSIX is
157+
* unaffected: cbm_run_posix passes the argv array straight to execv. */
158+
static size_t cbm_cmdline_append_arg(char *buf, size_t cap, size_t pos, const char *arg, bool first,
159+
bool *ovf) {
160+
if (!first) {
161+
pos = cbm_cmdline_put(buf, cap, pos, ' ', ovf);
162+
}
163+
pos = cbm_cmdline_put(buf, cap, pos, '"', ovf);
164+
for (const char *p = arg; *p;) {
165+
size_t nbs = 0;
166+
while (*p == '\\') {
167+
nbs++;
168+
p++;
169+
}
170+
if (*p == '\0') {
171+
/* Trailing backslashes precede the closing quote: double them so the
172+
* quote stays a delimiter, not an escaped literal. */
173+
for (size_t k = 0; k < nbs * 2; k++) {
174+
pos = cbm_cmdline_put(buf, cap, pos, '\\', ovf);
175+
}
176+
break;
177+
}
178+
if (*p == '"') {
179+
/* N backslashes then a quote -> 2N+1 backslashes then an escaped quote. */
180+
for (size_t k = 0; k < nbs * 2 + 1; k++) {
181+
pos = cbm_cmdline_put(buf, cap, pos, '\\', ovf);
182+
}
183+
pos = cbm_cmdline_put(buf, cap, pos, '"', ovf);
184+
p++;
185+
} else {
186+
for (size_t k = 0; k < nbs; k++) {
187+
pos = cbm_cmdline_put(buf, cap, pos, '\\', ovf);
188+
}
189+
pos = cbm_cmdline_put(buf, cap, pos, *p, ovf);
190+
p++;
191+
}
192+
}
193+
pos = cbm_cmdline_put(buf, cap, pos, '"', ovf);
194+
return pos;
195+
}
196+
197+
/* Build a full Windows CreateProcess command line from a NULL-terminated argv,
198+
* applying the MS C runtime quoting rules so the child re-parses byte-identical
199+
* argv. Returns true on success, false if the result would overflow `buf`.
200+
*
201+
* Defined unconditionally (pure string logic, no Windows headers) so the quoting
202+
* contract is unit-tested on Linux/macOS CI too — even though the real spawn path
203+
* only runs on Windows. Shared by cbm_run_win AND the UI http_server index spawn
204+
* so both escape identically; a naive `"%s"` wrap silently corrupts any argument
205+
* containing a quote (e.g. the index JSON {"repo_path":"…"}), corrupting the
206+
* spawned child's argv. */
207+
bool cbm_build_win_cmdline(char *buf, size_t cap, const char *const *argv) {
208+
if (!buf || cap == 0 || !argv) {
209+
return false;
210+
}
211+
size_t pos = 0;
212+
bool ovf = false;
213+
for (int i = 0; argv[i]; i++) {
214+
pos = cbm_cmdline_append_arg(buf, cap, pos, argv[i], i == 0, &ovf);
215+
if (ovf) {
216+
return false;
217+
}
218+
}
219+
buf[pos] = '\0';
220+
return true;
221+
}
222+
135223
#ifdef _WIN32
136224

137225
static int cbm_run_win(const cbm_proc_opts_t *opts, cbm_proc_result_t *out) {
138226
const char *bin = opts->bin;
139227
const char *const default_argv[] = {bin, NULL};
140228
const char *const *argv = opts->argv ? opts->argv : default_argv;
141229

142-
/* Build a quoted command line from argv. */
143230
char cmdline[8192];
144-
size_t pos = 0;
145-
for (int i = 0; argv[i]; i++) {
146-
int n = snprintf(cmdline + pos, sizeof(cmdline) - pos, "%s\"%s\"", (i ? " " : ""), argv[i]);
147-
if (n < 0 || (size_t)n >= sizeof(cmdline) - pos) {
148-
out->outcome = CBM_PROC_SPAWN_FAILED;
149-
out->exit_code = -1;
150-
out->term_signal = 0;
151-
return -1;
152-
}
153-
pos += (size_t)n;
231+
if (!cbm_build_win_cmdline(cmdline, sizeof(cmdline), argv)) {
232+
out->outcome = CBM_PROC_SPAWN_FAILED;
233+
out->exit_code = -1;
234+
out->term_signal = 0;
235+
return -1;
154236
}
155237

156238
HANDLE hlog = INVALID_HANDLE_VALUE;

‎src/foundation/subprocess.h‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,7 @@
2121
#define CBM_SUBPROCESS_H
2222

2323
#include <stdbool.h>
24+
#include <stddef.h> /* size_t (cbm_build_win_cmdline) */
2425

2526
/* How a supervised child ended. */
2627
typedef enum {
@@ -73,4 +74,17 @@ cbm_proc_outcome_t cbm_proc_classify(bool exited_normally, int exit_code, int te
7374
/* Stable lowercase name for an outcome (for structured logs / skip reasons). */
7475
const char *cbm_proc_outcome_str(cbm_proc_outcome_t o);
7576

77+
/* Build a Windows CreateProcess command line from a NULL-terminated argv, applying
78+
* the Microsoft C runtime quoting rules (quote-wrap + escape embedded quotes and
79+
* their preceding backslashes) so the spawned child re-parses byte-identical argv.
80+
* Returns true on success, false on overflow (buf then holds a truncated string).
81+
*
82+
* CreateProcess re-parses a SINGLE command string into argv, so a naive `"%s"` wrap
83+
* silently corrupts any element containing a double-quote — e.g. the index worker's
84+
* JSON arg {"repo_path":"…"} arrives as {repo_path:…}, the Windows index-worker bug.
85+
* Exposed (and compiled on every platform — it is pure string logic) so the quoting
86+
* is unit-tested on Linux/macOS CI, and so both spawn sites (cbm_subprocess_run and
87+
* the UI http_server index spawn) escape through one shared, tested implementation. */
88+
bool cbm_build_win_cmdline(char *buf, size_t cap, const char *const *argv);
89+
7690
#endif /* CBM_SUBPROCESS_H */

‎src/mcp/index_supervisor.c‎

Lines changed: 23 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -192,7 +192,10 @@ int cbm_index_spawn_worker(const char *args_json, bool single_thread, const char
192192
opts.argv = argv;
193193
opts.log_file = log_path;
194194
opts.quiet_timeout_ms = worker_quiet_timeout_ms();
195-
opts.delete_log_on_exit = true;
195+
/* We manage log deletion ourselves after reaping (below): keep it on failure
196+
* for post-mortem, delete it only on a clean run. See the observability
197+
* note at the reap site. */
198+
opts.delete_log_on_exit = false;
196199

197200
cbm_proc_result_t r;
198201
int run_rc = cbm_subprocess_run(&opts, &r);
@@ -209,6 +212,7 @@ int cbm_index_spawn_worker(const char *args_json, bool single_thread, const char
209212

210213
if (run_rc != 0) {
211214
(void)remove(resp_path);
215+
(void)remove(log_path); /* empty/partial log from a failed spawn — nothing to keep */
212216
cbm_log_warn("index.supervisor.spawn_failed", "action", "degrade_in_process");
213217
return -1;
214218
}
@@ -222,9 +226,25 @@ int cbm_index_spawn_worker(const char *args_json, bool single_thread, const char
222226
(void)remove(resp_path);
223227

224228
char sig[16];
229+
char exit_buf[16];
225230
snprintf(sig, sizeof(sig), "%d", r.term_signal);
226-
cbm_log_info("index.supervisor.reap", "outcome", cbm_proc_outcome_str(r.outcome), "signal",
227-
sig);
231+
snprintf(exit_buf, sizeof(exit_buf), "%d", r.exit_code);
232+
cbm_log_info("index.supervisor.reap", "outcome", cbm_proc_outcome_str(r.outcome), "exit_code",
233+
exit_buf, "signal", sig);
234+
235+
/* Observability: on a CLEAN run the worker log is noise → delete it. On
236+
* ANY failure keep it and surface its path + raw exit code, so the worker's own
237+
* stdout/stderr (pipeline logs, any assert/abort text, the exact exit code) is
238+
* available post-mortem instead of vanishing. Previously the log was ALWAYS
239+
* deleted and only outcome+signal were logged, so a worker that exited non-zero
240+
* left nothing to diagnose — the CI blind spot that hid this bug (a mangled JSON
241+
* arg → "repo_path is required" exit) behind a generic "crashed on a file". */
242+
if (r.outcome == CBM_PROC_CLEAN) {
243+
(void)remove(log_path);
244+
} else {
245+
cbm_log_warn("index.supervisor.worker_failed", "outcome", cbm_proc_outcome_str(r.outcome),
246+
"exit_code", exit_buf, "log", log_path);
247+
}
228248
return 0;
229249
}
230250

‎src/pipeline/artifact.c‎

Lines changed: 40 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,7 @@ enum {
2121
#include "foundation/compat_fs.h"
2222
#include "foundation/compat.h"
2323
#include "foundation/log.h"
24+
#include "foundation/str_util.h" /* cbm_validate_shell_arg — git shell-out hardening (Windows quoting audit) */
2425

2526
#include "zstd_store.h"
2627

@@ -201,10 +202,43 @@ static int write_file_atomic(const char *path, const char *data, size_t len,
201202
return 0;
202203
}
203204

205+
/* Reject a repo_path that could break out of the quoted `git -C "…"` argument
206+
* before it is interpolated into a shell command. Mirrors git_context.c's
207+
* git_validate_repo_path (the best-hardened git shell-out): cbm_validate_shell_arg
208+
* rejects quote / backslash / substitution metacharacters, and on Windows we also
209+
* reject the cmd.exe expansion metacharacters % ! ^. The command then uses DOUBLE
210+
* quotes (honored by both POSIX sh and cmd.exe, unlike single quotes on cmd.exe),
211+
* so a repo path may legitimately contain spaces. Previously these two sites used
212+
* single-quoted, UNVALIDATED interpolation — broken for spaced paths on Windows and
213+
* the only unguarded git shell-out (Windows quoting audit). */
214+
static bool artifact_git_repo_ok(const char *repo_path) {
215+
if (!cbm_validate_shell_arg(repo_path)) {
216+
return false;
217+
}
218+
#ifdef _WIN32
219+
for (const char *p = repo_path; *p; p++) {
220+
if (*p == '%' || *p == '!' || *p == '^') {
221+
return false;
222+
}
223+
}
224+
#endif
225+
return true;
226+
}
227+
228+
#ifdef _WIN32
229+
#define ARTIFACT_NULL_DEV "NUL"
230+
#else
231+
#define ARTIFACT_NULL_DEV "/dev/null"
232+
#endif
233+
204234
/* Get current git HEAD hash. buf must be >= CBM_SZ_64. Returns false on error. */
205235
static bool git_head_hash(const char *repo_path, char *buf, size_t bufsz) {
206236
char cmd[CBM_SZ_1K];
207-
snprintf(cmd, sizeof(cmd), "git -C '%s' rev-parse HEAD 2>/dev/null", repo_path);
237+
if (!artifact_git_repo_ok(repo_path)) {
238+
buf[0] = '\0';
239+
return false;
240+
}
241+
snprintf(cmd, sizeof(cmd), "git -C \"%s\" rev-parse HEAD 2>" ARTIFACT_NULL_DEV, repo_path);
208242
FILE *fp = cbm_popen(cmd, "r");
209243
if (!fp) {
210244
buf[0] = '\0';
@@ -357,8 +391,12 @@ static void ensure_gitattributes(const char *repo_path) {
357391
}
358392

359393
/* Best-effort: configure merge driver */
394+
if (!artifact_git_repo_ok(repo_path)) {
395+
return;
396+
}
360397
char cmd[CBM_SZ_1K];
361-
snprintf(cmd, sizeof(cmd), "git -C '%s' config merge.ours.driver true 2>/dev/null", repo_path);
398+
snprintf(cmd, sizeof(cmd), "git -C \"%s\" config merge.ours.driver true 2>" ARTIFACT_NULL_DEV,
399+
repo_path);
362400
FILE *p = cbm_popen(cmd, "r");
363401
if (p) {
364402
(void)cbm_pclose(p);

‎src/ui/http_server.c‎

Lines changed: 13 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -33,6 +33,7 @@
3333
#include "foundation/compat_fs.h"
3434
#include "foundation/str_util.h"
3535
#include "foundation/compat_thread.h"
36+
#include "foundation/subprocess.h" /* cbm_build_win_cmdline — shared MS-CRT arg quoting */
3637

3738
#include <sqlite3/sqlite3.h>
3839
#include <yyjson/yyjson.h>
@@ -957,13 +958,20 @@ static void *index_thread_fn(void *arg) {
957958
snprintf(log_file, sizeof(log_file), "%s\\cbm_index_%d.log",
958959
getenv("TEMP") ? getenv("TEMP") : ".", (int)_getpid());
959960

960-
/* Build command line for CreateProcess */
961-
char cmdline[2048];
962-
/* --index-worker: this http_server spawn is already the crash-isolation layer,
961+
/* Build command line for CreateProcess through the shared MS-CRT quoter so the
962+
* JSON arg's embedded quotes survive the child's argv re-parse — a naive
963+
* `"%s"` wrap dropped them, corrupting {"repo_path":"…"} into {repo_path:…}.
964+
* --index-worker: this http_server spawn is already the crash-isolation layer,
963965
* so the child runs indexing in-process rather than spawning its own supervisor
964966
* (avoids redundant process nesting). */
965-
snprintf(cmdline, sizeof(cmdline), "\"%s\" cli --index-worker index_repository \"%s\"", bin,
966-
json_arg);
967+
char cmdline[2048];
968+
const char *const idx_argv[] = {bin, "cli", "--index-worker", "index_repository",
969+
json_arg, NULL};
970+
if (!cbm_build_win_cmdline(cmdline, sizeof(cmdline), idx_argv)) {
971+
snprintf(job->error_msg, sizeof(job->error_msg), "index command line too long");
972+
atomic_store(&job->status, 3);
973+
return NULL;
974+
}
967975

968976
cbm_log_info("ui.index.spawn", "bin", bin, "log", log_file);
969977

0 commit comments

Comments
 (0)