Skip to content

Commit 4e80d2f

Browse files
Kagin007Martin Vogel
authored andcommitted
feat(graph): temporal properties on FILE_CHANGES_WITH edges and File nodes (#257)
Adds three changes that landed together in PR #257, with a JSON-escape hardening on the new File node properties so unusual file extensions can't produce malformed output. Mode A — search_graph results expose node properties_json --------------------------------------------------------- emit_search_results in src/mcp/mcp.c now calls enrich_node_properties for each result and parks the parsed yyjson_doc* in an out-array. The two exit paths in handle_search_graph (success and sq_type_error) walk that array and yyjson_doc_free each entry after yy_doc_to_str runs, so zero-copy strings stay alive until serialization completes. Callers no longer have to round-trip through get_code_snippet just to inspect a node's properties. Mode B — Cypher bare edge return surfaces full properties JSON -------------------------------------------------------------- binding_get_virtual in src/cypher/cypher.c returns e->properties_json (or "{}" if absent) when prop is NULL, instead of "". `RETURN r` on an edge variable is now actually useful for inspecting timestamps, weights, etc. without naming each property up front. Temporal data on FILE_CHANGES_WITH edges and File nodes ------------------------------------------------------- cbm_change_coupling_t gains last_co_change. cbm_commit_files_t and the internal commit_t gain a timestamp field. Both git-log paths (libgit2 and popen --pretty=format:COMMIT:%H:%ct) populate it. cbm_compute_change_coupling carries a parallel pair_timestamps hash table and writes the max timestamp to last_co_change. The edge properties JSON now includes "last_co_change":<unix-epoch>. cbm_pipeline_githistory_compute additionally aggregates per-file change_count and last_modified into a new cbm_file_temporal_t array exposed via result->file_temporal. cbm_pipeline_githistory_apply upserts those values onto matching File nodes so callers can run hotspot queries (most-changed, recently-changed) directly. Both the sequential cbm_pipeline_pass_githistory and the parallel run_githistory in pipeline.c free file_temporal and trigger apply when either couplings or temporal entries are present. The extension stored on the File node properties is now JSON-escaped via cbm_json_escape before snprintf'ing into the props blob. The original PR concatenated the raw extension; a hypothetical path with a quote or backslash in the extension would have produced malformed JSON. Practical risk was tiny (real extensions never contain those characters) but the escape is cheap and correct. Tests ----- - githistory_coupling_carries_last_co_change pins last_co_change as the max of the per-pair commit timestamps even when an unrelated pair sits between them in the input order. - cypher_bare_edge_return_exposes_properties_json regresses Mode B by asserting `RETURN r` on an HTTP_CALLS edge surfaces its url_path in the response. - tool_search_graph_includes_node_properties regresses Mode A by asserting search_graph's response contains the signature/return_type/ is_exported keys from a HandleRequest node's properties_json. Existing githistory tests had their cbm_commit_files_t initializers extended with a zero timestamp (backward-compatible — last_co_change falls back to 0 for callers that don't supply timestamps). Cherry-picked from #257; renamed the timestamp field types to long long for portability across 32-/64-bit time_t platforms, used the existing cbm_json_escape utility instead of raw concatenation for the extension, and added explicit tests for all three changes. Closes #257. Co-authored-by: Adam Schulte <[email protected]>
1 parent cee318b commit 4e80d2f

8 files changed

Lines changed: 279 additions & 15 deletions

File tree

‎src/cypher/cypher.c‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2227,7 +2227,10 @@ static const char *binding_get_virtual(binding_t *b, const char *var, const char
22272227
/* Fall through to normal lookup */
22282228
cbm_edge_t *e = binding_get_edge(b, var);
22292229
if (e) {
2230-
return prop ? edge_prop(e, prop) : "";
2230+
/* Bare `RETURN r` on an edge: surface the full properties JSON
2231+
* (or "{}" if none) so callers can inspect timestamps, weights,
2232+
* etc. without naming each property. */
2233+
return prop ? edge_prop(e, prop) : (e->properties_json ? e->properties_json : "{}");
22312234
}
22322235
cbm_node_t *n = binding_get(b, var);
22332236
if (n) {

‎src/mcp/mcp.c‎

Lines changed: 37 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1278,10 +1278,25 @@ static char *bm25_search(cbm_store_t *store, const char *project, const char *qu
12781278
return json;
12791279
}
12801280

1281-
/* Emit the cbm_store_search results as a JSON "results" array on the doc. */
1281+
/* Forward declaration — defined later. enrich_node_properties parses the
1282+
* node's properties_json and grafts the parsed values onto the result item.
1283+
* It returns the parsed yyjson_doc which must outlive the serialization
1284+
* because yyjson_mut_obj_add_val uses zero-copy strings into that doc. */
1285+
static yyjson_doc *enrich_node_properties(yyjson_mut_doc *doc, yyjson_mut_val *obj,
1286+
const char *properties_json);
1287+
1288+
/* Emit the cbm_store_search results as a JSON "results" array on the doc.
1289+
* Property docs created via enrich_node_properties are collected in
1290+
* *out_pdocs (count in *out_pdoc_count) and must be freed by the caller
1291+
* AFTER serializing doc, since yyjson_mut strings are zero-copy pointers
1292+
* into those parsed docs. The caller also frees out_pdocs itself. */
12821293
static void emit_search_results(yyjson_mut_doc *doc, yyjson_mut_val *root,
12831294
const cbm_search_output_t *out, cbm_store_t *store,
1284-
const char *relationship, bool include_connected, int offset) {
1295+
const char *relationship, bool include_connected, int offset,
1296+
yyjson_doc ***out_pdocs, int *out_pdoc_count) {
1297+
yyjson_doc **pdocs =
1298+
out->count > 0 ? malloc((size_t)out->count * sizeof(yyjson_doc *)) : NULL;
1299+
int pdoc_count = 0;
12851300
yyjson_mut_obj_add_int(doc, root, "total", out->total);
12861301
yyjson_mut_val *results = yyjson_mut_arr(doc);
12871302
for (int i = 0; i < out->count; i++) {
@@ -1298,10 +1313,16 @@ static void emit_search_results(yyjson_mut_doc *doc, yyjson_mut_val *root,
12981313
if (include_connected && sr->node.id > 0) {
12991314
enrich_connected(doc, item, store, sr->node.id, relationship);
13001315
}
1316+
yyjson_doc *pdoc = enrich_node_properties(doc, item, sr->node.properties_json);
1317+
if (pdoc && pdocs) {
1318+
pdocs[pdoc_count++] = pdoc;
1319+
}
13011320
yyjson_mut_arr_add_val(results, item);
13021321
}
13031322
yyjson_mut_obj_add_val(doc, root, "results", results);
13041323
yyjson_mut_obj_add_bool(doc, root, "has_more", out->total > offset + out->count);
1324+
*out_pdocs = pdocs;
1325+
*out_pdoc_count = pdoc_count;
13051326
}
13061327

13071328
/* Extract keyword strings from a yyjson array into `keywords`. Returns the
@@ -1444,7 +1465,10 @@ static char *handle_search_graph(cbm_mcp_server_t *srv, const char *args) {
14441465
yyjson_mut_val *root = yyjson_mut_obj(doc);
14451466
yyjson_mut_doc_set_root(doc, root);
14461467

1447-
emit_search_results(doc, root, &out, store, relationship, include_connected, offset);
1468+
yyjson_doc **props_docs = NULL;
1469+
int props_doc_count = 0;
1470+
emit_search_results(doc, root, &out, store, relationship, include_connected, offset,
1471+
&props_docs, &props_doc_count);
14481472

14491473
/* Add diagnostic hint when zero results */
14501474
if (out.total == 0) {
@@ -1467,6 +1491,10 @@ static char *handle_search_graph(cbm_mcp_server_t *srv, const char *args) {
14671491
bool sq_type_error = run_semantic_query(doc, root, args, store, project, limit);
14681492

14691493
if (sq_type_error) {
1494+
for (int pi = 0; pi < props_doc_count; pi++) {
1495+
yyjson_doc_free(props_docs[pi]);
1496+
}
1497+
free(props_docs);
14701498
yyjson_mut_doc_free(doc);
14711499
cbm_store_search_free(&out);
14721500
free(project);
@@ -1484,6 +1512,12 @@ static char *handle_search_graph(cbm_mcp_server_t *srv, const char *args) {
14841512
}
14851513

14861514
char *json = yy_doc_to_str(doc);
1515+
/* Property docs are zero-copy referenced by the mut doc — they must
1516+
* outlive yy_doc_to_str. Free them once serialization is complete. */
1517+
for (int pi = 0; pi < props_doc_count; pi++) {
1518+
yyjson_doc_free(props_docs[pi]);
1519+
}
1520+
free(props_docs);
14871521
yyjson_mut_doc_free(doc);
14881522
cbm_store_search_free(&out);
14891523

‎src/pipeline/pass_githistory.c‎

Lines changed: 107 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -87,6 +87,7 @@ typedef struct {
8787
char **files;
8888
int count;
8989
int cap;
90+
long long timestamp; /* unix epoch of this commit; 0 when unknown */
9091
} commit_t;
9192

9293
static void commit_add_file(commit_t *c, const char *file) {
@@ -189,6 +190,7 @@ static int parse_git_log(const char *repo_path, commit_t **out, int *out_count)
189190
cap *= PAIR_LEN;
190191
commits = safe_realloc(commits, cap * sizeof(commit_t));
191192
}
193+
current.timestamp = (long long)ct;
192194
commits[count++] = current;
193195
} else {
194196
commit_free(&current);
@@ -224,7 +226,7 @@ static int parse_git_log(const char *repo_path, commit_t **out, int *out_count)
224226

225227
char cmd[CBM_SZ_1K];
226228
snprintf(cmd, sizeof(cmd),
227-
"cd '%s' && git log --name-only --pretty=format:COMMIT:%%H "
229+
"cd '%s' && git log --name-only --pretty=format:COMMIT:%%H:%%ct "
228230
"--since='1 year ago' --max-count=10000 2>/dev/null",
229231
repo_path);
230232

@@ -257,6 +259,12 @@ static int parse_git_log(const char *repo_path, commit_t **out, int *out_count)
257259
commits[count++] = current;
258260
memset(&current, 0, sizeof(current));
259261
}
262+
/* Parse the unix timestamp from "COMMIT:<hash>:<unix_epoch>".
263+
* Older callers / stripped-down git output without %ct land on 0. */
264+
const char *hash_end = strchr(line + SLEN("COMMIT:"), ':');
265+
if (hash_end) {
266+
current.timestamp = strtoll(hash_end + 1, NULL, 10);
267+
}
260268
continue;
261269
}
262270

@@ -294,6 +302,7 @@ static void free_counter(const char *key, void *val, void *ud) {
294302
/* Context for collect_coupling_result callback. */
295303
typedef struct {
296304
CBMHashTable *file_counts;
305+
CBMHashTable *pair_timestamps; /* pair_key → long long*: max commit ts */
297306
cbm_change_coupling_t *out;
298307
int out_count;
299308
int max_out;
@@ -344,12 +353,18 @@ static void collect_coupling_cb(const char *pair_key, void *val, void *ud) {
344353
snprintf(cc->file_b, sizeof(cc->file_b), "%s", file_b);
345354
cc->co_change_count = co_count;
346355
cc->coupling_score = score;
356+
long long *ts = cbm_ht_get(cctx->pair_timestamps, pair_key);
357+
cc->last_co_change = ts ? *ts : 0;
347358
}
348359

349360
int cbm_compute_change_coupling(const cbm_commit_files_t *commits, int commit_count,
350361
cbm_change_coupling_t *out, int max_out) {
351362
CBMHashTable *file_counts = cbm_ht_create(CBM_SZ_1K);
352363
CBMHashTable *pair_counts = cbm_ht_create(CBM_SZ_2K);
364+
/* Parallel table mapping pair_key → max commit timestamp seen for that
365+
* pair, so the resulting edge can carry last_co_change. The pair_counts
366+
* table consumes its key on insert; pair_timestamps gets its own copy. */
367+
CBMHashTable *pair_timestamps = cbm_ht_create(CBM_SZ_2K);
353368

354369
for (int c = 0; c < commit_count; c++) {
355370
if (commits[c].count > GH_MAX_FILES) {
@@ -378,26 +393,39 @@ int cbm_compute_change_coupling(const cbm_commit_files_t *commits, int commit_co
378393
}
379394
size_t la = strlen(a);
380395
size_t lb = strlen(b);
381-
char *pk = malloc(la + SKIP_ONE + lb + SKIP_ONE);
396+
size_t pk_len = la + SKIP_ONE + lb + SKIP_ONE;
397+
char *pk = malloc(pk_len);
382398
memcpy(pk, a, la);
383399
pk[la] = '\x01';
384400
memcpy(pk + la + SKIP_ONE, b, lb + SKIP_ONE);
385401

386402
int *val = cbm_ht_get(pair_counts, pk);
387403
if (val) {
388404
(*val)++;
405+
long long *ts = cbm_ht_get(pair_timestamps, pk);
406+
if (ts && commits[c].timestamp > *ts) {
407+
*ts = commits[c].timestamp;
408+
}
389409
free(pk);
390410
} else {
391411
int *nv = malloc(sizeof(int));
392412
*nv = SKIP_ONE;
413+
/* pair_counts takes ownership of pk; pair_timestamps
414+
* needs its own copy. */
415+
char *pk2 = malloc(pk_len);
416+
memcpy(pk2, pk, pk_len);
393417
cbm_ht_set(pair_counts, pk, nv);
418+
long long *nts = malloc(sizeof(long long));
419+
*nts = commits[c].timestamp;
420+
cbm_ht_set(pair_timestamps, pk2, nts);
394421
}
395422
}
396423
}
397424
}
398425

399426
collect_coupling_ctx_t cctx = {
400427
.file_counts = file_counts,
428+
.pair_timestamps = pair_timestamps,
401429
.out = out,
402430
.out_count = 0,
403431
.max_out = max_out,
@@ -406,6 +434,8 @@ int cbm_compute_change_coupling(const cbm_commit_files_t *commits, int commit_co
406434

407435
cbm_ht_foreach(pair_counts, free_counter, NULL);
408436
cbm_ht_free(pair_counts);
437+
cbm_ht_foreach(pair_timestamps, free_counter, NULL);
438+
cbm_ht_free(pair_timestamps);
409439
cbm_ht_foreach(file_counts, free_counter, NULL);
410440
cbm_ht_free(file_counts);
411441

@@ -416,13 +446,16 @@ int cbm_compute_change_coupling(const cbm_commit_files_t *commits, int commit_co
416446

417447
/* Pre-computed coupling result buffer for fused post-pass parallelism. */
418448
#define MAX_COUPLINGS 8192
449+
#define MAX_FILE_TEMPORAL 16384
419450

420451
/* Compute change couplings without touching the graph buffer.
421452
* Can run on a separate thread while other passes use the gbuf. */
422453
int cbm_pipeline_githistory_compute(const char *repo_path, cbm_githistory_result_t *result) {
423454
result->couplings = NULL;
424455
result->count = 0;
425456
result->commit_count = 0;
457+
result->file_temporal = NULL;
458+
result->file_temporal_count = 0;
426459

427460
commit_t *commits = NULL;
428461
int commit_count = 0;
@@ -446,11 +479,50 @@ int cbm_pipeline_githistory_compute(const char *repo_path, cbm_githistory_result
446479
for (int c = 0; c < commit_count; c++) {
447480
cf[c].files = commits[c].files;
448481
cf[c].count = commits[c].count;
482+
cf[c].timestamp = commits[c].timestamp;
449483
}
450484

451485
cbm_change_coupling_t *couplings = malloc(MAX_COUPLINGS * sizeof(cbm_change_coupling_t));
452486
int coupling_count = cbm_compute_change_coupling(cf, commit_count, couplings, MAX_COUPLINGS);
453487

488+
/* Per-file temporal aggregation: change_count + last_modified.
489+
* Single hash-table pass over the same commit set used for coupling so
490+
* we don't re-scan history. NULL on OOM is fine — the caller still
491+
* gets the couplings. */
492+
cbm_file_temporal_t *ft_arr = malloc(MAX_FILE_TEMPORAL * sizeof(cbm_file_temporal_t));
493+
int ft_count = 0;
494+
if (ft_arr) {
495+
CBMHashTable *file_idx = cbm_ht_create(CBM_SZ_1K);
496+
for (int c = 0; c < commit_count; c++) {
497+
if (cf[c].count > GH_MAX_FILES) {
498+
continue;
499+
}
500+
for (int f = 0; f < cf[c].count; f++) {
501+
const char *fp = cf[c].files[f];
502+
int *idx = cbm_ht_get(file_idx, fp);
503+
if (idx) {
504+
ft_arr[*idx].change_count++;
505+
if (cf[c].timestamp > ft_arr[*idx].last_modified) {
506+
ft_arr[*idx].last_modified = cf[c].timestamp;
507+
}
508+
} else if (ft_count < MAX_FILE_TEMPORAL) {
509+
int new_idx = ft_count++;
510+
snprintf(ft_arr[new_idx].file_path, sizeof(ft_arr[new_idx].file_path), "%s",
511+
fp);
512+
ft_arr[new_idx].change_count = 1;
513+
ft_arr[new_idx].last_modified = cf[c].timestamp;
514+
int *nidx = malloc(sizeof(int));
515+
*nidx = new_idx;
516+
cbm_ht_set(file_idx, strdup(fp), nidx);
517+
}
518+
}
519+
}
520+
cbm_ht_foreach(file_idx, free_counter, NULL);
521+
cbm_ht_free(file_idx);
522+
result->file_temporal = ft_arr;
523+
result->file_temporal_count = ft_count;
524+
}
525+
454526
free(cf);
455527
for (int c = 0; c < commit_count; c++) {
456528
commit_free(&commits[c]);
@@ -483,13 +555,42 @@ int cbm_pipeline_githistory_apply(cbm_pipeline_ctx_t *ctx, const cbm_githistory_
483555
}
484556

485557
char props[CBM_SZ_128];
486-
snprintf(props, sizeof(props), "{\"co_changes\":%d,\"coupling_score\":%.2f}",
487-
cc->co_change_count, cc->coupling_score);
558+
snprintf(props, sizeof(props),
559+
"{\"co_changes\":%d,\"coupling_score\":%.2f,\"last_co_change\":%lld}",
560+
cc->co_change_count, cc->coupling_score, cc->last_co_change);
488561

489562
cbm_gbuf_insert_edge(ctx->gbuf, node_a->id, node_b->id, "FILE_CHANGES_WITH", props);
490563
edge_count++;
491564
}
492565

566+
/* Apply per-file temporal metadata to existing File nodes so callers
567+
* can query change_count / last_modified for hotspot analysis. The
568+
* extension is re-derived and JSON-escaped to keep the props blob
569+
* well-formed even for paths with quotes or backslashes. */
570+
for (int i = 0; i < result->file_temporal_count; i++) {
571+
const cbm_file_temporal_t *ft = &result->file_temporal[i];
572+
char *qn = cbm_pipeline_fqn_compute(ctx->project_name, ft->file_path, "__file__");
573+
const cbm_gbuf_node_t *node = cbm_gbuf_find_by_qn(ctx->gbuf, qn);
574+
free(qn);
575+
if (!node) {
576+
continue;
577+
}
578+
579+
const char *base = strrchr(ft->file_path, '/');
580+
base = base ? base + SKIP_ONE : ft->file_path;
581+
const char *ext = strrchr(base, '.');
582+
char ext_escaped[CBM_SZ_64];
583+
cbm_json_escape(ext_escaped, (int)sizeof(ext_escaped), ext ? ext : "");
584+
585+
char props[CBM_SZ_256];
586+
snprintf(props, sizeof(props),
587+
"{\"extension\":\"%s\",\"last_modified\":%lld,\"change_count\":%d}",
588+
ext_escaped, ft->last_modified, ft->change_count);
589+
590+
cbm_gbuf_upsert_node(ctx->gbuf, node->label, node->name, node->qualified_name,
591+
node->file_path, node->start_line, node->end_line, props);
592+
}
593+
493594
return edge_count;
494595
}
495596

@@ -502,11 +603,12 @@ int cbm_pipeline_pass_githistory(cbm_pipeline_ctx_t *ctx) {
502603
cbm_pipeline_githistory_compute(ctx->repo_path, &result);
503604

504605
int edge_count = 0;
505-
if (result.count > 0) {
606+
if (result.count > 0 || result.file_temporal_count > 0) {
506607
edge_count = cbm_pipeline_githistory_apply(ctx, &result);
507608
}
508609

509610
free(result.couplings);
611+
free(result.file_temporal);
510612

511613
cbm_log_info("pass.done", "pass", "githistory", "commits", itoa_log(result.commit_count),
512614
"edges", itoa_log(edge_count));

‎src/pipeline/pipeline.c‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -733,12 +733,13 @@ static int run_githistory(cbm_pipeline_t *p, cbm_pipeline_ctx_t *ctx) {
733733
}
734734

735735
int gh_edges = 0;
736-
if (gh_result.count > 0) {
736+
if (gh_result.count > 0 || gh_result.file_temporal_count > 0) {
737737
gh_edges = cbm_pipeline_githistory_apply(ctx, &gh_result);
738738
}
739739
cbm_log_info("pass.done", "pass", "githistory", "commits", itoa_buf(gh_result.commit_count),
740740
"edges", itoa_buf(gh_edges));
741741
free(gh_result.couplings);
742+
free(gh_result.file_temporal);
742743
return 0;
743744
}
744745

‎src/pipeline/pipeline_internal.h‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -117,14 +117,30 @@ typedef struct {
117117
char file_b[CBM_SZ_512];
118118
int co_change_count;
119119
double coupling_score;
120+
/* Unix epoch of the most recent commit that touched both files together.
121+
* 0 when no timestamp was available (e.g. older callers / popen path
122+
* without %ct). */
123+
long long last_co_change;
120124
} cbm_change_coupling_t;
121125

122126
/* Commit data for coupling analysis */
123127
typedef struct {
124128
char **files;
125129
int count;
130+
/* Unix epoch of the commit. 0 means unknown — coupling computation
131+
* still works but last_co_change on the resulting edge will be 0. */
132+
long long timestamp;
126133
} cbm_commit_files_t;
127134

135+
/* Per-file temporal metadata. Populated alongside change-coupling so File
136+
* nodes can carry change_count and last_modified for hotspot / risk
137+
* analysis queries. */
138+
typedef struct {
139+
char file_path[CBM_SZ_512];
140+
int change_count;
141+
long long last_modified; /* unix epoch of most recent commit */
142+
} cbm_file_temporal_t;
143+
128144
/* Compute change coupling from commit history.
129145
* Returns number of couplings written to out (up to max_out).
130146
* Caller owns out[]. */
@@ -383,6 +399,10 @@ typedef struct {
383399
cbm_change_coupling_t *couplings;
384400
int count;
385401
int commit_count;
402+
/* Per-file temporal data (change_count + last_modified) for File nodes.
403+
* NULL when the history pass had no commits to analyse. */
404+
cbm_file_temporal_t *file_temporal;
405+
int file_temporal_count;
386406
} cbm_githistory_result_t;
387407

388408
/* Compute change couplings without touching the graph buffer.

0 commit comments

Comments
 (0)