Skip to content

Commit 14d4388

Browse files
committed
feat(scala): parse import selectors and resolve package-aware imports
Scala imports were parsed with the generic import fallback, which ignored selector groups, aliases (`{A => B}`) and wildcards, and the declared `package` clause was not recorded. IMPORTS edges then had to be guessed from the physical directory layout, which does not have to mirror the package in Scala projects. - parse Scala `import` statements into one CBMImport per selector, keeping the alias as the local name and the full path as the target - record the `package` clause as the file namespace; consecutive braceless clauses (`package a` / `package b`) are joined into `a.b` - thread the file language through import resolution so Scala imports are resolved against declared packages; the resolver fails closed at every step instead of guessing - build a (package, top-level name) -> node index once per pass, next to the namespace map, so package membership is an O(1) lookup instead of a scan of the package's file list per same-named hit; every file declaring a namespace contributes, so a Scala import of a Java class in a mixed package resolves. Only the Scala resolver reads it, so it is built only when a Scala file is among the pass's files - "top level" is decided against the file's language-aware module QN (stem vs directory), so a Java nested type (`Request.Builder`, one segment below the directory module) is not mistaken for a top-level `Builder` of the package - `import pkg.Name` binds only a top-level `Name` of a file declaring `pkg`: nested classes and methods neither match nor make the real top-level symbol look ambiguous; two top-level declarations of the same name in a split package yield no edge - `import pkg.Owner.member` binds the member under the owner's QN and otherwise yields no edge, so a missing member never turns `member()` into a CALLS edge to the owner class via resolve_import_map - `import pkg._` binds the lexicographically smallest declaring file so the target is identical on every platform; the index records that file (and the runner-up, for self-import) per package, so the pick is O(1) instead of a scan of the package's file list - incremental parity: both incremental routes hand the passes only the changed files, so the namespace map and the Scala index used to know only the changed files' packages, and an import into an unchanged package resolved through the symbol fallback (or, fail-closed, not at all) until the next full index. The passes now persist each file's package clause on its File node, and the incremental routes read the clauses of the files that stay back into the pass context so the maps cover the whole project exactly as on a full build - Scala package clauses feed only the Scala resolver: the generic first-declaring-file walk skips Scala files, so Java/Kotlin imports of a shared package prefix keep resolving exactly as before - import-map lookups therefore bind an aliased local name to the exact imported object, so `Alias.method()` resolves with the `import_map` strategy Tests: extraction of selectors/aliases, the package namespace and chained package clauses; edge_imports guards that packages may ignore the physical layout, that a Java import never binds to a Scala prefix declarer, that a Scala import binds a Java class of a mixed package, that a Java import of a package declared by Java and Scala files still binds the Java file, that two same-named top-level symbols in a split package give no edge, that a nested same-named symbol is ignored, that an unresolvable `Owner.member` gives neither an IMPORTS nor a CALLS edge to the owner, that chained package clauses resolve, that a Java nested type of the same name neither binds nor makes the top-level Scala symbol ambiguous, and that an incremental reindex (closure-repair and legacy-partial route, each proven by the route probe) keeps the IMPORTS and CALLS into an unchanged package; pipeline tests for alias resolution (sequential + parallel), for member imports in a split package, and that the Scala index is built only when a Scala file is among the importers. Closes DeusData#2153 Signed-off-by: Hubert Tarnacki <[email protected]>
1 parent 268a9d8 commit 14d4388

11 files changed

Lines changed: 1609 additions & 123 deletions

‎internal/cbm/extract_imports.c‎

Lines changed: 179 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,7 @@ static void parse_go_imports(CBMExtractCtx *ctx);
2424
static void parse_python_imports(CBMExtractCtx *ctx);
2525
static void parse_es_imports(CBMExtractCtx *ctx);
2626
static void parse_java_imports(CBMExtractCtx *ctx);
27+
static void parse_scala_imports(CBMExtractCtx *ctx);
2728
static void parse_rust_imports(CBMExtractCtx *ctx);
2829
static void parse_c_imports(CBMExtractCtx *ctx);
2930
static void parse_ruby_imports(CBMExtractCtx *ctx);
@@ -984,6 +985,146 @@ static void parse_generic_imports(CBMExtractCtx *ctx, const char *node_type) {
984985
ts_tree_cursor_delete(&cursor);
985986
}
986987

988+
// --- Scala imports ---
989+
990+
static char *scala_trim_span(CBMArena *a, const char *start, const char *end) {
991+
while (start < end && isspace((unsigned char)*start)) {
992+
start++;
993+
}
994+
while (end > start && isspace((unsigned char)end[-1])) {
995+
end--;
996+
}
997+
return cbm_arena_strndup(a, start, (size_t)(end - start));
998+
}
999+
1000+
static void scala_emit_import(CBMExtractCtx *ctx, const char *path, const char *local) {
1001+
if (!path || !path[0] || !local || !local[0]) {
1002+
return;
1003+
}
1004+
CBMImport imp = {.local_name = cbm_arena_strdup(ctx->arena, local),
1005+
.module_path = cbm_arena_strdup(ctx->arena, path)};
1006+
cbm_imports_push(&ctx->result->imports, ctx->arena, imp);
1007+
}
1008+
1009+
static void scala_parse_selector(CBMExtractCtx *ctx, const char *prefix, const char *start,
1010+
const char *end) {
1011+
char *selector = scala_trim_span(ctx->arena, start, end);
1012+
if (!selector || !selector[0] ||
1013+
(strncmp(selector, "given", 5) == 0 &&
1014+
(selector[5] == '\0' || isspace((unsigned char)selector[5])))) {
1015+
return;
1016+
}
1017+
if (strcmp(selector, "*") == 0 || strcmp(selector, "_") == 0) {
1018+
scala_emit_import(ctx, prefix, "*");
1019+
return;
1020+
}
1021+
1022+
char *rename = strstr(selector, "=>");
1023+
size_t op_len = 2;
1024+
if (!rename) {
1025+
rename = strstr(selector, " as ");
1026+
op_len = 4;
1027+
}
1028+
char *name = selector;
1029+
char *local = selector;
1030+
if (rename) {
1031+
local = scala_trim_span(ctx->arena, rename + op_len, selector + strlen(selector));
1032+
name = scala_trim_span(ctx->arena, selector, rename);
1033+
if (strcmp(local, "_") == 0 || strcmp(local, "*") == 0) {
1034+
return; // Scala 2 hidden selector: `Name => _`
1035+
}
1036+
}
1037+
if (!name[0] || !local[0]) {
1038+
return;
1039+
}
1040+
const char *path =
1041+
prefix && prefix[0] ? cbm_arena_sprintf(ctx->arena, "%s.%s", prefix, name) : name;
1042+
scala_emit_import(ctx, path, local);
1043+
}
1044+
1045+
static void scala_parse_import_expr(CBMExtractCtx *ctx, const char *start, const char *end) {
1046+
char *expr = scala_trim_span(ctx->arena, start, end);
1047+
if (!expr || !expr[0]) {
1048+
return;
1049+
}
1050+
char *open = strchr(expr, '{');
1051+
char *close = strrchr(expr, '}');
1052+
if (open && close && close > open) {
1053+
char *prefix = scala_trim_span(ctx->arena, expr, open);
1054+
size_t plen = strlen(prefix);
1055+
while (plen > 0 && (prefix[plen - 1] == '.' || isspace((unsigned char)prefix[plen - 1]))) {
1056+
prefix[--plen] = '\0';
1057+
}
1058+
const char *part = open + 1;
1059+
for (const char *p = part;; p++) {
1060+
if (p == close || *p == ',') {
1061+
scala_parse_selector(ctx, prefix, part, p);
1062+
if (p == close) {
1063+
break;
1064+
}
1065+
part = p + 1;
1066+
}
1067+
}
1068+
return;
1069+
}
1070+
1071+
char *rename = strstr(expr, " as ");
1072+
if (rename) {
1073+
char *path = scala_trim_span(ctx->arena, expr, rename);
1074+
char *local = scala_trim_span(ctx->arena, rename + 4, expr + strlen(expr));
1075+
scala_emit_import(ctx, path, local);
1076+
return;
1077+
}
1078+
size_t len = strlen(expr);
1079+
if ((len > 2 && strcmp(expr + len - 2, ".*") == 0) ||
1080+
(len > 2 && strcmp(expr + len - 2, "._") == 0)) {
1081+
expr[len - 2] = '\0';
1082+
scala_emit_import(ctx, expr, "*");
1083+
return;
1084+
}
1085+
if (len > 6 && strcmp(expr + len - 6, ".given") == 0) {
1086+
expr[len - 6] = '\0';
1087+
scala_emit_import(ctx, expr, "*");
1088+
return;
1089+
}
1090+
scala_emit_import(ctx, expr, path_last(ctx->arena, expr));
1091+
}
1092+
1093+
static void parse_scala_imports(CBMExtractCtx *ctx) {
1094+
TSTreeCursor cursor = ts_tree_cursor_new(ctx->root);
1095+
if (!ts_tree_cursor_goto_first_child(&cursor)) {
1096+
ts_tree_cursor_delete(&cursor);
1097+
return;
1098+
}
1099+
do {
1100+
TSNode node = ts_tree_cursor_current_node(&cursor);
1101+
if (strcmp(ts_node_type(node), "import_declaration") != 0) {
1102+
continue;
1103+
}
1104+
char *text = cbm_node_text(ctx->arena, node, ctx->source);
1105+
if (!text || strncmp(text, "import", 6) != 0) {
1106+
continue;
1107+
}
1108+
const char *part = text + 6;
1109+
int brace_depth = 0;
1110+
for (const char *p = part;; p++) {
1111+
if (*p == '{') {
1112+
brace_depth++;
1113+
} else if (*p == '}') {
1114+
brace_depth--;
1115+
}
1116+
if (*p == '\0' || (*p == ',' && brace_depth == 0)) {
1117+
scala_parse_import_expr(ctx, part, p);
1118+
if (*p == '\0') {
1119+
break;
1120+
}
1121+
part = p + 1;
1122+
}
1123+
}
1124+
} while (ts_tree_cursor_goto_next_sibling(&cursor));
1125+
ts_tree_cursor_delete(&cursor);
1126+
}
1127+
9871128
// --- Kotlin imports ---
9881129
// tree-sitter-kotlin nests imports: source_file -> import_list -> import_header*.
9891130
// parse_generic_imports only scans the DIRECT children of root, and "import" is
@@ -1659,6 +1800,37 @@ static void parse_embedded_imports(CBMExtractCtx *ctx) {
16591800
}
16601801
}
16611802

1803+
// Scala chains braceless clauses: `package a` followed by `package b` puts the
1804+
// file's symbols in a.b. Append every directly following braceless
1805+
// package_clause to the namespace already recorded from the first one. A
1806+
// braced clause (`package b { ... }`) scopes only its body and is left alone.
1807+
static void join_chained_scala_packages(CBMExtractCtx *ctx, TSTreeCursor *cursor) {
1808+
while (ts_tree_cursor_goto_next_sibling(cursor)) {
1809+
TSNode next = ts_tree_cursor_current_node(cursor);
1810+
const char *next_kind = ts_node_type(next);
1811+
if (strcmp(next_kind, "comment") == 0 || strcmp(next_kind, "block_comment") == 0) {
1812+
continue;
1813+
}
1814+
if (strcmp(next_kind, "package_clause") != 0) {
1815+
return;
1816+
}
1817+
TSNode body = next;
1818+
if (find_first_descendant_of(next, "template_body", &body)) {
1819+
return;
1820+
}
1821+
TSNode name_node = next;
1822+
if (!find_first_descendant_of(next, "package_identifier", &name_node)) {
1823+
return;
1824+
}
1825+
char *segment = cbm_node_text(ctx->arena, name_node, ctx->source);
1826+
if (!segment || !segment[0]) {
1827+
return;
1828+
}
1829+
ctx->result->namespace_name =
1830+
cbm_arena_sprintf(ctx->arena, "%s.%s", ctx->result->namespace_name, segment);
1831+
}
1832+
}
1833+
16621834
// --- Namespace / package declaration capture ---
16631835
// Java/Kotlin/C#/PHP put the file's symbols inside a namespace/package whose
16641836
// name is NOT reflected in the path-based QN scheme. Capturing it lets the
@@ -1670,6 +1842,7 @@ static void capture_namespace_decl(CBMExtractCtx *ctx) {
16701842
"file_scoped_namespace_declaration", // C# 10
16711843
"package_declaration", // Java / Kotlin
16721844
"package_header", // Kotlin
1845+
"package_clause", // Scala
16731846
"namespace_definition", // PHP
16741847
NULL};
16751848
TSTreeCursor cursor = ts_tree_cursor_new(ctx->root);
@@ -1686,6 +1859,7 @@ static void capture_namespace_decl(CBMExtractCtx *ctx) {
16861859
// The namespace name is the first qualified_name / scoped_identifier /
16871860
// namespace_name / identifier descendant.
16881861
static const char *name_kinds[] = {"qualified_name",
1862+
"package_identifier",
16891863
"scoped_identifier",
16901864
"namespace_name",
16911865
"identifier",
@@ -1703,6 +1877,9 @@ static void capture_namespace_decl(CBMExtractCtx *ctx) {
17031877
}
17041878
}
17051879
if (ctx->result->namespace_name) {
1880+
if (strcmp(kind, "package_clause") == 0) {
1881+
join_chained_scala_packages(ctx, &cursor);
1882+
}
17061883
break;
17071884
}
17081885
} while (ts_tree_cursor_goto_next_sibling(&cursor));
@@ -2969,6 +3146,7 @@ void cbm_extract_imports(CBMExtractCtx *ctx) {
29693146
switch (ctx->language) {
29703147
case CBM_LANG_JAVA:
29713148
case CBM_LANG_KOTLIN:
3149+
case CBM_LANG_SCALA:
29723150
case CBM_LANG_CSHARP:
29733151
case CBM_LANG_PHP:
29743152
capture_namespace_decl(ctx);
@@ -2996,7 +3174,7 @@ void cbm_extract_imports(CBMExtractCtx *ctx) {
29963174
parse_kotlin_imports(ctx);
29973175
break;
29983176
case CBM_LANG_SCALA:
2999-
parse_generic_imports(ctx, "import_declaration");
3177+
parse_scala_imports(ctx);
30003178
break;
30013179
case CBM_LANG_CSHARP:
30023180
parse_csharp_imports(ctx);

‎scripts/memory-core-baseline.txt‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -62,7 +62,7 @@ src/pipeline/pass_compile_commands.c 17
6262
src/pipeline/pass_complexity.c 15
6363
src/pipeline/pass_configlink.c 3
6464
src/pipeline/pass_cross_repo.c 8
65-
src/pipeline/pass_definitions.c 14
65+
src/pipeline/pass_definitions.c 12
6666
src/pipeline/pass_enrichment.c 15
6767
src/pipeline/pass_ensemble_routing.c 10
6868
src/pipeline/pass_envscan.c 2
@@ -71,15 +71,15 @@ src/pipeline/pass_importance.c 4
7171
src/pipeline/pass_infrascan.c 2
7272
src/pipeline/pass_k8s.c 6
7373
src/pipeline/pass_lsp_cross.c 50
74-
src/pipeline/pass_parallel.c 31
75-
src/pipeline/pass_pkgmap.c 82
74+
src/pipeline/pass_parallel.c 29
75+
src/pipeline/pass_pkgmap.c 80
7676
src/pipeline/pass_semantic.c 3
7777
src/pipeline/pass_tests.c 3
7878
src/pipeline/pass_usages.c 4
7979
src/pipeline/path_alias.c 42
8080
src/pipeline/pipeline.c 134
8181
src/pipeline/pipeline_delta.c 19
82-
src/pipeline/pipeline_incremental.c 144
82+
src/pipeline/pipeline_incremental.c 142
8383
src/pipeline/registry.c 17
8484
src/pipeline/worker_pool.c 2
8585
src/store/store.c 358

‎src/pipeline/pass_calls.c‎

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -118,7 +118,7 @@ static const cbm_gbuf_node_t *pc_find_by_computed_qn(const cbm_gbuf_t *gbuf, con
118118
return n;
119119
}
120120

121-
static int build_import_map(cbm_pipeline_ctx_t *ctx, const char *rel_path,
121+
static int build_import_map(cbm_pipeline_ctx_t *ctx, const char *rel_path, CBMLanguage language,
122122
const CBMFileResult *result, const char ***out_keys,
123123
const char ***out_vals, int *out_count) {
124124
*out_keys = NULL;
@@ -141,7 +141,7 @@ static int build_import_map(cbm_pipeline_ctx_t *ctx, const char *rel_path,
141141
continue;
142142
}
143143
const cbm_gbuf_node_t *target =
144-
cbm_pipeline_resolve_import_node(ctx, rel_path, file_qn, imp, NULL);
144+
cbm_pipeline_resolve_import_node(ctx, rel_path, file_qn, language, imp, NULL, NULL);
145145
if (!target) {
146146
continue;
147147
}
@@ -1053,7 +1053,7 @@ int cbm_pipeline_pass_calls(cbm_pipeline_ctx_t *ctx, const cbm_file_info_t *file
10531053
const char **imp_keys = NULL;
10541054
const char **imp_vals = NULL;
10551055
int imp_count = 0;
1056-
build_import_map(ctx, rel, result, &imp_keys, &imp_vals, &imp_count);
1056+
build_import_map(ctx, rel, files[i].language, result, &imp_keys, &imp_vals, &imp_count);
10571057

10581058
/* Compute module QN for same-module resolution (directory-based for
10591059
* Java/Go so it matches their def-node QNs in the registry). */
@@ -1216,7 +1216,8 @@ void cbm_pipeline_pass_fastapi_depends(cbm_pipeline_ctx_t *ctx, const cbm_file_i
12161216
const char **imp_keys = NULL;
12171217
const char **imp_vals = NULL;
12181218
int imp_count = 0;
1219-
build_import_map(ctx, files[i].rel_path, result, &imp_keys, &imp_vals, &imp_count);
1219+
build_import_map(ctx, files[i].rel_path, files[i].language, result, &imp_keys, &imp_vals,
1220+
&imp_count);
12201221

12211222
for (int d = 0; d < result->defs.count; d++) {
12221223
CBMDefinition *def = &result->defs.items[d];

‎src/pipeline/pass_definitions.c‎

Lines changed: 20 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -512,7 +512,9 @@ int cbm_pipeline_create_env_configures_for_file(cbm_pipeline_ctx_t *ctx,
512512
/* Create IMPORTS edges for one file's imports. Mirrors the resolution
513513
* logic in pass_parallel.c register_and_link_def — keep the two in sync. */
514514
static int create_import_edges_for_file(cbm_pipeline_ctx_t *ctx, const CBMFileResult *result,
515-
const char *rel, CBMHashTable *namespace_map) {
515+
const char *rel, CBMLanguage language,
516+
CBMHashTable *namespace_map,
517+
const cbm_scala_index_t *scala_index) {
516518
int count = 0;
517519
char *file_qn = cbm_pipeline_fqn_compute(ctx->project_name, rel, "__file__");
518520
const cbm_gbuf_node_t *source_node = cbm_gbuf_find_by_qn(ctx->gbuf, file_qn);
@@ -525,8 +527,8 @@ static int create_import_edges_for_file(cbm_pipeline_ctx_t *ctx, const CBMFileRe
525527
if (!imp->module_path) {
526528
continue;
527529
}
528-
const cbm_gbuf_node_t *target =
529-
cbm_pipeline_resolve_import_node(ctx, rel, file_qn, imp, namespace_map);
530+
const cbm_gbuf_node_t *target = cbm_pipeline_resolve_import_node(
531+
ctx, rel, file_qn, language, imp, namespace_map, scala_index);
530532
if (target && target->id != source_node->id) {
531533
char esc_ln[CBM_SZ_128];
532534
cbm_json_escape(esc_ln, sizeof(esc_ln), imp->local_name ? imp->local_name : "");
@@ -938,7 +940,8 @@ int cbm_pipeline_pass_definitions(cbm_pipeline_ctx_t *ctx, const cbm_file_info_t
938940
* resolve to defs already in the graph, but the file's
939941
* own defs are now persisted before the lookup. No namespace
940942
* map is available without the cache (single-file scope). */
941-
total_imports += create_import_edges_for_file(ctx, result, rel, NULL);
943+
total_imports +=
944+
create_import_edges_for_file(ctx, result, rel, files[i].language, NULL, NULL);
942945
create_channel_edges_for_file(ctx, result, rel);
943946
cbm_pipeline_create_env_configures_for_file(ctx, result, rel);
944947
cbm_free_result(result);
@@ -952,16 +955,19 @@ int cbm_pipeline_pass_definitions(cbm_pipeline_ctx_t *ctx, const cbm_file_info_t
952955
if (local_cache) {
953956
/* Build a namespace/package → File-QN map so that namespace imports
954957
* (C# `using`, Java/Kotlin `import`, PHP `use`) resolve to the file
955-
* that declares the namespace. */
956-
const char **rels = (const char **)calloc((size_t)file_count, sizeof(char *));
957-
if (rels) {
958+
* that declares the namespace, and the Scala top-level index. */
959+
CBMHashTable *namespace_map = NULL;
960+
cbm_scala_index_t *scala_index = NULL;
961+
const char **namespaces =
962+
cbm_calloc(CBM_MEM_CLASS_OTHER, (size_t)file_count * sizeof(char *));
963+
if (namespaces) {
958964
for (int i = 0; i < file_count; i++) {
959-
rels[i] = files[i].rel_path;
965+
namespaces[i] = local_cache[i] ? local_cache[i]->namespace_name : NULL;
960966
}
967+
cbm_pipeline_import_maps_build(ctx, files, file_count, namespaces, &namespace_map,
968+
&scala_index);
969+
cbm_free(CBM_MEM_CLASS_OTHER, namespaces);
961970
}
962-
CBMHashTable *namespace_map =
963-
cbm_pipeline_namespace_map_build(ctx->project_name, local_cache, rels, file_count);
964-
free(rels);
965971
for (int i = 0; i < file_count; i++) {
966972
if (cbm_pipeline_check_cancel(ctx)) {
967973
break;
@@ -970,12 +976,12 @@ int cbm_pipeline_pass_definitions(cbm_pipeline_ctx_t *ctx, const cbm_file_info_t
970976
if (!result) {
971977
continue;
972978
}
973-
total_imports +=
974-
create_import_edges_for_file(ctx, result, files[i].rel_path, namespace_map);
979+
total_imports += create_import_edges_for_file(
980+
ctx, result, files[i].rel_path, files[i].language, namespace_map, scala_index);
975981
create_channel_edges_for_file(ctx, result, files[i].rel_path);
976982
cbm_pipeline_create_env_configures_for_file(ctx, result, files[i].rel_path);
977983
}
978-
cbm_pipeline_namespace_map_free(namespace_map);
984+
cbm_pipeline_import_maps_free(namespace_map, scala_index);
979985
if (owns_local_cache) {
980986
for (int i = 0; i < file_count; i++) {
981987
if (local_cache[i]) {

0 commit comments

Comments
 (0)