Repository navigation
fix(extract): give each Go init() its own QN - #1915
Conversation
|
Thanks for opening this — it has been seen, and it is queued. This note is automated, but it is not a brush-off: it exists so you know where your PR stands instead of having to guess from silence. Current review status: working through a backlog. What that means for this PR, concretely:
Things that will genuinely speed it up whenever review does happen:
If this fixes a bug, a reproduction we can run is worth more than a description of the symptom. Thanks for contributing, and sorry in advance for the wait. |
b22b1a2 to
4e03c2b
Compare
|
Reviewed the last commit. The diagnosis is right, the safety argument holds, and I have one substantive concern about the suffix itself. The diagnosis is confirmed by the schema. Your safety claim holds, and I checked it both ways. Nothing in The precedents you cite are real. The concern: the suffix is not stable, and both precedents areThis is the part I would like you to weigh, because it is the one property those two precedents share that yours does not. The Rust cfg twin derives its suffix from the
GoogleTest derives its from the macro arguments —
The file basename alone will not do it — your own I am raising this rather than ruling on it — you may have considered ordinals and rejected them for a reason I cannot see from here. If so, say why and I will take it. AlsoThe whole stack is currently Thanks for measuring the loss rather than describing it — "21 decoder-registration |
A Go selector call x.foo() whose receiver the Go LSP cannot type falls through to the generic registry resolver, which binds it by bare short name to an arbitrary same-named project symbol. Stdlib calls are the worst case: f.Close() on an *os.File gets a CALLS edge to whatever project Close wins candidate ranking (measured on a real Go repo: confidence 0.11, 15 candidates; suffix_match + unique_name were 36% of all CALLS edges, and one 14-line stdlib-only function got 3 out of 3 false outbound edges). Extend the TS/JS receiver-aware guard (DeusData#592/DeusData#606) to Go: - extract_calls.c: flag Go call_expression with a selector_expression callee as is_method, mirroring the TS/JS member_expression flag. - registry.c: add cbm_go_suppress_weak_method_match. Unlike the TS/JS drop-list, field_type_hint is KEPT (Go struct fields carry declared types, so the hint is receiver-aware — lrp_go_s8_field_type_hint), and unique_name is dropped only when its confidence carries the import-unreachability penalty (the stdlib-hijack shape); an unpenalized lone candidate inside the caller's import closure never enters the field-type-hint upgrade and must survive. - pass_calls.c / pass_parallel.c: feed the Go gate next to the TS/JS one; the drop still defers to the emit path so service/route/HTTP edges stay main-identical. Reproduce-first: pipeline_go_receiver_suppresses_weak_method_edge is RED without the extractor flag (the f.Close -> project Close edge exists) and GREEN with it; typed same-package calls, bare local calls and import-qualified cross-package calls still resolve. The old extraction contract test used Go as the flag-exempt language — Python takes that role, and extract_go_selector_call_flags_is_method pins the new behavior. Signed-off-by: Ilya Brykau <[email protected]>
4e03c2b to
fcbd12e
Compare
|
Rebased onto current main as part of the 1907→1913→1915→1936 stack; only test insertion-anchor drift in tests/test_pipeline.c, re-anchored. Full scripts/test.sh green on the stack head. |
A Go method's QN was the flat package form (proj.pkg.method) — the receiver was ignored, so every same-name method in a package collided on one QN and the graph upsert kept exactly one node. Measured on a real Go repo: 20 Process/Name methods across 15 files kept 2 nodes; 9 different Task() methods fused into one chimera node carrying all nine bodies' call edges; 19 structs pointed DEFINES_METHOD at a single shared method node; a _test.go mock Close outranked the production Close in the dedupe tie-break. The upsert's own comment calls kind-disambiguated QNs 'the real cure'. Qualify the QN with the receiver type (proj.pkg.Recv.method), the same shape as Go interface members and the C++ out-of-line method path right below it in extract_func_def: - extract_defs.c: def.qualified_name = parent_class + name whenever the receiver type resolves; go_receiver_type_name becomes the shared cbm_go_receiver_type_name (exported via helpers.h) so both sides of the contract use one formula. - extract_unified.c (compute_func_qn): mirror branch for method_declaration, so method-body calls keep exact source attribution instead of degrading to File-node fallback (calls_find_source). - Consumers already agree: pxc_build_lsp_def passes the def QN and parent_class (receiver_type) verbatim into the Go LSP registries, and check_go_class_implements explicitly supports class-qualified method QNs (its path (b)). Side effect: resolve_same_module's exact module.name hash no longer matches concrete methods, which kills the conf-0.9 false edges where an interface-typed call bound to an unrelated same-package method. Fixes DeusData#1909 Signed-off-by: Ilya Brykau <[email protected]>
Go allows any number of init() functions per package — even several in one file — and all of them run at start-up. On the flat QN (proj.pkg.init) they all collided and the graph upsert kept ONE node per package, silently dropping the rest: measured on a real Go repo, an event-source package registering each decoder in its own file's init() collapsed 21 init functions into one node (29 in source, 7 in the graph), erasing the whole registration pattern. Disambiguate with the DeusData#495 cfg-twin pattern: fold the file basename and line into the QN (proj.pkg.init#a.go:L5). Calling init explicitly is illegal in Go, so nothing ever joins on the plain QN; the call-scope side (compute_func_qn) mirrors the exact formula so init-body calls keep their source attribution instead of degrading to the calls_find_source File fallback. Reproduce-first: pipeline_go_multi_init_nodes_survive is RED without the def-side change (node count 1, expected 2) and GREEN with it; extract_go_multiple_init_disambiguated pins two same-file inits to distinct suffixed QNs. Fixes DeusData#1910 Signed-off-by: Ilya Brykau <[email protected]>
fcbd12e to
adff62e
Compare
|
You are right, and the ordinal is strictly better — implemented, not argued with. I had not weighed the incremental path when I chose the line number: on a full rebuild the line suffix is merely ugly, but under delta-merge the QN is node identity, so an insertion above any Now: The test now pins the property you asked for, not just distinctness: exact QNs Branch force-pushed with the stack rebase; full scripts/test.sh green on the stack head. |
|
The update I promised on 3 September, late and with apologies for that: the direction call on #1913 landed on 15 September (the receiver-qualified shape stays, stack order 1907 → 1913 as you already rebased it), so nothing is waiting on a decision any more. #1907 and #1913 are next in line for a branch update and a fresh CI run as soon as the hosted-runner queue drains from tonight's merges; once #1913 is in I will ping here, and a rebase of this branch onto it is then the only step left before review. Thank you for holding the stack steady through all of this, and for the ordinal fix — that reasoning about delta-merge identity was exactly right. |
Brings DeusData#1915 onto main 83109dd, after Stack A (DeusData#1907 and DeusData#1913) landed on main as 5a4af9c. This branch carried pre-rebase copies of the DeusData#1907 and DeusData#1913 commits (651ae3c, dbba757). Main holds their final versions (50ee111, da76194) plus the maintainer follow-ups made before Stack A merged, so every DeusData#1907/DeusData#1913 hunk resolves to main's side. The merged tree is main plus exactly this PR's own commit (adff62e): - internal/cbm/extract_defs.c, extract_unified.c, helpers.c, helpers.h: the init() hunks apply to main unchanged. - tests/test_extraction.c, tests/test_pipeline.c: main added newer Go tests at the anchors this PR used, so the two new tests and their RUN_TEST lines sit next to them. The test bodies are byte-identical. Signed-off-by: Martin Vogel <[email protected]>
DeusData#1910 gives every Go init() its own QN, pkg.init#<file>:<N>, on the def side and in compute_func_qn. The Go LSP computes its enclosing-function QN on its own and still produced the flat pkg.init. The pipeline joins LSP rows to textual calls by exact caller QN, so every LSP resolution inside an init body missed the join: typed calls fell back to the registry resolver and callable references (Register(fn)) were dropped. On kubernetes (8c078bc3), without this commit all 499 lsp_* CALLS from init bodies fell back (import_map, same_module, unique_name) and all 562 CALL_REFERENCE edges from them were lost; with it, the init edges match main's strategy mix. - helpers.c: cbm_go_init_qn is the one formula for all three sides (def extractor, compute_func_qn, Go LSP process_function). The per-file ordinal now walks the top-level declarations with a cursor: ts_node_named_child(root, i) is O(i) per call, so the indexed loop was quadratic on wide generated files. - go_lsp: GoLSPContext carries the file's rel_path. cbm_run_go_lsp, cbm_run_go_lsp_cross and cbm_run_go_lsp_cross_with_registry take it, so the per-file walk, the shared Tier-2 registry walk and the per-file fallback all name init callers like the def side. Tier 3 copies the per-file rows' caller QN, so it follows. - pass_lsp_cross: the dispatch passes rel to the Tier-2 walk and to the per-file fallback (cbm_pxc_run_one_with_php_bindings gains rel_path; the incremental route can run without shared registries). - compute_func_qn: a NULL base QN keeps the flat QN, as the receiver branch above it does. Tests: golsp_init_caller_qn_matches_def (per-file rows and the caller join), golsp_crossfile_init_caller_qn_per_file (both cross entry points), pipeline_go_init_lsp_edges_per_file and its _parallel twin (three inits in two files: lsp_ CALLS and a CALL_REFERENCE on the right init, no cross-wiring, nothing onto a decoy), and pipeline_go_init_dispatch_fallback_names_init_caller. Each fails with its piece reverted; the pipeline tests also fail with the compute_func_qn mirror disabled, which no test covered before. Signed-off-by: Martin Vogel <[email protected]>
|
Thank you, @ilyabrykau-orca, for this one and for holding the whole stack steady while #1913 was decided. Stack A is in (#1907 and #1913 landed as 5a4af9c), so instead of asking you for one more rebase we finished the last steps on your branch ourselves, as two commits on top of yours. Your commits are untouched. You got the heart of it right: one node per init(), an edit-stable per-file ordinal instead of a line number (your reasoning about delta-merge identity was exactly right), and the call-scope mirror in compute_func_qn. We checked it on kubernetes/kubernetes: the graph now holds all 1,024 init functions instead of 700, which matches the source exactly (the only other "func init()" lines there sit inside code-generator templates), and each of the 21 files in code-generator's validators package now shows its own init() calling RegisterTagValidator.
With both commits, the init edges on kubernetes match main's resolution mix exactly, and the 104 calls that main had merged into one shared init node per package now each start at their own init. This rides the index-format change from #1913, so users who upgrade rebuild once and get both. Please pull before pushing to this branch again, so these two commits stay on it. Thank you again for measuring the loss instead of describing it, and for the ordinal change, which is what made this safe to land. |
Signed-off-by: Martin Vogel <[email protected]>
|
Thank you for your patience, @ilyabrykau-orca. CI went green on this branch, but this evening's merges on main touched files this pull request also changes, so that green no longer covered today's main. We merged current main into the branch once more (one merge commit, a3ff99d; your changes are untouched) and will merge once the fresh checks are green. Please pull before pushing again. |
Signed-off-by: Martin Vogel <[email protected]>
|
Thank you for your patience, @ilyabrykau-orca. A pull request that merged into main a few minutes ago (#2290) changes internal/cbm/cbm.c, which this pull request also changes, so the CI run in progress here could no longer vouch for today's main. We merged current main into your branch once more (one merge commit, 19b57f3; it merged cleanly and your own change is untouched) and started a fresh run. We will merge once the required checks are green. Please pull before pushing again. |
test_pipeline.c: kept the main version and re-applied this branch diff; its three Go init RUN_TESTs precede the DeusData#2769 alias tests. test_go_lsp.c: the two DeusData#2769 alias tests pass NULL for the new rel_path parameter. Signed-off-by: Martin Vogel <[email protected]>
|
Thank you so much for this, @ilyabrykau-orca - giving every Go init() its own per-file QN is exactly the right fix, and your tests made it easy to verify once more. Main moved on after #2769 (the Go LSP now follows a type alias declared in another file of the same package) landed, so we merged current main into your branch (merge commit 7a9e6c6, signed off). Two things needed care:
On the merged tree the go_lsp, extraction, pipeline, lsp_resolution_probe, edge_types_probe, c_lsp and parallel suites all pass, including all seven of your new tests and the four new alias tests. A fresh CI run is on its way; we will merge once it is green. Please pull before pushing again so the merge commit is kept. |
Signed-off-by: Martin Vogel <[email protected]>
|
Thank you for your patience, @ilyabrykau-orca. Main has moved since the last CI run on this branch, and some of the new commits change internal/cbm/extract_defs.c, which this pull request also changes, so that run can no longer vouch for today's main. We merged current main into your branch (one merge commit, 7ee29d8; it merged cleanly and your own change is untouched) and started a fresh run. We will merge once the required checks are green. Please pull before pushing again. |
Signed-off-by: Martin Vogel <[email protected]>
|
Thank you for your patience, @ilyabrykau-orca. The macOS legs of the last CI run here were cancelled by GitHub itself: it is retiring its macOS 14 runners on November 2 and ran scheduled brownouts beforehand, so those jobs never got a runner. Nothing in your change caused it. Main now runs CI on macOS 15 (#2849), so we merged current main into your branch (one merge commit, 8c62cb3; it merged cleanly and your own change is untouched) to get a run on the new runners. Please pull before pushing again. |
|
Merged as 9faa3e5 - thank you, Ilya! Each Go init() now gets its own per-file qualified name, so a package with several init functions keeps every body and every registration edge instead of collapsing them onto one node. Thank you for the careful fix and the tests, and for your patience through all the main merges along the way. We will pick up your stacked follow-up #1936 next. |
Go gives `init` no binding in the package block: init functions cannot
be referred to from anywhere in a program, and a method or field named
init needs a selector. A bare `init` is therefore always a local name -
a closure (`init := func(ctx) {...}; init(ctx)`), a parameter, or the
`package init` clause of a directory named init.
Before DeusData#1915 same_module bound such a name to the flat package init.
With per-file init QNs it now falls through to the short-name fallback
and binds unrelated methods named init. On kubernetes that minted 24
CALLS (22 e2e storage DefineTests closures among them), 28 WRITES,
7 USAGE and one derived TESTS edge, all false.
cbm_go_suppress_bare_init_bind refuses, for a bare Go `init`, any
registry binding onto a Function (every Go Function named init is a
package init) and a weak short-name binding onto a Method or Field.
lsp_* answers and selector calls (`w.init()`) are untouched. Wired at
the CALLS, USAGE and READS/WRITES seams of both resolvers.
Kubernetes, full index, main vs branch: only those 60 edges removed,
nodes identical, no resolver edge added; the 14 LSP-typed selector
CALLS into init methods are unchanged.
Tests: pipeline_go_bare_init_binds_nothing (+ parallel twin) are RED
with the predicate disabled (closure CALLS, parameter CALLS, closure
WRITES, package-clause USAGE onto Watcher.init) and GREEN with it; the
selector control keeps Watcher.Start/Boot -> Watcher.init.
go_bare_init_never_binds_init_namesake pins the predicate.
Refs DeusData#1915
Signed-off-by: Martin Vogel <[email protected]>
What does this PR do?
Fixes #1910. Stacked on #1913 (which stacks on #1907) — same extraction area and shared test scaffolding; review only the last commit.
Go allows any number of
init()functions per package — even several in one file — and all of them run at start-up. On the flat QN (proj.pkg.init) they all collided and the graph upsert kept one node per package, silently dropping the rest (measured in #1910: 21 decoder-registrationinits in one package → 1 node; 29 in the repo → 7 nodes).The change, following the #495 Rust cfg-twin pattern (and the #1266 GoogleTest name-derivation precedent):
internal/cbm/extract_defs.c— suffix the QN with the file basename and line:proj.pkg.init#a.go:L5. Callinginitexplicitly is illegal in Go, so nothing ever joins on the plain QN — no resolution path is affected.internal/cbm/extract_unified.c(compute_func_qn) — mirror the exact formula, so init-body calls keep their source attribution instead of degrading to thecalls_find_sourceFile-node fallback.Tests (reproduce-first):
pipeline_go_multi_init_nodes_survive— two files, one package, oneinit()each: RED without the def-side change (ic == 1, expected 2), GREEN with it.extract_go_multiple_init_disambiguated— twoinits in ONE file get distinct suffixed QNs (the line component disambiguates within a file).Full
scripts/test.shleg green (ASan+UBSan, "All tests passed");git clang-format --diffclean on changed lines.Checklist
git commit -s) — required, CI rejectsunsigned commits (DCO, see CONTRIBUTING.md)
scripts/test.sh— full leg, ASan+UBSan, "All tests passed")git clang-format --diffclean on changed lines; clang-tidy/cppcheck via CI)