Skip to content

fix(extract): give each Go init() its own QN - #1915

Merged
DeusData merged 10 commits into
DeusData:mainfrom
ilyabrykau-orca:fix/go-init-qn
Oct 31, 2026
Merged

DeusData merged 10 commits into
DeusData:mainfrom
ilyabrykau-orca:fix/go-init-qn

Conversation

@ilyabrykau-orca

Copy link
Copy Markdown
Contributor

What does this PR do?

Fixes #1910. Stacked on #1913 (which stacks on #1907) — same extraction area and shared test scaffolding; review only the last commit.

Go allows any number of init() functions per package — even several in one file — and all of them run at start-up. On the flat QN (proj.pkg.init) they all collided and the graph upsert kept one node per package, silently dropping the rest (measured in #1910: 21 decoder-registration inits in one package → 1 node; 29 in the repo → 7 nodes).

The change, following the #495 Rust cfg-twin pattern (and the #1266 GoogleTest name-derivation precedent):

  1. internal/cbm/extract_defs.c — suffix the QN with the file basename and line: proj.pkg.init#a.go:L5. Calling init explicitly is illegal in Go, so nothing ever joins on the plain QN — no resolution path is affected.
  2. internal/cbm/extract_unified.c (compute_func_qn) — mirror the exact formula, so init-body calls keep their source attribution instead of degrading to the calls_find_source File-node fallback.

Tests (reproduce-first):

  • pipeline_go_multi_init_nodes_survive — two files, one package, one init() each: RED without the def-side change (ic == 1, expected 2), GREEN with it.
  • extract_go_multiple_init_disambiguated — two inits in ONE file get distinct suffixed QNs (the line component disambiguates within a file).

Full scripts/test.sh leg green (ASan+UBSan, "All tests passed"); git clang-format --diff clean on changed lines.

Checklist

  • Every commit is signed off (git commit -s) — required, CI rejects
    unsigned commits (DCO, see CONTRIBUTING.md)
  • Tests pass locally (scripts/test.sh — full leg, ASan+UBSan, "All tests passed")
  • Lint passes (git clang-format --diff clean on changed lines; clang-tidy/cppcheck via CI)
  • New behavior is covered by a test (reproduce-first for bug fixes)

@github-actions

Copy link
Copy Markdown

Thanks for opening this — it has been seen, and it is queued.

This note is automated, but it is not a brush-off: it exists so you know where your PR stands instead of having to guess from silence.

Current review status: working through a backlog. 0.9.1-rc.1 is out, so the release freeze that held reviews is over — but it left a large queue of open pull requests behind it, and we are reading through them oldest-first. The background is in discussion #1144.

What that means for this PR, concretely:

  • It will not be closed for inactivity. No stale bot touches pull requests here.
  • It may still sit a while before a human reads it. That is on us, not on you.
  • Older PRs are read first, so a recent one is not being skipped — it is behind a queue.

Things that will genuinely speed it up whenever review does happen:

  • Keep it rebased on main — the tree is moving quickly right now, and a conflicting branch cannot be reviewed as the diff you intended.
  • Get CI green, or say which failures you believe are pre-existing.
  • Keep the change to one claim. Bundled features and refactors get split before they get merged, which costs you a round trip.
  • Every commit needs a sign-off (git commit -s) — CI enforces DCO.

If this fixes a bug, a reproduction we can run is worth more than a description of the symptom.

Thanks for contributing, and sorry in advance for the wait.

@ilyabrykau-orca

Copy link
Copy Markdown
Contributor Author

Rebased onto current main (stacked on the rebased #1913). No semantic changes; full test leg green. Post-fix init-node recovery (29/29) is in the #1906 census comment.

@DeusData

DeusData commented Sep 1, 2026

Copy link
Copy Markdown
Owner

Reviewed the last commit. The diagnosis is right, the safety argument holds, and I have one substantive concern about the suffix itself.

The diagnosis is confirmed by the schema. src/store/store.c:257 declares UNIQUE(project, qualified_name) and the node upsert at :2364 is ON CONFLICT(project, qualified_name) DO UPDATE SET. So a flat proj.pkg.init really does mean every init in a package is the same row, and the last writer wins — which is exactly your 21 → 1.

Your safety claim holds, and I checked it both ways. Nothing in internal/cbm/, the LSP resolvers, or src/pipeline/ matches on the literal "init", so no resolver joins on the plain QN. And since calling init explicitly is illegal in Go, no call site produces init as a callee either. Both directions are clear.

The precedents you cite are real. extract_defs.c:2079 is the Rust cfg twin using "%s#%s", and :3544–:3613 is the GoogleTest derivation, whose comment names the same failure — "causing qualified-name collisions and node loss (#1266)". The shape is established house practice for this problem class.

The concern: the suffix is not stable, and both precedents are

This is the part I would like you to weigh, because it is the one property those two precedents share that yours does not.

The Rust cfg twin derives its suffix from the cfg(...) predicate text, and says why in the code:

Build a compact predicate suffix from the cfg(...) text, dropping whitespace and quotes so the QN stays readable and stable.

GoogleTest derives its from the macro arguments — TEST(Suite, Case) → TEST_Suite_Case. Both are semantic content that does not move when the file is edited.

proj.pkg.init#a.go:L5 embeds a line number. Add an import above that init and its qualified name changes — and since the QN is the node's identity under that UNIQUE constraint, an incremental reindex sees the old node vanish and a new one appear, for an edit that did not touch the function. That is node churn on every unrelated edit above any init, and it lands on the delta-merge path rather than a full rebuild.

The file basename alone will not do it — your own extract_go_multiple_init_disambiguated test requires two inits in one file to differ, and only the line separates them today. But a per-file ordinal would: proj.pkg.init#a.go:1, #a.go:2. That still satisfies that test, and it only churns if you reorder the inits in a file rather than every time you add a line above one.

I am raising this rather than ruling on it — you may have considered ordinals and rejected them for a reason I cannot see from here. If so, say why and I will take it.

Also

The whole stack is currently DIRTY — #1907, #1913 and this one all have conflicts. main was broken for a few hours today (duplicate struct members merged into type_registry.h) and was repaired an hour ago by #1993, so now is a good moment to rebase all three.

Thanks for measuring the loss rather than describing it — "21 decoder-registration inits in one package → 1 node; 29 in the repo → 7 nodes" is what makes this obviously worth fixing.

@DeusData DeusData added bug Something isn't working parsing/quality Graph extraction bugs, false positives, missing edges priority/high Needs near-term maintainer attention; high-impact bug, regression, safety issue, or release blocker. labels Sep 1, 2026
A Go selector call x.foo() whose receiver the Go LSP cannot type falls
through to the generic registry resolver, which binds it by bare short
name to an arbitrary same-named project symbol. Stdlib calls are the
worst case: f.Close() on an *os.File gets a CALLS edge to whatever
project Close wins candidate ranking (measured on a real Go repo:
confidence 0.11, 15 candidates; suffix_match + unique_name were 36% of
all CALLS edges, and one 14-line stdlib-only function got 3 out of 3
false outbound edges).

Extend the TS/JS receiver-aware guard (DeusData#592/DeusData#606) to Go:

- extract_calls.c: flag Go call_expression with a selector_expression
  callee as is_method, mirroring the TS/JS member_expression flag.
- registry.c: add cbm_go_suppress_weak_method_match. Unlike the TS/JS
  drop-list, field_type_hint is KEPT (Go struct fields carry declared
  types, so the hint is receiver-aware — lrp_go_s8_field_type_hint),
  and unique_name is dropped only when its confidence carries the
  import-unreachability penalty (the stdlib-hijack shape); an
  unpenalized lone candidate inside the caller's import closure never
  enters the field-type-hint upgrade and must survive.
- pass_calls.c / pass_parallel.c: feed the Go gate next to the TS/JS
  one; the drop still defers to the emit path so service/route/HTTP
  edges stay main-identical.

Reproduce-first: pipeline_go_receiver_suppresses_weak_method_edge is
RED without the extractor flag (the f.Close -> project Close edge
exists) and GREEN with it; typed same-package calls, bare local calls
and import-qualified cross-package calls still resolve. The old
extraction contract test used Go as the flag-exempt language — Python
takes that role, and extract_go_selector_call_flags_is_method pins the
new behavior.

Signed-off-by: Ilya Brykau <[email protected]>
@ilyabrykau-orca

Copy link
Copy Markdown
Contributor Author

Rebased onto current main as part of the 1907→1913→1915→1936 stack; only test insertion-anchor drift in tests/test_pipeline.c, re-anchored. Full scripts/test.sh green on the stack head.

A Go method's QN was the flat package form (proj.pkg.method) — the
receiver was ignored, so every same-name method in a package collided
on one QN and the graph upsert kept exactly one node. Measured on a
real Go repo: 20 Process/Name methods across 15 files kept 2 nodes;
9 different Task() methods fused into one chimera node carrying all
nine bodies' call edges; 19 structs pointed DEFINES_METHOD at a single
shared method node; a _test.go mock Close outranked the production
Close in the dedupe tie-break. The upsert's own comment calls
kind-disambiguated QNs 'the real cure'.

Qualify the QN with the receiver type (proj.pkg.Recv.method), the same
shape as Go interface members and the C++ out-of-line method path
right below it in extract_func_def:

- extract_defs.c: def.qualified_name = parent_class + name whenever
  the receiver type resolves; go_receiver_type_name becomes the shared
  cbm_go_receiver_type_name (exported via helpers.h) so both sides of
  the contract use one formula.
- extract_unified.c (compute_func_qn): mirror branch for
  method_declaration, so method-body calls keep exact source
  attribution instead of degrading to File-node fallback
  (calls_find_source).
- Consumers already agree: pxc_build_lsp_def passes the def QN and
  parent_class (receiver_type) verbatim into the Go LSP registries,
  and check_go_class_implements explicitly supports class-qualified
  method QNs (its path (b)).

Side effect: resolve_same_module's exact module.name hash no longer
matches concrete methods, which kills the conf-0.9 false edges where
an interface-typed call bound to an unrelated same-package method.

Fixes DeusData#1909

Signed-off-by: Ilya Brykau <[email protected]>
Go allows any number of init() functions per package — even several in
one file — and all of them run at start-up. On the flat QN
(proj.pkg.init) they all collided and the graph upsert kept ONE node
per package, silently dropping the rest: measured on a real Go repo, an
event-source package registering each decoder in its own file's init()
collapsed 21 init functions into one node (29 in source, 7 in the
graph), erasing the whole registration pattern.

Disambiguate with the DeusData#495 cfg-twin pattern: fold the file basename
and line into the QN (proj.pkg.init#a.go:L5). Calling init explicitly
is illegal in Go, so nothing ever joins on the plain QN; the
call-scope side (compute_func_qn) mirrors the exact formula so
init-body calls keep their source attribution instead of degrading to
the calls_find_source File fallback.

Reproduce-first: pipeline_go_multi_init_nodes_survive is RED without
the def-side change (node count 1, expected 2) and GREEN with it;
extract_go_multiple_init_disambiguated pins two same-file inits to
distinct suffixed QNs.

Fixes DeusData#1910

Signed-off-by: Ilya Brykau <[email protected]>
@ilyabrykau-orca

Copy link
Copy Markdown
Contributor Author

You are right, and the ordinal is strictly better — implemented, not argued with.

I had not weighed the incremental path when I chose the line number: on a full rebuild the line suffix is merely ugly, but under delta-merge the QN is node identity, so an insertion above any init would delete-and-recreate the node on an edit that never touched it. Both precedents being edit-stable is not a coincidence; it is the property that makes a QN suffix safe.

Now: pkg.init#file.go:N, where N is the 1-based source-order ordinal among the file's top-level init function_declarations. One formula, computed from the tree by a shared cbm_go_init_ordinal (helpers.c) on both the definition side and the call-scope side (compute_func_qn), so the two cannot drift. Reordering the inits within a file still churns — that is inherent to any content-free discriminator — but nothing else does.

The test now pins the property you asked for, not just distinctness: exact QNs t.init#x.go:1 / t.init#x.go:2, then the same file re-extracted with an import and a declaration inserted above both inits, asserting both QNs are byte-identical to before.

Branch force-pushed with the stack rebase; full scripts/test.sh green on the stack head.

@DeusData

DeusData commented Sep 3, 2026

Copy link
Copy Markdown
Owner

Status pointer, so this thread is not left hanging: the stack is now waiting on a direction call about the Go qualified-name shape surfaced by the #1913 census — details on #1913. Please hold the rebase of this branch along with the rest of the stack until that lands; I will update here.

@DeusData

Copy link
Copy Markdown
Owner

The update I promised on 3 September, late and with apologies for that: the direction call on #1913 landed on 15 September (the receiver-qualified shape stays, stack order 1907 → 1913 as you already rebased it), so nothing is waiting on a decision any more. #1907 and #1913 are next in line for a branch update and a fresh CI run as soon as the hosted-runner queue drains from tonight's merges; once #1913 is in I will ping here, and a rebase of this branch onto it is then the only step left before review. Thank you for holding the stack steady through all of this, and for the ordinal fix — that reasoning about delta-merge identity was exactly right.

Brings DeusData#1915 onto main 83109dd, after Stack A (DeusData#1907 and DeusData#1913) landed
on main as 5a4af9c.

This branch carried pre-rebase copies of the DeusData#1907 and DeusData#1913 commits
(651ae3c, dbba757). Main holds their final versions (50ee111,
da76194) plus the maintainer follow-ups made before Stack A merged, so
every DeusData#1907/DeusData#1913 hunk resolves to main's side. The merged tree is main
plus exactly this PR's own commit (adff62e):

- internal/cbm/extract_defs.c, extract_unified.c, helpers.c, helpers.h:
  the init() hunks apply to main unchanged.
- tests/test_extraction.c, tests/test_pipeline.c: main added newer Go
  tests at the anchors this PR used, so the two new tests and their
  RUN_TEST lines sit next to them. The test bodies are byte-identical.

Signed-off-by: Martin Vogel <[email protected]>
DeusData#1910 gives every Go init() its own QN, pkg.init#<file>:<N>, on the def
side and in compute_func_qn. The Go LSP computes its enclosing-function
QN on its own and still produced the flat pkg.init. The pipeline joins
LSP rows to textual calls by exact caller QN, so every LSP resolution
inside an init body missed the join: typed calls fell back to the
registry resolver and callable references (Register(fn)) were dropped.
On kubernetes (8c078bc3), without this commit all 499 lsp_* CALLS from
init bodies fell back (import_map, same_module, unique_name) and all
562 CALL_REFERENCE edges from them were lost; with it, the init edges
match main's strategy mix.

- helpers.c: cbm_go_init_qn is the one formula for all three sides
  (def extractor, compute_func_qn, Go LSP process_function). The
  per-file ordinal now walks the top-level declarations with a cursor:
  ts_node_named_child(root, i) is O(i) per call, so the indexed loop was
  quadratic on wide generated files.
- go_lsp: GoLSPContext carries the file's rel_path. cbm_run_go_lsp,
  cbm_run_go_lsp_cross and cbm_run_go_lsp_cross_with_registry take it,
  so the per-file walk, the shared Tier-2 registry walk and the per-file
  fallback all name init callers like the def side. Tier 3 copies the
  per-file rows' caller QN, so it follows.
- pass_lsp_cross: the dispatch passes rel to the Tier-2 walk and to the
  per-file fallback (cbm_pxc_run_one_with_php_bindings gains rel_path;
  the incremental route can run without shared registries).
- compute_func_qn: a NULL base QN keeps the flat QN, as the receiver
  branch above it does.

Tests: golsp_init_caller_qn_matches_def (per-file rows and the caller
join), golsp_crossfile_init_caller_qn_per_file (both cross entry
points), pipeline_go_init_lsp_edges_per_file and its _parallel twin
(three inits in two files: lsp_ CALLS and a CALL_REFERENCE on the right
init, no cross-wiring, nothing onto a decoy), and
pipeline_go_init_dispatch_fallback_names_init_caller. Each fails with
its piece reverted; the pipeline tests also fail with the
compute_func_qn mirror disabled, which no test covered before.

Signed-off-by: Martin Vogel <[email protected]>
@DeusData

Copy link
Copy Markdown
Owner

Thank you, @ilyabrykau-orca, for this one and for holding the whole stack steady while #1913 was decided. Stack A is in (#1907 and #1913 landed as 5a4af9c), so instead of asking you for one more rebase we finished the last steps on your branch ourselves, as two commits on top of yours. Your commits are untouched.

You got the heart of it right: one node per init(), an edit-stable per-file ordinal instead of a line number (your reasoning about delta-merge identity was exactly right), and the call-scope mirror in compute_func_qn. We checked it on kubernetes/kubernetes: the graph now holds all 1,024 init functions instead of 700, which matches the source exactly (the only other "func init()" lines there sit inside code-generator templates), and each of the 21 files in code-generator's validators package now shows its own init() calling RegisterTagValidator.

  1. 776f6f5 - Merge origin/main into fix/go-init-qn. Your branch still carried the pre-rebase copies of the fix(pipeline): suppress weak short-name matches for Go selector calls #1907 and fix(extract): receiver-qualify Go method QNs #1913 commits. Every one of those hunks resolves to main's final version, so the merged tree is main plus exactly your commit. Your production hunks apply unchanged, and your two tests sit next to the newer Go tests main added at the same spots, with identical bodies.

  2. 6e784c3 - fix(lsp): name Go init() callers per file in the Go LSP. This is the third side of the same contract you handled for methods in fix(extract): receiver-qualify Go method QNs #1913. The Go LSP builds its own enclosing-function QN in process_function (internal/cbm/lsp/go_lsp.c) and still produced the flat pkg.init, while the pipeline joins LSP results to textual calls by exact caller QN. So every LSP resolution inside an init body missed the join. On kubernetes, all 499 LSP-resolved calls from init bodies fell back to the name-based resolver, and all 562 function-value references from them (scheme registrations, for example) lost their precise edge. The commit:

  • moves the formula into one helper, cbm_go_init_qn, used by the def extractor, compute_func_qn and the Go LSP, so the three cannot drift;
  • passes the file path into the Go LSP (GoLSPContext.rel_path, plus a rel_path parameter on cbm_run_go_lsp and both cbm_run_go_lsp_cross entry points, wired through the cross-file dispatch including its per-file fallback);
  • computes the ordinal with a tree cursor, because ts_node_named_child(root, i) costs O(i) per call and made the loop quadratic on very wide generated files;
  • keeps the flat QN when the base QN is NULL (allocation failure only), like the receiver branch next to it.
    New tests cover the per-file and both cross-file LSP paths, the pipeline in its sequential and parallel venues (three inits in two files, where each init's calls and its function-value reference must land on its own node), and the fallback dispatch. Each one fails with its piece reverted, and the pipeline tests now also cover your compute_func_qn mirror, which no test exercised before.

With both commits, the init edges on kubernetes match main's resolution mix exactly, and the 104 calls that main had merged into one shared init node per package now each start at their own init. This rides the index-format change from #1913, so users who upgrade rebuild once and get both.

Please pull before pushing to this branch again, so these two commits stay on it. Thank you again for measuring the loss instead of describing it, and for the ordinal change, which is what made this safe to land.

@DeusData

Copy link
Copy Markdown
Owner

Thank you for your patience, @ilyabrykau-orca. CI went green on this branch, but this evening's merges on main touched files this pull request also changes, so that green no longer covered today's main. We merged current main into the branch once more (one merge commit, a3ff99d; your changes are untouched) and will merge once the fresh checks are green. Please pull before pushing again.

@DeusData

Copy link
Copy Markdown
Owner

Thank you for your patience, @ilyabrykau-orca. A pull request that merged into main a few minutes ago (#2290) changes internal/cbm/cbm.c, which this pull request also changes, so the CI run in progress here could no longer vouch for today's main. We merged current main into your branch once more (one merge commit, 19b57f3; it merged cleanly and your own change is untouched) and started a fresh run. We will merge once the required checks are green. Please pull before pushing again.

test_pipeline.c: kept the main version and re-applied this branch diff; its three Go init RUN_TESTs precede the DeusData#2769 alias tests. test_go_lsp.c: the two DeusData#2769 alias tests pass NULL for the new rel_path parameter.

Signed-off-by: Martin Vogel <[email protected]>
@DeusData

Copy link
Copy Markdown
Owner

Thank you so much for this, @ilyabrykau-orca - giving every Go init() its own per-file QN is exactly the right fix, and your tests made it easy to verify once more.

Main moved on after #2769 (the Go LSP now follows a type alias declared in another file of the same package) landed, so we merged current main into your branch (merge commit 7a9e6c6, signed off). Two things needed care:

  1. The only textual conflict was the RUN_TEST list in tests/test_pipeline.c, where both changes added Go tests at the same spot. We kept main's file, re-applied your additions unchanged, and registered your three tests right before the new alias tests.
  2. Your branch adds a rel_path parameter to cbm_run_go_lsp_cross and cbm_run_go_lsp_cross_with_registry, and the two new alias tests from fix(go-lsp): follow a type alias declared in another file of the package #2769 still called them the old way, so the merged tree did not compile. Those two calls now pass NULL for rel_path, the same way you updated the existing calls. No production code changed in the merge.

On the merged tree the go_lsp, extraction, pipeline, lsp_resolution_probe, edge_types_probe, c_lsp and parallel suites all pass, including all seven of your new tests and the four new alias tests. A fresh CI run is on its way; we will merge once it is green. Please pull before pushing again so the merge commit is kept.

@DeusData

Copy link
Copy Markdown
Owner

Thank you for your patience, @ilyabrykau-orca. Main has moved since the last CI run on this branch, and some of the new commits change internal/cbm/extract_defs.c, which this pull request also changes, so that run can no longer vouch for today's main. We merged current main into your branch (one merge commit, 7ee29d8; it merged cleanly and your own change is untouched) and started a fresh run. We will merge once the required checks are green. Please pull before pushing again.

@DeusData

Copy link
Copy Markdown
Owner

Thank you for your patience, @ilyabrykau-orca. The macOS legs of the last CI run here were cancelled by GitHub itself: it is retiring its macOS 14 runners on November 2 and ran scheduled brownouts beforehand, so those jobs never got a runner. Nothing in your change caused it. Main now runs CI on macOS 15 (#2849), so we merged current main into your branch (one merge commit, 8c62cb3; it merged cleanly and your own change is untouched) to get a run on the new runners. Please pull before pushing again.

@DeusData
DeusData merged commit 9faa3e5 into DeusData:main Oct 31, 2026
26 checks passed
@DeusData

Copy link
Copy Markdown
Owner

Merged as 9faa3e5 - thank you, Ilya! Each Go init() now gets its own per-file qualified name, so a package with several init functions keeps every body and every registration edge instead of collapsing them onto one node. Thank you for the careful fix and the tests, and for your patience through all the main merges along the way. We will pick up your stacked follow-up #1936 next.

DeusData added a commit to ilyabrykau-orca/codebase-memory-mcp that referenced this pull request Oct 31, 2026
Go gives `init` no binding in the package block: init functions cannot
be referred to from anywhere in a program, and a method or field named
init needs a selector. A bare `init` is therefore always a local name -
a closure (`init := func(ctx) {...}; init(ctx)`), a parameter, or the
`package init` clause of a directory named init.

Before DeusData#1915 same_module bound such a name to the flat package init.
With per-file init QNs it now falls through to the short-name fallback
and binds unrelated methods named init. On kubernetes that minted 24
CALLS (22 e2e storage DefineTests closures among them), 28 WRITES,
7 USAGE and one derived TESTS edge, all false.

cbm_go_suppress_bare_init_bind refuses, for a bare Go `init`, any
registry binding onto a Function (every Go Function named init is a
package init) and a weak short-name binding onto a Method or Field.
lsp_* answers and selector calls (`w.init()`) are untouched. Wired at
the CALLS, USAGE and READS/WRITES seams of both resolvers.

Kubernetes, full index, main vs branch: only those 60 edges removed,
nodes identical, no resolver edge added; the 14 LSP-typed selector
CALLS into init methods are unchanged.

Tests: pipeline_go_bare_init_binds_nothing (+ parallel twin) are RED
with the predicate disabled (closure CALLS, parameter CALLS, closure
WRITES, package-clause USAGE onto Watcher.init) and GREEN with it; the
selector control keeps Watcher.Start/Boot -> Watcher.init.
go_bare_init_never_binds_init_namesake pins the predicate.

Refs DeusData#1915

Signed-off-by: Martin Vogel <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working parsing/quality Graph extraction bugs, false positives, missing edges priority/high Needs near-term maintainer attention; high-impact bug, regression, safety issue, or release blocker.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Go: multiple init() per package collapse onto one QN — bodies and registration edges silently dropped

2 participants