Skip to content

fix(swift): preserve declaration identity and compatible call edges - #2436

Merged
DeusData merged 89 commits into
DeusData:mainfrom
DavidHLP:feat/issue-2061-swift-identity
Nov 10, 2026
Merged

DeusData merged 89 commits into
DeusData:mainfrom
DavidHLP:feat/issue-2061-swift-identity

Conversation

@DavidHLP

@DavidHLP DavidHLP commented Sep 30, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Refs #2061. Give Swift overloads declaration-qualified identities, preserve bare node names, and use the uncapped selector inside receiver-typed candidates. Keep static/class, async, and explicit return markers. Qualified types retain their last segment, and the associated-type spelling in where T.Element is retained pending maintainer follow-up.

  • Match extensions to owners in visible SwiftPM targets using the registry's type-path owner map. Preserve both declaration and extension markers when they share a node, and include extension initializers in typed lookup.
  • Honor literal SwiftPM target paths and nested package boundaries. Changes to Package.swift invalidate source-only repair and trigger a full reindex.
  • Continue call chains when all compatible candidates agree on the return type. Preserve declaration ownership through conditional compilation, wrapped trailing closures, lexical closure alias shapes, and serial argument keywords.
  • Keep the index at format 5 with a single migration entry. The branch includes main through e0c7a1da, including fix(extraction): attribute Swift lifecycle calls #2395.
  • Cover receiver collisions, two same-named types in separate targets, cross-file extensions, manifest changes, lexical aliases, and serial, parallel, and incremental paths in the regression suites.

Validation

AI assistance: This update was prepared with an AI agent acting on behalf of DavidHLP.

  • Unmodified PR production build at 1a6b2923 passed the default production regression guards. On Alamofire bda9ed57, the chain tails at Tests/SessionTests.swift:1632 and Tests/DataStreamTests.swift:582 each resolved to the expected method. On swift-nio eabcc14f, the NIOFS and _NIOFileSystem cross-target CALLS check returned zero.
  • In the Alamofire checkout, changing the target path from Source to Source/Core triggered incremental.force_full. The rebuilt and fresh graphs of that same modified checkout matched on node and edge rows, including properties. Earlier general incremental comparisons recorded SIMILAR_TO and decorator_tags differences separately.
  • Local combined validation tree used 1a6b2923 plus the remaining test-environment and baseline fixes from draft fix: isolate validation and handle empty initialization state #2809, retaining main's mimalloc 3.5.3. The default scripts/test.sh passed with GCC, ASan and UBSan, and UBSAN_OPTIONS=halt_on_error=1. It completed 192 suites with 10,363 passed, zero failed, and 10 skips. The incremental suite ran all 164 tests using FastAPI 0.99.1 at dd4e78ca, with no skips. The fix: isolate validation and handle empty initialization state #2809 changes remain separate from this PR.
  • GitHub CI at 1a6b2923 passed Linux, macOS, Windows CLANG64, TSan, lint, CodeQL, DCO, shard completeness, and ci-ok. PR workflow · DCO

DeusData and others added 8 commits September 25, 2026 18:49
…Data#2061)

Overloads that share a base QN (Java/C#/C++/Kotlin/Swift/Scala/ObjC) merge
into one graph node today, and that node collects the CALLS of every
overload. The agreed cure is one signature-qualified callable identity for
all overloading languages:

    callable_qn := base_qn [<tparams>] (params) [cvref]

This change lands the plumbing with every language still at
CBM_CALLABLE_ID_NONE, so the graph is byte-identical; each language is
switched on later in its own change (with its index-format bump).

- internal/cbm/callable_sig.{c,h}: the per-language mode side table
  (NONE|TYPED|LABELED_TYPED|LABELED|ARITY; kept out of CBMLangSpec like
  cbm_string_dispatch_suffixes, since ~160 positional rows would otherwise
  all need the member under -Wmissing-field-initializers), the shared
  builder cbm_callable_sig[_mode] for the def and the call-scope QN, and
  the inverse cbm_qn_callable_base_len[_named]. Suffix contract: no '.',
  '::' or '->'; whitespace-normalized; comments/annotations/attributes
  dropped; qualified types keep their last segment; '->' spelled '=>';
  variadic '[]' (Java) / '~'; C++ top-level cv dropped, arrays decay,
  cvref kept; <= 200 bytes, longer ones keep whole leading entries and end
  in '#<fnv1a64 of the full suffix>)'. The builder is iterative (cursor
  walk, bounded declarator loop) and self-checks against the inverse.
- The inverse is anchored to the node's bare name wherever the name is
  known: real QNs already end in ')' without being callables (a Java field
  whose name is its declarator `X = f()`, make `$(VAR)` targets), and only
  the anchor makes the inverse exact on them.
- Leaf splitters strip a suffix first (no-op today): registry by-name key
  (indexed by the bare name for a suffixed callable), cbm_lsp_bare_segment,
  cbm_pipeline_qn_class_method_tail, pxc_last_component, pxc_qn_leaf,
  go_index_visit_node, collect_prod_defs, sg_qn_prefix_len.
- New unused fields: CBMDefinition.qn_sig_off, CBMResolvedCall.callee_sig
  (relocated by result compaction), CBMRegisteredFunc.sig, CBMLSPDef.sig.
  The lsp_surface codec is unchanged (no format bump in this change).
- get_code_snippet and trace_path gain base-QN tiers
  (cbm_store_find_nodes_by_qn_base) that find nothing until suffixes exist.
- store: cbm_store_find_nodes_by_qn_suffix built its LIKE pattern in a
  512-byte buffer (a long suffix was truncated into a pattern that matched
  nothing) and did not escape '_'/'%' (`my_func` also matched `myXfunc`).
  The pattern is now heap-built, escaped, and matched with ESCAPE '\'.

Proof: full sorted node+edge+lsp_surface dumps are byte-identical before
and after on 7 corpora (this repo, elasticsearch libs, Exposed, dotnet
System.Linq, System.Collections, coreclr/gc, a C corpus).

Refs DeusData#2061

Signed-off-by: Martin Vogel <[email protected]>
Enable typed, labeled Swift callable identities on the shared DeusData#2061 plumbing. Resolve calls by compatible labels, defaulted parameters, and trailing closures; retain all compatible overload edges with candidate counts. Cover registry, pipeline, and signature behavior.

Refs DeusData#2061

Signed-off-by: DavidHLP <[email protected]>
Force the parallel call pass on a Swift fixture and assert both label-compatible type overloads retain CALLS edges with the ambiguity count.

Refs DeusData#2061

Signed-off-by: DavidHLP <[email protected]>
A Swift call that shares its bare name with project symbols but matches
none of their overloads was handled differently by the two resolvers:
the parallel path cleared the bare-name match and fell through as
unresolved (keeping each resolver's empty-resolution service
fallbacks), while the sequential path dropped the call outright. Skip
the bare-name registry bind on the sequential path too — it would bind
a wrong overload — and continue as unresolved, so count < 0 behaves
exactly like any other unresolved call on each resolver.

Pipeline regressions pin the global-fetch fallback (DeusData#856) on both
paths: the incompatible local overload gets no wrong CALLS edge, and
the URL call keeps its HTTP_CALLS edge. The route-registration fallback
differs between the paths on main (sequential additionally requires a
'/'-leading first argument); that pre-existing divergence is out of
scope here.

Refs DeusData#2061

Signed-off-by: DavidHLP <[email protected]>
@github-actions

Copy link
Copy Markdown

Thanks for opening this — it has been seen, and it is queued.

This note is automated, but it is not a brush-off: it exists so you know where your PR stands instead of having to guess from silence.

Current review status: working through a backlog. 0.9.1-rc.1 is out, so the release freeze that held reviews is over — but it left a large queue of open pull requests behind it, and we are reading through them oldest-first. The background is in discussion #1144.

What that means for this PR, concretely:

  • It will not be closed for inactivity. No stale bot touches pull requests here.
  • It may still sit a while before a human reads it. That is on us, not on you.
  • Older PRs are read first, so a recent one is not being skipped — it is behind a queue.

Things that will genuinely speed it up whenever review does happen:

  • Keep it rebased on main — the tree is moving quickly right now, and a conflicting branch cannot be reviewed as the diff you intended.
  • Get CI green, or say which failures you believe are pre-existing.
  • Keep the change to one claim. Bundled features and refactors get split before they get merged, which costs you a round trip.
  • Every commit needs a sign-off (git commit -s) — CI enforces DCO.

If this fixes a bug, a reproduction we can run is worth more than a description of the symptom.

Thanks for contributing, and sorry in advance for the wait.

@DeusData
DeusData force-pushed the feat/callable-identity-plumbing branch 2 times, most recently from 3441da4 to 5538355 Compare September 30, 2026 22:34
@DeusData DeusData added enhancement New feature or request parsing/quality Graph extraction bugs, false positives, missing edges priority/normal Standard review queue; useful PR with ordinary maintainer urgency. labels Oct 1, 2026
@DeusData

DeusData commented Oct 1, 2026

Copy link
Copy Markdown
Owner

Thank you for the detailed status update and for clearly distinguishing the actual PR head from unpublished work and unverified results. The changed-file scope includes callable identity, resolution and persisted metadata, so we need more time for a complete review together with the parent change. This remains queued for deeper review; this comment is not an approval or a request for another routine rebase. The queue is full, and we appreciate your patience and the care taken to document the remaining work.

Swift-scope checkpoint for DeusData#2061: capture bounded trailing-closure labels, match Swift overload candidates conservatively, and cover extraction, registry, compact/spill, serial, parallel, and incremental behavior.

Validation: ASan passed on the combined WIP source only; this A-only checkpoint has not been independently verified. GCC TSan, smoke, and memory gates remain failed/open. This commit is a scope checkpoint, not acceptance.

Signed-off-by: David HLP <[email protected]>
@DavidHLP
DavidHLP force-pushed the feat/issue-2061-swift-identity branch from 4dbfd27 to eea3143 Compare October 3, 2026 09:49
Keep extension owners within visible targets and respect literal custom
SwiftPM roots, nested package boundaries and manifest invalidation.
Continue compatible call chains through declared return types and bounded
initializer inference. Recover conditional declaration ownership and
wrapped trailing closures, and preserve serial argument keywords.

Add extraction, registry and serial/parallel/incremental regressions.

Signed-off-by: DavidHLP <[email protected]>
@DavidHLP

DavidHLP commented Nov 4, 2026

Copy link
Copy Markdown
Contributor Author

Thank you, @DeusData, for rerunning the census and documenting the exact call sites. The two filesystem targets in swift-nio and the generated Service example made the extension lookup problem easy to reproduce. The follow-up is ready for review at 1a6b2923.

  1. Declaration and extension markers. When a declaration and an extension share a graph node, the node now retains both swift_type_path and swift_extension. The registry can still identify the declaration as an owner.
  2. Target visibility. Extension lookup now considers the declaring target and targets that can see it. The same rule applies to initializers declared in extensions. The lookup also respects literal SwiftPM target paths and nested package boundaries. On swift-nio eabcc14f, I found zero cross-target CALLS between NIOFS and _NIOFileSystem, including calls from NIOFSTests into _NIOFileSystem.
  3. Registry lookup and regression coverage. The registry builds the type-path-to-owners map once and uses it during call resolution. The regression suite covers same-named types with separate extensions and callers, and the Swift complexity fixture now exercises extension lookup.
  4. Call chains. When all compatible overloads agree on a return type, resolution carries that type into the next call. On Alamofire bda9ed57, the two sites you flagged now each resolve to one expected chain tail. SessionTests.swift:1632 reaches DataRequest.responseDecodable, and DataStreamTests.swift:582 reaches DataStreamRequest.responseStream. The extraction changes also preserve ownership for declarations inside conditional compilation blocks, including the Session methods you flagged.

I also changed Alamofire’s target path from Source to Source/Core to check manifest invalidation. The existing index took the full rebuild path, and its node and edge rows, including properties, matched a fresh index of the same edited checkout. That comparison covers this specific manifest change.

The PR head passed the default production regression checks. The local validation tree, which included the separate fixes from draft #2809, completed all 192 suites under ASan and UBSan with 10,363 passed, 0 failed, and 10 skipped. All 164 FastAPI incremental tests ran and passed. GitHub CI for the PR head passed Linux, macOS, Windows CLANG64, TSan, CodeQL, DCO, and ci-ok. [CI run](https://github.com/DeusData/codebase-memory-mcp/actions/runs/41848187305)

I kept the real-repository rerun focused on the reported call sites and manifest invalidation. I haven’t rerun the complete census or the generated 60/120-directory benchmark. Thanks again for the detailed review.

@DeusData

DeusData commented Nov 4, 2026

Copy link
Copy Markdown
Owner

Thank you, @DavidHLP, for another fast and careful round, and for the clear point-by-point reply. 1a6b292 closes the three structural problems from this morning, and the conditional-compilation work goes further than we asked: it fixes the missing Session owner we mentioned as a pre-existing gap.

We re-ran the same census as this morning (Alamofire bda9ed57, swift-nio eabcc14f, helm 9b88efd6 as the non-Swift control, every run indexed from scratch), comparing 05c7ba7 to 1a6b292:

  • Same-named types in different targets: call edges between the swift-nio NIOFS and _NIOFileSystem copies went from 431 to 0 (main has 244), and call sites whose targets span two targets went from 351 to 0. Tests/NIOFSTests/FileOpenOptionsTests.swift:68 now resolves only into NIOFS.
  • The generated repository with k directories (struct Service, an extension in a second file, a typed caller in a third): 3 call edges per caller at both k=60 and k=120 (this morning 121 and 241), with no edge into another directory. That holds with and without a Package.swift.
  • Declaration and extension in one file: nodes marked only as an extension although their file declares the type went from 53 to 1 on Alamofire and from 322 to 8 on swift-nio; 52 and 317 nodes now carry both markers.
  • Chains: SessionTests.swift:1632 now reaches DataRequest.responseDecodable and DataStreamTests.swift:582 reaches DataStreamRequest.responseStream, each with one candidate.
  • Conditional compilation: lines the parser could not read went from 652 to 32 on Alamofire and from 6,188 to 2,907 on swift-nio, and many members that used to be loose functions are now methods of their types (Alamofire 459 functions / 1,961 methods, was 632 / 1,755).
  • Totals: resolved call sites 4,995 -> 5,362 on Alamofire and 23,290 -> 24,655 on swift-nio; single-candidate edges 3,661 -> 3,891 and 18,922 -> 20,138; the candidate maximum is unchanged (7 and 32).
  • Repeated runs are byte-identical, sequential and parallel indexing produce the same call edges, the indexing time matches main, and helm is still identical to main.
  • In a sample of the newly resolved call sites almost all were right, for example ServerBootstrap(...).serverChannelOption(...).childChannelOption(...), buffer.readString(length:) into ByteBuffer-aux.swift, and session.request(url, headers:).cURLDescription into Request.

One thing went the other way: 1,976 call sites on swift-nio (103 on Alamofire) that 05c7ba7 resolved now get no edge, and for 1,362 of them (61 on Alamofire) main resolves the same target. In a sample about half of these were right before, so this is a real loss. It also shows up as test links: TESTS on swift-nio drops from 52 to 30. A few shapes we saw:

  • Tests/NIOPosixTests/SyscallAbstractionLayerContext.swift:97, result.salWait(context:), where let result = try self.runSALOnEventLoop(...) and the declared return type of runSALOnEventLoop is its generic parameter Result. It looks like the chain takes Result as a concrete type, finds no salWait on it, and drops the call.
  • IntegrationTests/tests_04_performance/test_01_resources/test_future_assume_isolated_lots_of_callbacks.swift:26, p.futureResult.assumeIsolated().flatMap { ... }.map { ... } with let p = loop.makePromise(of: Int.self) and loop: EventLoop: the whole chain, including assumeIsolated() itself, has no edge any more. This is where the TESTS links go.
  • Sources/_NIOFileSystem/BufferedReader.swift:145, chunk.moveReaderIndex(forwardBy:): var chunk = try await self.readFromFile(self.capacity) now resolves to one readFromFile returning ByteBuffer, but the type does not reach chunk.
  • Tests/NIOPosixTests/VsockAddressTest.swift:93-96: let socket = try ServerSocket(...) now gets a correct initializer edge, but socket.close(), socket.bind(to:) and socket.getLocalVsockAddress() get none (close() is declared on ServerSocket itself; bind(to:) and getLocalVsockAddress() are on its base class BaseSocket and its extension in VsockAddress.swift).
  • Tests/NIOHTTP1Tests/HTTPDecoderTesting.swift:273 and Tests/NIOPosixTests/FileRegionTest.swift:251, buffer.writeString(...) / buffer.writeBytes(...) with var buffer = channel.allocator.buffer(capacity: ...): these methods are now correctly members of ByteBuffer, and with the receiver type unknown the call is dropped instead of being matched by its labels as before.
  • Alamofire Source/Core/Session.swift:1389, mutableState.read { ... } on a Protected<MutableState>.

Could you:

  1. when the receiver's type cannot be established, or the inferred type has no member with that name and labels, fall back to the label match as 05c7ba7 did instead of dropping the call, and treat a callee's generic parameter used as its return type (like Result above) as unknown rather than as a type name;
  2. carry the type through try await initializers, and include members of the base class and its extensions for a typed receiver (the BufferedReader and ServerSocket sites above);
  3. keep the re-parsed tree from the conditional-compilation retry only when it has fewer error regions than the first parse. Two files got worse: Alamofire Source/Core/Protected.swift (3 -> 14 unreadable lines, now 84-95 and 97, so the Protected initializer node is gone; most likely the #error(...) line from the #else branch stays in the class body after masking) and swift-nio Tests/NIOPosixTests/NonBlockingFileIOTest.swift (12 -> 15);
  4. add a regression test for two or three of these shapes (a generic return type, a try await initializer, a base-class extension member on a subclass receiver)?

Two smaller observations, not required for this PR: 288 Alamofire call sites such as session.request(.default) now get 6 targets (the three Endpoint overloads in TestHelpers.swift plus the three URLConvertible overloads in Session.swift), which is expected from labels alone now that those overloads have their owner. And struct X: Error conformances in swift-nio resolve to project types named Error (IOError.Error in parallel, the newly visible NonBlockingFileIO.Error in sequential). That is older resolver behavior, but it widens the sequential/parallel INHERITS difference from 35 to 86 entries.

We are also looking at a few further points and will follow up on those separately. Thank you again; the structural part is in very good shape now, and the census shows how much the conditional-compilation fix adds.

@DeusData

DeusData commented Nov 4, 2026

Copy link
Copy Markdown
Owner

Hi @DavidHLP, a quick follow-up to our comment from earlier today: to save you another round, we will make the four changes from that list ourselves on your branch (maintainer edits are enabled), and re-run the same census on the result. If you have already started on any of them, just say so here and we will build on your work instead of duplicating it. Otherwise, please hold off pushing to this branch until we post the result, or pull first, so neither of us loses commits.

Thank you again for all the work on this series - the structural part is in very good shape now.

…known

A variable whose type comes only from its initializer call
(`let p = loop.makePromise(of: Int.self)`) replaced the label match with
a typed lookup, and a failed lookup dropped the call. On swift-nio that
left 1,976 call sites without an edge that the label match had resolved.

The inferred type now only narrows the label match. When the inference
fails (overloads with different return types, an unknown callee or root,
a cycle) the receiver stays unknown, and when the inferred type has no
member with the call's name and labels the call keeps the label match.
Written types (annotations, parameters, constructor calls) and
non-project receivers behave as before. Inferring an initializer's own
type never uses a label match, so a guess cannot become a type.

A callee's own generic parameter used as its return type
(`func runSALOnEventLoop<Result>(...) -> Result`) names no type. It is
read from the declaration's generic clause in the qualified name and
leaves the bound variable unknown.

The four rows of pipeline_swift_capped_chain_returns_and_metatype_receivers
that expected no edge for an unknown receiver now expect exactly one
label-matched upload(_:) edge.

Signed-off-by: Martin Vogel <[email protected]>
`let socket = try ServerSocket(...)` and `let reader = try await
Reader(...)` left the variable untyped, so every member call on it went
to a label-only guess or was dropped. The initializer is now unwrapped
from `try` / `await` before its constructor type is read. Only a bare
type name counts there: `try Foo.make()` returns what make returns,
which the registry still infers from the call.

With the type in place the existing base-class and extension walk
applies: on swift-nio socket.close(), socket.bind(to:) and
socket.getLocalVsockAddress() in VsockAddressTest.swift resolve to
ServerSocket, BaseSocket and the BaseSocket extension in VsockAddress.swift.

Signed-off-by: Martin Vogel <[email protected]>
The conditional-compilation retry always kept the masked re-parse. When
the dropped branch holds an `#error(...)` line inside a type body the
re-parse reads less of the file: Alamofire Protected.swift went from 3
to 14 unreadable lines and lost its Protected initializer.

The re-parse is now kept only when its error regions cover no more lines
than those of the first parse. Counting regions instead would reject
re-parses that split one large region into a few small ones; on
swift-nio that gave back 1,312 of the recovered lines.

Signed-off-by: Martin Vogel <[email protected]>
pipeline_swift_generic_return_keeps_label_match_issue2061: a callee's
generic parameter used as its return type, colliding with a project type
of the same name, leaves the variable unknown and its call keeps the
label match.

pipeline_swift_try_initializer_types_receiver_issue2061: `try await
Reader(...)` and `try ServerSocket(...)` type the variable; a subclass
receiver reaches its base class and the base class's extension in
another file, not same-labelled decoy members.

swift_conditional_retry_keeps_the_better_parse_issue2061: an `#error`
line from a dropped `#else` branch in a class body keeps the first parse,
so the class and its initializer stay in the graph.

The pipeline tests run sequential and parallel. Each test fails with its
fix reverted.

Signed-off-by: Martin Vogel <[email protected]>
Brings the branch up to date with main (612eee2). Two textual
conflicts, both where main added its Java external-call guard (DeusData#2667)
next to this branch's Swift candidate guard:

- src/pipeline/pass_calls.c: the registry lookup still runs only when
  the Swift candidates allow it, and now passes java_external.
- src/pipeline/pass_parallel.c: the registry fallback condition keeps
  both !java_external and count == 0.

No other file conflicted.

Signed-off-by: Martin Vogel <[email protected]>
@DavidHLP

DavidHLP commented Nov 5, 2026

Copy link
Copy Markdown
Contributor Author

Thank you, @DeusData . I haven’t started working on these fixes yet, so there are no changes from my side to coordinate. I appreciate you handling the fixes and rerunning the census.

@DavidHLP

DavidHLP commented Nov 5, 2026

Copy link
Copy Markdown
Contributor Author

Hi @DeusData,

Thank you again for taking this on. Following my earlier reply, I ran some verification in an isolated Docker environment, comparing 05c7ba75 with 1a6b2923. I haven’t started implementing fixes or made any additional commits or pushes, and I’ll continue holding off while you work on the branch.

I reproduced the generic-return and try-wrapped constructor regressions, as well as the swift-nio TESTS drop from 52 to 30. One finding that may help: the diagnostic traces suggest that failed initializer inference is cached as an empty string, which then follows the incompatible-receiver path rather than falling back to label matching.

There was one difference in my run: Sources/_NIOFileSystem/BufferedReader.swift:145 resolved successfully, with chunk inferred as ByteBuffer. I haven’t established why our results differ on that particular site, but wanted to mention it in case it helps when comparing runs.

These observations are specific to 1a6b2923; I haven’t reproduced the complete per-call-site census or run the full C test matrix.

@DeusData

DeusData commented Nov 9, 2026

Copy link
Copy Markdown
Owner

Thank you again, @DavidHLP, for holding the branch for us. As promised, we made the four changes from our last comment ourselves. They are on feat/issue-2061-swift-identity as separate commits on top of your 1a6b292, followed by a merge of current main. Please pull before pushing again.

Thank you also for your verification notes from November 5. Your trace was spot on: a failed initializer inference ended up on the incompatible-receiver path instead of falling back to the label match, which is exactly the path the first commit below changes. And your BufferedReader.swift:145 result matches ours - see the note under commit 2.

  1. 8372793 fix(swift): keep the label match when an inferred receiver type is unknown
    A variable whose type comes only from its initializer call (for example let p = loop.makePromise(of: Int.self)) now narrows the label match instead of replacing it. When that inference fails (overloads that return different types, an unknown callee or root, a cycle), or when the inferred type has no member with the call's name and labels, the call keeps the label match it had in 05c7ba7. Declared types (annotations, parameters, constructor calls) and non-project types behave exactly as before. A callee's own generic parameter used as its return type (runSALOnEventLoop<Result>(...) -> Result) is now unknown rather than a type name; the generic clause is read from the declaration's qualified name, so for a signature long enough to be capped the first rule applies instead. Inferring an initializer's type never uses a label match, so a guess cannot turn into a type. The four rows of pipeline_swift_capped_chain_returns_and_metatype_receivers that expected no edge for an unknown receiver (mixedFactory, unknownFactory, unknownRoot, cyclicFactory) now expect exactly one label-matched upload(_:) edge.
  2. 8087c1c fix(swift): read a constructor type through try and await
    let socket = try ServerSocket(...) and let reader = try await Reader(...) now type the variable the way ServerSocket(...) already did. Under try / await only a bare type name counts, so try Foo.make() is still inferred from the call. The base-class and extension walk was already in place: once socket has its type, socket.close() resolves to ServerSocket.close(), socket.bind(to:) to BaseSocket.bind(to:) plus the BaseSocket extension's bind(to: VsockAddress), and getLocalVsockAddress() to the extension in VsockAddress.swift. The var chunk = try await self.readFromFile(...) case was in fact already typed on 1a6b292: BufferedReader.swift:145 resolves to ByteBuffer.moveReaderIndex(forwardBy:) and shares one edge with the call on line 126, which is why it did not show up as a separate call site in our census. Sorry for the false lead.
  3. 15906eb fix(swift): keep the first parse when masking directives reads less
    The masked re-parse is now kept only when its error regions cover no more lines than those of the first parse. We measured the "fewer error regions" rule we suggested first: it fixes Protected.swift but gives back 1,312 lines on swift-nio, because some re-parses split one large region into several small ones (AsyncChannelBootstrapTests.swift would go from 69 back to 1,377 unreadable lines). Counting lines keeps all of the swift-nio gain and fixes Protected.swift (14 -> 3 unreadable lines; the Protected initializer node is back). NonBlockingFileIOTest.swift stays at 15 rather than 12: its first parse has one large raw region that the declarations inside it recover later, and a comparison of the two trees cannot see that.
  4. ce6035d test(swift): pin generic returns, try initializers and the parse retry
    pipeline_swift_generic_return_keeps_label_match_issue2061 and pipeline_swift_try_initializer_types_receiver_issue2061 (each sequential and parallel), and swift_conditional_retry_keeps_the_better_parse_issue2061. Each of them fails with its fix reverted.
  5. aeeb271 Merge origin/main: two mechanical conflicts where main added its Java external-call check next to the Swift checks in pass_calls.c and pass_parallel.c; both conditions are kept.

Same census as before, 1a6b292 -> this branch (Alamofire bda9ed57, swift-nio eabcc14f, helm 9b88efd6, every run indexed from scratch):

  • Call sites that 05c7ba7 resolved and 1a6b292 left without an edge: 1,662 of 1,976 have one again on swift-nio (1,136 of the 1,362 where main agrees, to main's target), and 83 of 103 on Alamofire (50 of 61).
  • Nothing is lost: compared per caller and target, the branch keeps every call edge of 1a6b292 and adds 1,848 on swift-nio and 112 on Alamofire. TESTS on swift-nio goes from 30 to 64 (main has 52).
  • In a sample of the recovered call sites 20 of 24 were right. Of the few edges that neither 05c7ba7 nor 1a6b292 had (63 on swift-nio, 24 on Alamofire), 17 of 24 sampled were right.
  • Unchanged: no call edges between the NIOFS and _NIOFileSystem copies, 3 call edges per caller in the generated k=60 and k=120 repository, the two chain sites, and the candidate maximum (7 and 32).
  • Unreadable lines on Alamofire 32 -> 21 (swift-nio unchanged at 2,907); nodes marked only as an extension although their file declares the type 1 -> 0 on Alamofire.
  • Totals: resolved call sites 5,362 -> 5,469 on Alamofire and 24,655 -> 26,317 on swift-nio; single-candidate edges 3,891 -> 3,988 and 20,138 -> 21,713.
  • Repeated runs are byte-identical, sequential and parallel indexing produce the same call edges, and helm is still identical to main.

One example from our comment still has no edge: Alamofire Source/Core/Session.swift:1389. Protected's members end up as free functions in both parses (the #if inside its class body), so mutableState, typed Protected by its constructor, has no read. That is a parser limit rather than resolution, and we left it out of this round.

We are also still looking at a few further points and will follow up on those separately. Thank you again for this series and for your patience with our review rounds.

cppcheck 2.20 (the CI lint) reports variableScope at
src/pipeline/registry.c:4176: swift_candidates_for_file declared `graph`
at function scope, but only the named-receiver branch reads it. Declare
it inside that branch. No behaviour change.

Signed-off-by: Martin Vogel <[email protected]>
A receiver typed only by its constructor call (`let x = Foo()`, also
through `try` / `await`) now narrows the label match the way a receiver
with an inferred initializer type already does: a member the type has
still wins, and when the type has no member with the call's name and
labels, the call keeps the label match instead of getting no edge. One
such site is Alamofire's Session.swift:1389, where `mutableState` is a
`Protected(...)` whose members the parser leaves as free functions.

This applies only to a type the project declares. A constructor of a
type the project does not know (DispatchSemaphore(), Task {...}) or only
extends (Data()) still gets no edge: on swift-nio and Alamofire a label
match for those receivers mostly lands on unrelated project methods,
for example `semaphore.wait()` on EventLoopFuture.wait. A name the
extractor reads as a constructor but that resolves to a project function
(`Foo.make()`) keeps the label match like an inferred call result.

Annotations and parameters still decide alone, and inferring an
initializer's type still never uses a label match. The extractor marks a
binding as `constructed` when its type comes from the initializer's
constructor call rather than from an annotation.

On swift-nio eabcc14f and Alamofire bda9ed57 this adds 947 and 166 call
edges and removes none. pipeline_swift_constructor_receiver_keeps_label_match_issue2061
(sequential and parallel) pins the fallback, a member that still wins,
and the annotated, parameter, unknown-type and extended-type cases.

Signed-off-by: Martin Vogel <[email protected]>
Brings the branch up to date with main (e2a93e4): DeusData#2205 (daemon UI
setting on manual start), DeusData#2905 (Retrofit client HTTP_CALLS) and two
GitHub Actions version bumps. No conflicts.

Signed-off-by: Martin Vogel <[email protected]>
The previous commit kept the label match for a constructor-typed receiver
only when the project declares the constructed type. After the census the
maintainers decided to recover every constructor-typed receiver like one
with an inferred initializer type: when the type has no member with the
call's name and labels, the call keeps the label match, whether or not
the project declares the type (NSLock(), DispatchSemaphore(), Data()).

For constructors of types the project does not declare, that label match
is often an unrelated project method (a semaphore's wait() lands on
EventLoopFuture.wait on swift-nio). It is the same label match a receiver
of unknown type gets; tightening it for receiver chains that pass through
a property is a separate follow-up.

Annotations and parameters still decide alone, and inferring an
initializer's type still never uses a label match. The NSLock and Data
rows of pipeline_swift_constructor_receiver_keeps_label_match_issue2061
now expect the label-matched edge.

Signed-off-by: Martin Vogel <[email protected]>
@DeusData

DeusData commented Nov 9, 2026

Copy link
Copy Markdown
Owner

Thank you again, @DavidHLP, for holding the branch for us while we finished this round. We added three small commits on top of aeeb271 and merged current main. Please pull before pushing again.

  1. bbd727f style(swift): narrow the scope of the graph pointer in the call lookup
    The lint job failed on aeeb271 because of our own earlier commit, not yours: cppcheck 2.20 reported graph in swift_candidates_for_file (src/pipeline/registry.c) as declared in a wider scope than needed. The declaration now sits in the one branch that reads it. No behaviour change.
  2. 102c564 fix(swift): keep the label match for a constructor-typed receiver
    This follows the measurement from our last round. A receiver typed only by its constructor call (let x = Foo(), also with try / await) now behaves like a receiver with an inferred initializer type: a member the type has still wins, and when the type has no member with the call's name and labels, the call keeps the label match instead of getting no edge. That brings back Alamofire's Session.swift:1389 (mutableState.read, now Protected.read). Annotations and parameters decide alone as before, and inferring an initializer's type still never uses a label match. The new test pipeline_swift_constructor_receiver_keeps_label_match_issue2061 (sequential and parallel) fails without the change.
  3. 7ca9ea4 Merge origin/main (fix(daemon): preserve disabled UI on manual start #2205, fix(extract): emit HTTP_CALLS for Retrofit interface methods (#2723) #2905 and two Actions version bumps), no conflicts.
  4. fa5286c fix(swift): let every constructor-typed receiver keep the label match
    Commit 2 first applied the fallback only to types the project declares. After looking at the census we decided to treat every constructor type the same way, including types the project does not declare (NSLock(), DispatchSemaphore(), Data()). The NSLock and Data rows of the new test now expect the label-matched edge.

Same census as before, aeeb271 -> this branch (Alamofire bda9ed57, swift-nio eabcc14f, helm 9b88efd6, every run indexed from scratch):

  • Call sites that 05c7ba7 resolved and 1a6b292 left without an edge: swift-nio 1,662 -> 1,779 of 1,976 (1,136 -> 1,207 of the 1,362 where main agrees, to main's target); Alamofire 83 -> 95 of 103 (50 -> 56 of 61).
  • Compared per caller and target, nothing is lost: +1,251 call edges on swift-nio and +209 on Alamofire.
  • Resolved call sites 5,469 -> 5,637 (Alamofire) and 26,317 -> 27,464 (swift-nio); single-candidate edges 3,988 -> 4,117 and 21,713 -> 22,776.
  • Unchanged: no call edges between the NIOFS and _NIOFileSystem copies, 3 call edges per caller in the generated k=60 and k=120 repository (a constructor-typed variant stays at a constant 5 per caller), TESTS on swift-nio (64), the two Alamofire sites from our first comment, the candidate maximum (7 and 32), and helm.
  • Repeated runs are byte-identical, and sequential and parallel indexing produce the same call edges.

The added edges come from the same label match that main uses, so they share its limits. For constructors of types the project does not declare, the label match is often an unrelated project method (on swift-nio a semaphore's wait() lands on EventLoopFuture.wait). The most common miss on swift-nio is channel.pipeline.syncOperations.addHandler(...), which lands on the async ChannelPipeline.addHandler rather than the SynchronousOperations one; main has the same edge at most of those sites. We will tighten the label match for receiver chains that pass through a property in a separate follow-up after this PR lands.

We are also still looking at a few further points and will follow up on those separately. Thank you again for this series and for your patience with our review rounds.

@DavidHLP

Copy link
Copy Markdown
Contributor Author

Hi @DeusData, thank you for the latest fixes. I pulled your branch updates before pushing. I added one follow-up commit, cdb7a293, to address the Ubuntu CI failure. Your constructor/inferred-receiver fallback changes remain in place.

Navigation CI failure and fix

The failed Ubuntu job stopped at pipeline_arg_url_skips_client_navigation: a navigation call had an HTTP_CALLS edge where the test expected none.

I reproduced this in isolated Linux Docker runs by changing only the repository directory name:

  • The unchanged navigation fixture under a neutral project name produced no navigation HTTP edges.
  • Names containing Dio or cpr produced six false navigation HTTP edges, with 218 nodes / 238 edges—the same totals as the failing CI fixture.
  • The old 1a6b2923 binary also reproduced those six false edges.

The service classifier was treating the project-identity prefix of a resolved QN as library evidence. The exact random directory name was not retained in the CI artifact, so I cannot identify its particular suffix.

The fix excludes the exact project prefix when classifying graph-qualified symbols, in both sequential and parallel resolution. Raw external library identities retain their existing treatment. The existing navigation test keeps all its assertions and now deliberately includes Dio in its temporary directory name, making this case deterministic.

Validation of the follow-up code: Ubuntu 24.04 x86_64 Docker, GCC 13.3, ASan/UBSan, isolated HOME/Git configuration, network disabled:

  • 20 focused tests passed with UBSAN_OPTIONS=halt_on_error=1.
  • Complete pipeline and infrascan suites passed 498 tests with the default sanitizer settings. The existing Perl UBSan diagnostics remain, so that run is not sanitizer-clean; the Perl failure was separately reproduced on both the older and updated branch.
  • Eight fresh indexing controls passed: neutral/Dio/cpr project names in sequential and parallel modes, plus a project named exactly axios in both modes. Navigation HTTP edges were absent; axios, fetch and explicit HTTP import aliases still produced their expected edges.
  • CI-mode lint passed without changing thresholds or suppressions.

The new GitHub workflow is still running as I post this; DCO has passed.

Separate Swift finding for your review

I also found an explicit-type boundary case while independently verifying the previous head. This valid Swift example produces a false edge from the outer semaphore.wait() to Decoy.wait():

import Foundation

class Decoy { func wait() {} }
func identity<T>(_ value: T) -> T { value }

func declaredShadow(_ flag: Bool) {
    let semaphore: DispatchSemaphore = DispatchSemaphore(value: 0)
    if flag {
        let semaphore = identity(DispatchSemaphore(value: 0))
        semaphore.signal()
    }
    semaphore.wait()
}

With the same fixture, 1a6b2923 has no such edge. Its direct child 83727930 and the later f25f8b3d produce it in both sequential and parallel indexing, with strategy=swift_labels, candidates=1, and confidence=0.9. This reproduces before the subsequent main merges.

The trace points to swift_binding_type: a failed inference for the inner same-name binding makes it return NULL even after seeing the outer declared type; the caller then takes the label fallback. I have kept this finding separate from the navigation CI repair and have not changed the Swift binding resolver.

I have not rerun the full Alamofire/swift-nio/helm census on cdb7a293, so the earlier census numbers should not be treated as validation of this new head. Please let me know if you would prefer the CI repair split out of this PR.

@DeusData
DeusData merged commit 0d835c5 into DeusData:main Nov 10, 2026
26 checks passed
@DeusData

Copy link
Copy Markdown
Owner

Merged - thank you so much, @DavidHLP, for this whole series, for holding the branch while we worked on it, and for tracking down the last CI failure so carefully.

Your diagnosis of the Ubuntu failure was exactly right, and it was not specific to this branch: we reproduced it on main as well. The navigation fixture from pipeline_arg_url_skips_client_navigation, indexed from a directory whose name contains Dio, cpr or axios, got six false navigation HTTP_CALLS edges on main, while a neutral name got none, so the random temp directory name decided whether that test passed. With cdb7a29 every one of those names produces the same graph as a neutral name, and the intended HTTP calls keep their edges. We also checked a real project checked out in a directory named axios (axios/axios itself): before your change the directory name alone added 154 HTTP_CALLS edges and 126 Route nodes, after it the graph matches the same checkout under a neutral name. It is a small, self-contained commit that fixed this branch's one red job, so we kept it in this PR.

Thank you also for the Swift shadowing example. We confirmed it, and it comes from our own commit 8372793, not from yours: when an inner same-name binding cannot be inferred, the lookup falls back to the label match although the outer binding has a declared type. Treating that case as a conflict would also drop correct edges on Alamofire's var urlRequest = try urlRequest.asURLRequest() pattern, so the proper fix is to pick the binding that is actually in scope at the call. We will take that on ourselves in a follow-up, together with tightening the label match for receiver chains that pass through a property. Nothing is needed from you for either.

Where this series lands: declarations keep their identity across files, extensions stay within their SwiftPM target, conditional compilation is read instead of dropped, and on swift-nio and Alamofire the typed call graph is both larger and more precise than before, with repeated runs byte-identical. We are also still looking at a few further points and will follow up on those separately.

Thank you again for your persistence and care throughout this series. Merged in 0d835c5.

@DeusData

Copy link
Copy Markdown
Owner

@DavidHLP, a short follow-up to our merge note: the Swift shadowing case you found is now fixed on main by #2947 (merge 50079ed). A receiver now reads the binding that is actually in scope at the call, so your semaphore example no longer gets the false Decoy.wait() edge, while Alamofire's var urlRequest = try urlRequest.asURLRequest() pattern keeps its edges. Thank you again for the precise example. The property-chain part needs typed property chains first, so that one comes later as its own change.

DeusData added a commit to AmirF194/codebase-memory-mcp that referenced this pull request Nov 10, 2026
A receiver's type came from every same-name binding of the enclosing
function, wherever in the function it was declared. When an inner
binding's type could not be inferred, the receiver counted as unknown
even where only an outer, declared binding was in scope, and the call
took the label match. In

    let semaphore: DispatchSemaphore = DispatchSemaphore(value: 0)
    if flag {
        let semaphore = identity(DispatchSemaphore(value: 0))
        semaphore.signal()
    }
    semaphore.wait()

the outer wait() got an edge to an unrelated project wait() (DeusData#2436).

Each Swift binding now records where it is in scope: from the end of
its declaration to the end of its block, where a parameter's block is
its function. Only the bindings in scope at the call count, and of
those the innermost block's, so a local shadows a same-name local of an
enclosing block or a parameter. `var request = try request.make()`
reads the parameter on its right-hand side and the new variable after
it. Bindings of the same block that disagree still select nothing, and
fields and module-level bindings are unchanged.

pipeline_swift_in_scope_binding_issue2436 (sequential and parallel)
covers the example, the shadowed-parameter idiom of Alamofire's
encoders and two sibling blocks; it fails without the change.

Signed-off-by: Martin Vogel <[email protected]>
DeusData added a commit to AmirF194/codebase-memory-mcp that referenced this pull request Nov 10, 2026
Brings the branch up to date with main 6d34ca1. Two resolutions:

- tests/test_parallel.c (textual conflict): main added
  parallel_csharp_nested_type_and_global_namespace_binding and
  parallel_csharp_parse_damaged_namespace_is_not_global at the same spot
  as this branch's parallel_csharp_keyword_receiver_no_name_guess_issue2121.
  Resolved as main's file plus this branch's own hunk (all three tests
  and their RUN_TEST lines kept); the zero-context patch-id of the
  branch's change to the file is unchanged.
- src/pipeline/registry.c (semantic conflict; merged cleanly as text but
  would not compile): main's cbm_registry_swift_candidates (DeusData#2436) calls
  receiver_chain_admits with two arguments, and this branch gives it a
  third, the calling file's language. The Swift caller now passes
  CBM_LANG_SWIFT. Behaviour is unchanged: the type-keyword branch applies
  to CBM_LANG_CSHARP only.

Signed-off-by: Martin Vogel <[email protected]>
@DavidHLP

Copy link
Copy Markdown
Contributor Author

@DeusData Thanks for following up and for fixing the shadowing case in #2947. I appreciate the scope-aware approach and the care taken to preserve Alamofire’s shadowed-parameter pattern. The separate follow-up for typed property chains makes sense as well. Thank you for the detailed explanation and validation throughout this series!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request parsing/quality Graph extraction bugs, false positives, missing edges priority/normal Standard review queue; useful PR with ordinary maintainer urgency.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants