Skip to content

fix(pipeline): DATA_FLOWS link every caller and handler of a route - #2689

Merged
DeusData merged 3 commits into
mainfrom
fix/data-flows-every-caller
Oct 18, 2026
Merged

DeusData merged 3 commits into
mainfrom
fix/data-flows-every-caller

Conversation

@DeusData

Copy link
Copy Markdown
Owner

fix(pipeline): DATA_FLOWS link every caller and handler of a route

create_data_flows copied each route's callers (HTTP_CALLS + ASYNC_CALLS) into a 64-slot buffer and its
INFRA_MAPS handlers into a 32-slot one. Each buffer kept the first entries in edge-insertion order, which on the
parallel resolver is worker interleaving. That had two consequences:

  1. Every caller past the 64th, and every infra handler past the 32nd, lost its DATA_FLOWS edge.
  2. Which flows existed changed between two indexes of the same tree with the same binary: a work cap decided
    graph content.

Change

The pass now walks the graph buffer's own edge arrays and creates the full caller x handler product for each
route.

  • A (caller, handler) pair that is reached twice, through two routes or both directly and via infra, keeps
    cbm_gbuf_insert_edge's order-free property winner.
  • The arrays stay valid while flows are added: inserting a DATA_FLOWS edge never appends to a HANDLES,
    INFRA_MAPS, HTTP_CALLS or ASYNC_CALLS array, and only edge deletion rebuilds the indexes.

Tests (infrascan suite)

test asserts
infrascan_data_flows_link_every_caller_of_a_route 70 callers of one route all get their flow
infrascan_data_flows_reach_every_infra_handler one caller reaches all 40 handlers behind an INFRA_MAPS endpoint
infrascan_data_flows_do_not_depend_on_caller_insertion_order the flow set is identical when the 70 callers' HTTP_CALLS are inserted in reverse order

Without the fix (main plus these tests): all three fail on their behaviour assertions, at
test_infrascan.c:423, :442 and :474; infrascan is 7 passed, 3 failed.

With the fix: pipeline + infrascan pass completely, 374 passed.

Corpus check

The six repositories whose baseline runs were not identical from run to run, each indexed twice, on main
(354ccec) and with this change:

repository main: DATA_FLOWS r1 vs r2 this change: r1 vs r2 DATA_FLOWS main -> this change
thingsboard/thingsboard 40 of 392 differ (10.2%) identical 392 -> 478 (+86, none removed)
pola-rs/polars, spree/spree, firebase/flutterfire identical identical unchanged
apache/spark, swiftlang/swift-package-manager identical identical unchanged

No other edge type moves beyond run-to-run noise. spark (IMPORTS) and swift-package-manager (CALLS,
IMPORTS, USAGE) differ between two runs on main and with this change alike. That nondeterminism is unrelated
to DATA_FLOWS.

Sequential and parallel resolvers. On thingsboard the sequential resolver has 0 DATA_FLOWS against 478.
The cause is upstream: on main only the parallel resolver runs the arg-URL heuristic (2 vs 2,101
HTTP_CALLS). This pass is the same code on both resolvers, and both are deterministic.

Skipped local legs

  • Linux/Windows and the remaining suites: left to PR CI, per the project's PR-CI-first rule. The change is
    platform-neutral C in one pass.
  • cppcheck: 2.20, the CI version, run on the changed file: clean. clang-format: clean.

create_data_flows copied a route's callers (HTTP_CALLS + ASYNC_CALLS)
into a 64-slot buffer and its INFRA_MAPS handlers into a 32-slot one,
each keeping the first entries in edge-insertion order. In a parallel
index that order is worker interleaving, so which DATA_FLOWS existed
changed between two indexes of the same tree with the same binary
(thingsboard, 2026-10-17), and every caller past the 64th lost its flow
outright.

The pass now walks the graph buffer's own edge arrays and creates the
full caller x handler product per route. A (caller, handler) pair that
is reached twice keeps cbm_gbuf_insert_edge's order-free property
winner, so the flow set and its properties no longer depend on the
order edges were inserted in.

Signed-off-by: Martin Vogel <[email protected]>
…route

A route with 70 callers must link all 70 to its handler, a route whose
INFRA_MAPS endpoint has 40 handlers must link its caller to all 40, and
the flow set must not change when the callers' HTTP_CALLS edges are
inserted in reverse order. Before the fix the first 64 callers and the
first 32 infra handlers in insertion order were kept, so all three fail.

Signed-off-by: Martin Vogel <[email protected]>
@DeusData
DeusData merged commit ea5c1f7 into main Oct 18, 2026
29 checks passed
DeusData added a commit to woodsonl/codebase-memory-mcp that referenced this pull request Oct 19, 2026
… R2)

Design decision 2026-10-18 (R1 + R2): a handler defined in a test file
pairs only with callers in that same file, and a caller in a test file
pairs only with handlers in that same file. A test's request to the
application, a production caller matched to a test's mock server, and
the private mock servers of two tests pairing across files are not
data flows of the application. DeusData#2689 links every caller of a route to
every handler of it, so these pairs were made on main.

try_create_data_flow now refuses a pair whose caller or handler is in a
test file (cbm_is_test_file, the per-language rule extraction uses)
when the two files differ. HTTP_CALLS and HANDLES are unchanged; only
the pairing is scoped. The pass log counts the refused pairs
(skipped_test_scope).

Signed-off-by: Martin Vogel <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant