Skip to content

feat(infra): env and service joins, kustomize variants, Terraform / Terragrunt, Ansible - #2643

Merged
DeusData merged 19 commits into
mainfrom
feat/infra-followup
Oct 14, 2026
Merged

DeusData merged 19 commits into
mainfrom
feat/infra-followup

Conversation

@DeusData

Copy link
Copy Markdown
Owner

Infra mono-repos, part 2: env and service joins, kustomize variants, Terraform / Terragrunt, Ansible

Follow-up to #2597 (Phases 0-4), planned for v0.12.0. Sixteen packages, each built and measured on its own and committed separately, plus one merge of main. Every package was measured on 17 pinned open-source infrastructure repositories against an independent reading (a separate script that never calls the code under test) before any test suite ran.

What the graph gains

Package What Measured
F1 A container's EnvVar reaches the code that reads it inside its own image (CONFIGURES {strategy: env_read}): the image's files are what its Dockerfile COPY/ADD puts in, minus .dockerignore 80/80
F2 CONNECTS_TO: a workload or compose service to the Service its environment addresses (*_ADDR, URL hosts, <svc>.<ns>.svc), same bundle and namespace only; values never stored 129 edges, all correct by hand
F3 curl / wget in scripts, Make and CI: HTTP_CALLS to the Route of every literal URL (method from flags, query and user info dropped) 92 edges, all correct; 81/82 truth
P5.1 Kustomize patches: OVERRIDE {via: patch} patch → patched object (path, inline, JSON6902, components); patch files IMPORTS from their kustomization 126/126, 0 extra
P5.2 Kustomize helmCharts: DEPLOYS {via: helmcharts} to the shared external Chart; values file IMPORTS 20/20
P5.3 Resource labels and labelSelector patch targets 130/130; labels equal on all 1,804 objects
P5.4 namePrefix / nameSuffix: a variant Resource with the deployed name (ecosystem: kustomize), OVERRIDE {via: namePrefix} → the declared object; kustomize's own skip list (CRD, Namespace, APIService) 56/56
P6.1 Terraform / OpenTofu references (var., local., module., data., resources) → USAGE {via: terraform, ref} to the declaring block; depends_on → DEPENDS_ON 7,771/7,771 file pairs, 13,397/13,397 block pairs
P6.2 Local module calls IMPORTS the module's Folder; module.x.out USAGE {ref: output} 349/349 imports, 644/644 output reads
P6.3 .tfvars → OVERRIDE {via: tfvars} to the variable block; a tfvars file with no declarations beside it links the nearest ancestor module, marked inferred 238/238 (124 own + 114 inferred)
P6.4 Terragrunt: include / read_terragrunt_config IMPORTS, dependency DEPENDS_ON, local terraform.source IMPORTS 30/30
P6.5 Terraform kubernetes_* resources with a literal name become Resources of the same identity scheme as manifest objects (ecosystem: terraform), DEPLOYS {via: terraform} from the block; manifests of the same bundle join them 21/21
FIX A .tf / .tofu / .tfvars / .hcl change takes the full rebuild (the repair recomputed changed files only; cross-file Terraform edges went stale) 18/18 incremental scenarios equal a full index
P7.1 Ansible composition: import_playbook, include_tasks / import_tasks, roles → IMPORTS {via: ansible} 211/211
P7.2 Ansible tasks and handlers are Functions (source: ansible, kind, module); notify → CALLS {via: notify} to the handlers of the task's role (nested roles kept apart). Ansible is recognised by structure (playbook, role tasks/handlers, included files), not by "top-level YAML list", which would mint ~3,000 false tasks from JSON patches and image lists 1,490/1,490 tasks, 276/276 notifies, 0 elsewhere

Every package's diff against the previous one adds only the edges and nodes listed. One exception, accepted: Ansible tasks join the similarity corpus, so 4 SEMANTICALLY_RELATED scores in kubespray's Python module moved up slightly and 1 pair was added.

For users

  • Index format 8 → 9: an index built before rebuilds once on upgrade. Inside v0.12.0 this is the same single rebuild the release already brings.
  • New Resource properties: version, labels, ecosystem (terraform, kustomize, besides compose).
  • Incremental indexing declines to a full rebuild where a cross-file edge would otherwise go stale: Terraform inputs (FIX) and Ansible content (P7.2).

How it was verified

  • Per package: an independent truth script and a gate against it on all 17 repositories; a scope diff against the previous package; three identical runs (nodes, edges, properties); incremental-equals-full scenarios; revert checks (each part of each rule removed alone; 155 in total, 154 turn their test red for the right reason, one, a defensive filter in F3 that no input reaches, is recorded as such); cppcheck 2.20 (CI's version), clang-format, memory-core and no-suppress lints; the LSP originality check clean on every commit's tree.
  • Full macOS suite on the last package's tree: 9,662 passed, 11 skipped. The merge of main (CLI, UI licences, README rewrite) was checked with the build and the cli suite, 361/361.
  • An independent precision sample of every new edge kind (≥ 50 edges per kind, read against the source) is running and will be posted here.

Not in this PR (follow-ups)

Kubernetes manifests applied by Ansible kube / k8s tasks (built, proving), Jinja variable references, config ↔ code keys, cross-repo links (images, charts, Terraform git modules, service DNS).

Signed-off-by lines are on every commit; merge with a merge commit, not squash (DCO).

The variables a container sets (K8s env, compose environment, Dockerfile ENV)
were container-scoped EnvVars of their own, joined to nothing: the readers of
a name point at the global EnvVar, which every program reading that name
shares, so a join through it was in the right program for 23 % of the pairs.

Each scoped EnvVar now CONFIGURES {strategy:env_read, dockerfile} the readers
of its name inside the files the container's image is built from:
- the image: a K8s container's literal image, built from exactly one
  Dockerfile (Image -DEPENDS_ON {via:build}->); a compose service's own build,
  else its image's; a Dockerfile's own ENV. A K8s image reference now names
  its container (USAGE {ref:image, container}), so a two-container pod joins
  each container to its own image.
- its files: the context sources COPY / ADD put into the final stage, the
  stages it is built FROM and the stages it copies from, resolved in the
  build context (the Dockerfile's directory or the repository root, the one
  the sources exist under), less the context's .dockerignore, never a test
  file. They are kept on the Dockerfile's Module (image_context, image_files,
  image_ignore).

pass_docker now reads COPY / ADD from the instruction's own text, as Docker
does: the grammar lost a JSON form opening with `/` and, after a word with a
trailing blank, took the following instructions for more sources.

Incremental: every run recomputes every join edge. A closure repair reads the
inputs of the files it leaves alone from the store and replaces the stored
edges wholesale; a .dockerignore (now a semantic-manifest control) changing,
or a deletion of a file an image copies, rebuilds in full.
CBM_SEMANTIC_INDEX_VERSION 8 -> 9.

Measured on 17 infrastructure repositories: 80 pairs, 80 correct by hand;
the graph matches an independent reading of the Dockerfiles 80/80.

Signed-off-by: Martin Vogel <[email protected]>
…nment

A workload's environment names the services it talks to
(PRODUCT_CATALOG_SERVICE_ADDR: productcatalogservice:3550, a URL, a
<svc>.<ns>.svc name); the graph had no edge for it, so the service topology
of an application was invisible.

A Kubernetes workload or a compose service now CONNECTS_TO the Service its
containers' environment addresses, with {via: env, key, container} -- never
the value:
- the address: the host of a URL or of host:port, a <svc>.<ns>.svc name in
  the value, or the whole value when the key names an address (*_ADDR,
  *_ADDRESS, *_HOST, *_URL, *_ENDPOINT, *_URI) and it is a bare DNS name; a
  templated value, localhost or an IP addresses nothing, and only service DNS
  forms count (svc, svc.ns, svc.ns.svc...), never another domain's host
  (cbm_env_address, shared by both sides);
- Kubernetes: a reference resolved like every other one -- a Service of the
  same bundle, in the named namespace or the object's own;
- compose: a service of the same file; a variable without a value takes the
  project's .env, whose values compose expands against the lines before them
  (CART_ADDR=cart:${CART_PORT}) -- the dotenv reader does that now too.

Consumers: trace_path's topology mode, the CLI's edge list and the README
(which also gains DEPLOYS and the env join of the previous commit).

Measured on 17 infrastructure repositories: 129 edges, every one correct by
hand; an independent reading of the values finds 133 pairs repo-wide, the 5
without an edge belong to another bundle or namespace.

Signed-off-by: Martin Vogel <[email protected]>
Shell scripts, Make recipes and CI steps download tools and call services
with curl and wget; the graph had no edge for any of them, so neither what
a repository fetches nor the scripts that call its own services were
visible.

Such a command now HTTP_CALLS the Route of each URL it states, with
{via: curl | wget, method, url_path, host, line}:
- the method from the flags: -X / --request / --method, a body (-d, -F,
  --data*, --json, --form*, wget --post-data / --post-file) POST, -T PUT,
  -I HEAD, -G back to GET;
- the URL from constant words (VER=1.2.3 ... v${VER}), unknown parts `{}`
  as in a Route path -- also in an unquoted word tree-sitter-bash splits
  around its expansions; a URL whose host is not literal, or the value of a
  proxy / referer option, is no request;
- never the user info, the query or the fragment: a token in a URL does not
  reach the graph;
- the Route: path-only (__route__POST__/v1/items) when the host is a
  Kubernetes Service of the repository in a service DNS form, where that
  service's handler meets it as a code client's request does; any other
  host -- a download, localhost, a compose service -- keeps it
  (__route__GET__https://dl.k8s.io/release/{}/bin/linux/amd64/kubectl,
  props {method, host, external}), and route_nodes makes no path-only twin
  of it.

Incremental indexing now drops a Route without a file that no edge touches
any more -- its last caller's file changed or went. A full index never
holds one; before, the Route of a deleted code client stayed behind too.

Measured on 17 infrastructure repositories: 92 edges, every one correct by
hand; an independent reading of the scripts and Makefiles finds 82
requests, 81 of them in the graph (the last is lost in a file
tree-sitter-bash cannot parse past `[ "$v" \< "1.33" ]`). No property of
any other node or edge changes.

Signed-off-by: Martin Vogel <[email protected]>
A kustomize overlay changes its base through patches; the graph read none of
the v5 `patches:` entries (path / patch / target mappings), so a patch file
showed up as a second copy of the object it patches, with no edge to it.

Each patch of a kustomization now OVERRIDEs, child -> base, the objects of
its build it changes, with {via: patch, kustomization, patch, line}:
- a patch file's documents: the object of the same group, kind and name
  (and namespace, when the document declares one);
- an inline patch (from the kustomization's File) or a JSON6902 operation
  file (from its own File): what its `target` selects -- kind, group,
  version, name, namespace; a label or annotation selector is not
  evaluated and names nothing;
- in the build of the kustomization stating it; a component's in the build
  of each kustomization including it.
A patch file is an IMPORTS (local_name: patches) of its kustomization and
never a member of its build: its documents select nothing, are selected by
nothing.

Resources now keep the version of their apiVersion (`version`), which the
identity leaves out and a target may name.

Each build indexes its own objects by name once: a lookup in the
repository-wide name index would visit every copy of a name for every
patch. The complexity fixture gains patch files (one of four documents), an
inline and a kind-only patch.

Measured on 17 infrastructure repositories: 126 edges, every one in an
independent reading of the 173 patches (122 resolvable in the
repositories); the 4 pairs without an edge use a label selector.

Signed-off-by: Martin Vogel <[email protected]>
A kustomization's `helmCharts` inflate Helm charts into its build -- the
fourth way a repository puts a chart on a cluster, beside a Flux
HelmRelease, a Terraform helm_release and a helm command; the graph read
none of them.

Each entry now DEPLOYS from the kustomization -- its Kustomization
Resource, else its File: the unit a Flux or Argo deploy of it reaches -- to
the shared external Chart of that name, with {via: helmcharts, external,
chart, url, version, release, namespace, values, line}. A chart name with
a path names nothing; the values file inside the repository is an IMPORTS
(local_name: valuesFile) of the kustomization.

Measured on 17 infrastructure repositories: 20 edges, every one in an
independent reading of the kustomizations (20 entries, cluster-api 10,
k8s.io 10); no other node or edge changes.

Signed-off-by: Martin Vogel <[email protected]>
A Kubernetes object's metadata.labels were not in the graph: neither a
search for an object by label nor a kustomize patch whose target selects
by label could find anything.

A Resource now keeps its labels as `labels: "k=v,k2=v2"` (sorted; a
templated label, or one a selector cannot name, is left out), and a
patch target's labelSelector meets the objects of its build carrying its
terms -- `k=v` / `k==v`, `k!=v`, `k`, `!k`; a set-based term names no key
and meets nothing. A target with only a selector selects among every
kind; an annotationSelector is not evaluated and names nothing.

Measured on 17 infrastructure repositories: 1,956 Resources gain labels
and nothing else; an independent reading of metadata.labels agrees on
every one of the 1,804 objects both read (152 more are Go-templated,
which only the graph reads, through its skeleton). The 4 label-selector
patch pairs now resolve: kustomize patches 130 of 130.

Signed-off-by: Martin Vogel <[email protected]>
…laring it

Terraform references were resolved by the generic name matcher: by their
last name segment, across the whole repository. `format()` CALLS a
variable named format of another module, `local.x` meets a resource named
local, `.id` an output id anywhere. Measured on the 7 Terraform
repositories of 17 infrastructure corpora: 12,437 edges, recall 0.63 and
precision 0.39 against an independent reading of the references, and not
one of them left the block holding its reference.

A reference -- var.X, local.X, module.X, data.T.N, T.N -- now USAGEs
{ref, via: terraform, line}, from the top-level block holding it, the
block declaring it in the same module: the .tf / .tofu files of its
directory (a locals entry: its locals block); an entry of the block's
depends_on list DEPENDS_ON the object instead. The generic matcher makes
no edge for a .tf / .tofu file any more; a name only mentioned in string
text is no reference, nor is a block's reference to itself. A changed
file re-resolves its whole directory, so an unchanged file's reference
meets the declaration a change adds and loses the one it removes.

A bare block label (`variable region {}`, valid HCL) now names its block
like a quoted one; such blocks used to share the keyword's name and QN.

Measured: 7,771 of 7,771 file-level and 13,397 of 13,397 block-level
reference pairs (edge type included) and nothing else; the 10 other
corpora are unchanged; deterministic over three runs; the incremental
re-index equals a full index in 7 scenarios (rename a variable, a module,
a locals entry; delete the variables file; add a file; add a declaration
an unchanged file references; quote a bare label); index time unchanged
(within 1 % on the four largest Terraform corpora).

Signed-off-by: Martin Vogel <[email protected]>
…eads its output

A module call with a local source -- `module "net" { source =
"./modules/net" }` -- linked nothing: 379 such calls on the 7 Terraform
repositories of 17 infrastructure corpora, and every `module.net.id`
reached at most the calling block, never the output it reads.

Such a call now IMPORTS {local_name, via: terraform, line} the Folder of
its source directory (the repository root: the Project), when that
directory lies inside the repository and holds .tf / .tofu files; a
registry, git or http source, or a path not starting with ./ or ../,
makes none. `module.net.id` -- also `module.net[0].id`, `module.net.0.id`
and `module.net[*].id` -- USAGEs {ref: output} the `output "id"` of that
directory, beside the USAGE of `module.net`.

Output reads cross directories, so a run touching any .tf / .tofu file
now resolves every module directory of the repository: an unchanged
caller keeps its reads when the callee file is re-parsed. A full index
takes as long as before; an incremental re-index touching a .tf file
pays for parsing them all (+0.24 s on a 2,000-file repository).

Measured: 349 of the 379 calls name a directory that exists in the
repository and IMPORT it, nothing else (the 30 others name paths a test
harness creates or that are absent); 644 of 644 output reads (holding
block to output) against an independent reading; the reference pairs
of the module resolver stay exact; only edges are added; deterministic
over three runs; the incremental re-index equals a full index in 6
scenarios (an output's value edited in the callee, its outputs file
deleted, a source moved, a widely called module edited, the root
module's caller edited).

Signed-off-by: Martin Vogel <[email protected]>
A .tfvars file's keys were Variable nodes connected to nothing: nothing
said which variable `region = "eu"` sets.

An assignment to a variable the module of the file's own directory
declares now OVERRIDEs {via: tfvars, line} that `variable` block, from
the file's Variable (child to base, as a values or patch overlay). A
.tfvars file elsewhere is applied by a -var-file the repository does not
state, and links nothing. A run touching a .tfvars file re-resolves like
one touching a .tf file.

Measured on 17 infrastructure repositories: 124 of 124 assignments of
the 19 files that set variables of their own directory, nothing else;
deterministic over three runs; the incremental re-index equals a full
index in 4 scenarios (a value edited, the variables file edited, a
.tfvars file deleted).

Signed-off-by: Martin Vogel <[email protected]>
A Terraform module's output read can live in one file (outputs.tf:
module.net.vpc_id) and the module's source in another (main.tf). When
only main.tf changed -- its source pointed elsewhere -- the incremental
repair recomputed that file's edges alone, so the read, held by two
unchanged files, stayed in the graph although a full reindex no longer
has it. Measured on kube-hetzner: 18 stale output reads after moving
one module's source.

Terraform, OpenTofu and Terragrunt inputs reach past their own file the
way Kubernetes bundles do: the files of a module directory resolve each
other's references, a module call decides what another file reads, a
.tfvars file sets the variables beside it. A change to a .tf, .tofu,
.tfvars or .hcl file now declines the repair to the full rebuild, as a
kustomization or manifest change already does.

The incremental re-index now equals a full index in all 18 Terraform
scenarios, the stale-read one included; a Terraform edit costs a full
rebuild (fabric: 1.64 -> 1.90 s). A pipeline test moves a module's
source and requires the read to go.

Signed-off-by: Martin Vogel <[email protected]>
A Terragrunt unit (terragrunt.hcl) composes itself from other files --
`include` blocks, read_terragrunt_config calls, dependencies on other
units -- and the graph held none of it: its blocks only met same-named
blocks elsewhere through the generic name matcher.

Every unit is now evaluated in its own context, the way Terragrunt
evaluates it (also the files it includes): an include IMPORTS
{local_name, via: terragrunt, line} the file its path names; a
read_terragrunt_config IMPORTS {via: terragrunt, read, line} the file it
names -- from the block holding the call or, for a call in an included
file, from the unit's include block that brings that file in, `read`
naming the file holding the call; a dependency DEPENDS_ON the other
unit's terragrunt.hcl; a local terraform source IMPORTS its Folder.
Paths may be string templates of find_in_parent_folders (searched from
the unit's parent directory upward), dirname and get_terragrunt_dir;
anything else is unknown and links nothing. A run holding any .hcl file
re-links every unit (a .hcl change takes the full rebuild since the
previous commit).

Measured on 17 infrastructure repositories: 30 of 30 composition pairs
of the 6 units of the Terragrunt repository against an independent
evaluator, nothing else; deterministic over three runs; the incremental
re-index equals a full index in 5 scenarios (a read file edited or
deleted, the included root file edited or reading another file, a unit
edited).

Signed-off-by: Martin Vogel <[email protected]>
Ansible's composition produced no structure: kubespray's 40 playbooks,
roles and task files, tied together by 217 import_playbook /
include_tasks / import_tasks / include_role / import_role / roles:
directives, were unconnected YAML files.

A playbook or task file -- a YAML file whose top level is a sequence;
manifests, values, CI and compose files are mappings -- now IMPORTS
{local_name: <directive>, via: ansible, line} what each directive names:
import_playbook a playbook from its directory; include_tasks /
import_tasks / include a task file from its directory, else from its
role's tasks/; roles: items and include_role / import_role a role's
Folder, roles/<name> at or above its directory (a role by "./" / "../"
path from the directory). The ansible.builtin. prefix is understood; a
templated target links nothing.

Measured on 17 infrastructure repositories: 211 of 211 composition pairs
of kubespray against an independent reading, nothing else in any
corpus; deterministic over three runs; the incremental re-index equals
a full index in 4 scenarios (an included file edited or deleted, an
include pointed elsewhere, a playbook edited).

Signed-off-by: Martin Vogel <[email protected]>
…rred

A .tfvars file in a directory without Terraform -- kubespray's
sample-inventory/cluster.tfvars, eks-blueprints' spokes/workspaces/*.tfvars --
is applied to a module further up with a -var-file the repository does
not state, and linked nothing.

When the file's own directory declares none of its names, it now sets
the variables of the nearest ancestor module that does: each assignment
to a declared variable OVERRIDEs {via: tfvars, inferred: true, line} the
`variable` block, from the file's Variable. A file whose own directory
declares one of its names keeps the exact edge, without `inferred`.

Measured on 17 infrastructure repositories: 238 of 238 assignments --
124 to their own directory's module, 114 inferred to an ancestor's --
against an independent reading, with the inferred flag; nothing else
changed; deterministic over three runs; the incremental re-index equals
a full index when an ancestor-linked file changes.

Signed-off-by: Martin Vogel <[email protected]>
A `kubernetes_*` resource in a .tf file -- eks-blueprints' network
policies and storage classes, k8s.io's metrics-server stack -- declares
a cluster object, but the graph only had the Terraform block, named by
its Terraform address: no node carried the object's kind and name.

A kubernetes_<type> resource (also _vN) whose metadata name is a literal
is now a Resource of the same identity scheme as a manifest's object:
kind from the resource type, or a kubernetes_manifest's own apiVersion
and kind; name and namespace from the object's own metadata, never from
spec; `ecosystem: terraform` on its properties. The .tf file DEFINES it
and the block DEPLOYS it {via: terraform, line}. The objects are minted
before the Kubernetes joins, so a manifest of the same bundle references
them as it does a manifest's object (an Ingress maps to the Service). A
computed name, and a type that acts on an object declared elsewhere
(kubernetes_annotations, kubernetes_labels, ...), make none. Labels are
not read, so a Service selector does not reach a Terraform workload.

Measured on 17 infrastructure repositories: 21 of 21 literal-named
objects against an independent reading, with group, kind, name and
namespace; nothing else changed (21 nodes, 42 edges added, no property
of an existing node); deterministic over three runs; the incremental
re-index equals a full index when an object is renamed, gains a
namespace, or its file is added or deleted.

Signed-off-by: Martin Vogel <[email protected]>
Cluster API deploys its manager as `capi-controller-manager` in
`capi-system`: config/default/kustomization.yaml puts `namePrefix: capi-`
and `namespace: capi-system` on the `controller-manager` of
config/manager/manager.yaml. The graph only had the declared name, so the
name the cluster runs -- the one docs, scripts and tests quote -- matched
no node.

A kustomization with a namePrefix or nameSuffix now gives every object of
its build a variant: a Resource named with the deployed name, in the
namespace the build gives it, `ecosystem: kustomize`, which the
kustomization's File DEFINES and which OVERRIDEs {via: namePrefix |
nameSuffix, kustomization, line} the object. Prefixes of nested
kustomizations compose outermost first, suffixes innermost first, and the
outermost namespace wins. CustomResourceDefinitions, Namespaces and
apiregistration APIServices keep their names, as kustomize's own
transformers skip them. Variants are minted after the Kubernetes joins:
inside a build, references use the declared names.

Measured on 17 infrastructure repositories: 56 of 56 renamed objects in
the 6 renaming kustomizations against an independent reading, with
deployed name, namespace and base object; nothing else changed (56 nodes,
112 edges added, no property of an existing node); deterministic over
three runs; the incremental re-index equals a full index when a prefix
changes, goes, nests, or a renamed object is renamed or deleted.

Signed-off-by: Martin Vogel <[email protected]>
kubespray's 1,425 tasks and 65 handlers were text inside YAML files: a
search for "Containerd | restart containerd" found no node, and nothing
said which task triggers which handler.

Each task and handler of an Ansible file is now a Function, named by its
`name:` (else its module), {source: ansible, kind, module}, which its file
DEFINES -- a playbook's pre_tasks / tasks / post_tasks / handlers, a role's
tasks/ and handlers/ files, and the files those include; a block's block /
rescue / always tasks, never the block itself. A notify CALLS {via:
notify, line} every handler of that name or listen topic in the task's own
role (the directory before its tasks/, so nested roles stay apart), else
in its file.

A top-level YAML list is not Ansible by itself: JSON patches, image maps
and service lists are lists too, and would have made some 3,000 false
tasks in five other repositories. A file counts when it is a playbook, a
role's tasks or handlers file, or included by one (to a fixpoint).
Quoted names have their YAML escapes decoded. A notify joins two files,
so a change to Ansible content takes the full rebuild
(closure_ansible_decline); without it, renaming a listen topic or adding
a notifying task left the incremental graph different from a full index.

Measured on 17 infrastructure repositories: 1,490 of 1,490 tasks and
handlers (file, line, kind, name, module) and 276 of 276 notify edges
against an independent reading, all in kubespray; nothing in the other
16; the P7.1 include edges unchanged (211 of 211); deterministic over
three runs; the incremental re-index equals a full index for handler and
task file edits, a deleted handler file and a playbook edit. The only
other change: as every function joins the similarity corpus, four
SEMANTICALLY_RELATED scores in kubespray's Python module moved and one
pair was added.

Signed-off-by: Martin Vogel <[email protected]>
…host

security-static allows URL literals in C sources only on its allow-list; a comment continuation line is scanned like code, so the curl example in pass_shell.c named a real host. It now uses the placeholder https://host/... that scripts/security-audit.sh accepts.

Signed-off-by: Martin Vogel <[email protected]>
@DeusData

Copy link
Copy Markdown
Owner Author

Independent precision sample for the new edge kinds

Every new edge kind was sampled (seed 20261014, spread across repositories) and each sampled edge was checked by reading the source at the cited lines, independently of the code under test.

Result: 849 of 850 sampled edges correct (0.9988); every kind at or above 0.98.

Kind Edges Repos Checked Precision
F1 CONFIGURES env_read 82 4 50 0.980
F2 CONNECTS_TO 129 5 50 1.000
F3 HTTP_CALLS curl/wget 94 13 50 1.000
P5.1 OVERRIDE patch / IMPORTS patches 126 / 89 5 50 / 50 1.000
P5.2 DEPLOYS helmcharts / IMPORTS valuesFile 20 / 19 2 all 1.000
P5.3 OVERRIDE labelSelector 4 1 all 1.000
P5.4 OVERRIDE namePrefix (+ variants) 56 1 all 1.000
P6.1 USAGE / DEPENDS_ON terraform 12,606 / 791 7 50 / 50 1.000
P6.2 IMPORTS / USAGE ref:output 349 / 644 6 50 / 50 1.000
P6.3 OVERRIDE tfvars (own / inferred) 124 / 114 3 50 / 50 1.000
P6.4 Terragrunt IMPORTS 36 1 all 1.000
P6.5 DEPLOYS terraform → Resource 21 2 all 1.000
P7.1 IMPORTS ansible 211 1 all 1.000
P7.2 CALLS notify / task Functions 276 / 1,490 1 all 1.000

Kinds found in fewer than three of the 17 repositories (the constructs exist nowhere else) were checked in full, by hand up to 50 edges and by an independent script beyond that.

The one wrong edge (microservices-demo): EnvVar PORT → Variable server in src/paymentservice/index.js. The variable that reads process.env['PORT'] is PORT (line 70); server (line 73) only uses it. F1 takes its reader from the existing env_access edge, which already points at the consumer. That attribution predates this PR and is a follow-up.

Not covered by the corpora: Terragrunt dependency → DEPENDS_ON has no instance in the 17 repositories; its probe test covers it.

Follow-ups found, not blocking:

  • Repeated Terraform block types in one file (two locals {}, several moved {}) share one node carrying one occurrence's line range. The references are correct, but in 1.9% of Terraform edges the cited line lies outside that range.
  • Two inline kustomize patches on the same object, from one kustomization, fold into one edge (the edge key is source, target and type).
  • Secret hygiene: none of the 1,305 .tfvars Variable nodes stores a value.

…ffset

GCC 13 on x86-64 (-O1, ASan/UBSan, -Werror) rejected an_key's memmove(out, out + bl, strlen(out + bl) + 1) as -Wstringop-overread, a region of size 0; arm64 GCC and clang accept it. The key length is now measured once and the prefix compared with memcmp. Every C file this branch changes compiles warning-free with that GCC.

Signed-off-by: Martin Vogel <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant