TL;DR

~7.3× faster wall-clock time. 696→700 findings (intentional, not a regression).
No timeouts. Correctness preserved. Memory footprint stable (~1.4–1.5 GB RSS throughout).

All benchmarks below use cloud-inventory with -t Terraform (28,914 files).
Peak RSS measured with /usr/bin/time -l.

The scan goes from 337 s on main to 46 s (with --x-parallelparsing) with all improvements stacked.
The dominant individual win is a single change, the Terraform per-directory variable cache,
which alone cuts wall time from 134 s to 49 s (2.7×).

Wall Time

scanner-00-main           ████████████████████████████████████████████████████  337 s  (baseline, serial)
scanner-01-parallel       █████████████████                                     137 s  (−59%)
scanner-02-inspector      █████████████████                                     134 s  (−60%)
scanner-03-tf-parser      ██████                                                 49 s  (−85%)  ← biggest leap
scanner-04-modules        ██████                                                 48 s  (−86%)
scanner-05-runner         ██████                                                 46 s  (−86%)  ← final (includes terraform.rego fix)

Results at a Glance

Phase Breakdown (CPU time)

CPU time is aggregate across all goroutines; wall time for a phase can be shorter when work is parallelised.

prepare_sources: file parsing phase

scanner-00  ████████████████████████████████████████████████████████████  593 s  serial
scanner-01  ████████████████████████████████████████████████████████████████████████████████████  859 s  (more CPU, less wall — parallel)
scanner-02  ████████████████████████████████████████████████████████████████████████████████  823 s
scanner-03  ████████                                                                           86 s  ← dirVarsCache lands
scanner-04  ███████                                                                            74 s
scanner-05  ███                                                                                28 s  ← HCL body cache

start_scan: OPA rule evaluation phase

scanner-00  ██████████████████████████████████████████████████████  54.4 s
scanner-01  ████████████████████████████████████████████████████████████  60.6 s  (more queries, parallel parse overhead)
scanner-02  █████████████████████████████████████████████████████████████  60.9 s
scanner-03  █████████████████████████████████████████████████████████  57.5 s
scanner-04  ██████████████████████████████████████████████           46.3 s  ← precomputed stores start helping
scanner-05  ████████████████████████████████                         32.8 s  ← runner buffer pool

Full phase table (CPU seconds)

Memory Profile (peak RSS)

Measurements are peak resident set size from /usr/bin/time -l.

scanner-00-main      ████████████████████████████████████████████████████████████████████████  1.40 GB  (baseline)
scanner-01-parallel  ████████████████████████████████████████████████████████████████████████  1.48 GB
scanner-02-inspector ████████████████████████████████████████████████████████████████████████  1.44 GB
scanner-03-tf-parser ████████████████████████████████████████████████████████████████████████  1.43 GB
scanner-04-modules   ████████████████████████████████████████████████████████████████████████  1.46 GB
scanner-05-runner    ████████████████████████████████████████████████████████████████████████  1.47 GB

Memory is flat across all steps (~1.4–1.5 GB). The caches (dirVarsCache, HCL body cache)
trade CPU time for speed, not memory — the saved allocations are offset by retaining parsed
structures, leaving net RSS unchanged. The dominant cost is the OPA input data (one document
per Terraform resource) held in memory during start_scan; that does not change across steps.

Per-Optimization Analysis

01: Race fix + --x-parallelparsing

Wall: 337 s → 137 s (−59%)

Parallel file parsing cuts wall time significantly by spreading the I/O and parse load across cores.
prepare_sources CPU increases (more total work due to thread overhead and contention), but wall
time drops because the work runs concurrently across cores instead of serially.

The underlying race fixes are necessary prerequisites:

• vulnerability_builder.go: removed SetupLogs shared-state write under concurrency
• docker_detect.go: changed shallow slice alias to a deep copy preventing concurrent mutations
• inspector.go: removed the decode-phase wall-clock deadline that was silently dropping findings

02: OPA v1 + precomputed stores + pre-parsed Rego

Wall: 137 s → 134 s (−2%)

Wall time barely moves at this step because prepare_sources still dominates. The OPA changes
precompute one inmem.Store per platform from the merged input data once per scan (instead of
rebuilding it from JSON for every query), and pre-parse the 82 KB of shared Rego library text once
at startup via rego.ParsedModule(). The benefit materialises in start_scan CPU time — it
sets up the gains visible in later steps.

03: Terraform per-directory variable cache (dirVarsCache)

Wall: 134 s → 49 s (−63%)

The single biggest improvement in the suite.

Without the cache, every .tf file in a directory triggers getInputVariables +
getDataSourcePolicy, which re-reads and re-parses all .tf files in that directory to
build the variable/locals/data-source map. In a large repository with many
files per directory, this is O(N²) re-parsing. The dirVarsCache + singleflight.Group
reduces it to one parse per directory, shared (read-only cloned) by all files in that directory.

04: Module HCL body cache + parallel parseHCLBodies

Wall: 49 s → 48 s (−2%)

Two benefits:

1. Body cache (parsedBodyCache sync.Map): the same .tf bytes are never lexed/parsed more
   than once per process lifetime. In multi-phase scans (module discovery, then engine inspection)
   this eliminates a full second pass over all files.
2. Parallel parse: parseHCLBodies now dispatches parse work to a worker pool (GOMAXPROCS
   workers), then merges in original file order for deterministic locals/variable shadowing.

05: Runner buffer pool + terraform.rego fix

Wall: 48 s → 46 s (−4%)

Two distinct sub-changes:

Runner buffer pool (scanReadBufferPool sync.Pool): reuses the 1 MiB read buffers across
files instead of allocating one per file. Reduces allocation pressure in prepare_sources
(CPU: 74 s → 28 s) and start_scan (CPU: 46 s → 33 s).

terraform.rego fix (resolve_reference_name): changes the rule to collect all matching
candidates into a set before selecting one, avoiding OPA's "multiple-output" error when the same
resource name appears across multiple module instances in separate documents. This is why
violations tick up from 696 to 700 — findings that were being silently suppressed by OPA
evaluation errors are now correctly reported (see Correctness section).

Correctness Check

All binaries produce the same exit code (60 = CRITICAL findings present) and scan the same
28,914 Terraform files (-t Terraform on cloud-inventory).

+4 violations, +1 rule (05 onwards): This is the terraform.rego fix for
resolve_reference_name. Before the fix, OPA's multi-value rule evaluation silently failed when
the same resource appeared across multiple module instance documents, suppressing findings. The
fix makes that function deterministic, surfacing 4 previously suppressed findings and making 1
additional rule produce results. This is a correctness improvement, not a regression.

Key Takeaways

1. ~7.3× end-to-end speedup (337 s → 46 s with --x-parallelparsing) over main on
   cloud-inventory / -t Terraform.

2. The terraform dirVarsCache is the most impactful single change: cuts wall time from
   134 s to 49 s (−63%). The root cause was O(N²) re-parsing of directory contents for every
   file in that directory.

3. Memory footprint is flat (~1.4–1.5 GB RSS) across all steps. The caches trade CPU cycles
   for speed without adding net memory.

4. No regressions. All scans complete, finding counts either match or increase for documented
   correctness reasons, and no rules time out on any improved binary.

Author Checklist

• I have reviewed my own PR.
• I have added or updated relevant unit tests where necessary. If no tests are added, I've explained why.
• All new and existing tests pass.
• I have tested my changes on staging (if applicable).
• I have updated any relevant documentation (if applicable).

I submit this contribution under the Apache-2.0 license.