Motivation

In server mode (serve), one process handles many /analyze requests, but today every request rebuilds the full scan pipeline from scratch, it recompiles all rules, re-parses libraries, and discards the result. Warm scans are therefore as slow as cold ones, dominated by redundant per-request Rego compilation rather than the work that actually changed.

This PR makes repeated server-mode scans fast by eliminating that redundant work, behind opt-in experimental flags. The default path and the CLI are unchanged.

JIRA Ticket K9CODESEC-2670

Changes

Three independent, opt-in server-mode optimizations plus a correctness fix:

• Compiled-rule cache (--x-use-rules-cache): process-global cache of compiled OPA queries, keyed by rule identity + library hash + base-store data hash. Warm scans reuse compiled rules and skip recompilation. Off by default; never enabled for one-shot CLI scans (pure memory cost there). Cache keys use xxhash (allocation-free on the hot path).
• Shared compilation (--x-disable-rule-isolation): co-compile all rules + libraries once into a single compiler (rules rewritten to unique packages) instead of isolating each rule, so the shared library AST is retained once rather than copied into every compiled query. Rules that fail to parse/compile fall back to isolated compilation, preserving correctness. Uses WithCapabilities (not the deprecated WithUnsafeBuiltins) to keep http.send / opa.runtime blocked.
• Parallel file parsing (--x-parallelparsing): fan the per-file parse across CPUs in the in-memory source provider. Reuses the existing provider worker-count policy.
• Deterministic Terraform module data: sort the module Resources slice so merged module input data is byte-stable across scans. Required for the compiled-rule cache to hit when modules are present, and a reproducibility fix in its own right.

Reading the table:

• Cache alone is the biggest single speed win (−78% warm) but retains every isolated compiled query → 590 MB. This is the memory problem.
• Disabling isolation is what tames that memory: cache + shared compilation is just as fast but drops the heap from 590 MB → 54 MB (~11× less), because the library AST is retained once instead of ~687 times.
• Parallel parsing alone does nothing (parsing isn't the bottleneck while compilation dominates), but adds a further ~13% once compilation is cached away (0.73 s → 0.65 s).

Bottom line (all flags on): warm scans go from 1.96 s → 0.65 s (−67%) and CPU from 17 s → 1.7 s (−90%), while live heap stays at ~53 MB — only ~20 MB above the no-cache baseline and ~11× below caching-with-isolation. Disabling rule isolation is the lever that makes the cache affordable.

Author Checklist

• I have reviewed my own PR.
• I have added or updated relevant unit tests where necessary. If no tests are added, I've explained why.
• All new and existing tests pass.
• I have tested my changes on staging (if applicable).
• I have updated any relevant documentation (if applicable).

QA Instruction

All flags are off by default; the default scan path and CLI are unchanged, so existing behavior should be byte-identical.

To verify the optimizations in server mode:

1. Build: make build.
2. Start a server: ./bin/datadog-iac-scanner serve --port 8000 --enable-shutdown (optionally with --x-use-rules-cache --x-disable-rule-isolation --x-parallelparsing).
3. POST a repo + rules to /ide/v1/iac/analyze twice and confirm:
   • Findings are identical with and without the flags (the key correctness check).
   • With --x-use-rules-cache, the second (warm) request is dramatically faster than the first.
   • With --x-disable-rule-isolation, the server logs Rule isolation disabled: N/N queries served from shared compiler.

Blast Radius

Datadog IaC Scanner only. The changes are gated behind experimental, hidden flags that default off; with no flags set, the engine, CLI, and existing server behavior are unchanged. The one always-on change is sorting the Terraform module Resources slice, it makes module input data deterministic (correctness/repro improvement) and does not change findings.

Additional Notes

• The compiled-rule cache is currently unbounded (no eviction/size cap). So if you do use -x-use-rules-cache you also must use --x-disable-rule-isolation
• Flags are intentionally experimental/hidden

If you need to share anything else along with your PR, please do it here.

I submit this contribution under the Apache-2.0 license.