[K9CODESEC-2306] Resolve local Terraform modules during IaC scans#192
Conversation
|
🎯 Code Coverage (details) 🔗 Commit SHA: 8588f33 | Docs | Datadog PR Page | Give us feedback! |
2e7a195 to
ab8066f
Compare
aa80166 to
8411105
Compare
8411105 to
ea8f6d5
Compare
…solved attributes to rule payloads.
…ed module resources merge into the OPA input without double-counting.
ea8f6d5 to
3d10e81
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 3d10e81bad
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if !tfmodules.LooksLikeLocalModuleSource(cleanSource) { | ||
| continue | ||
| } | ||
| if calledDirs[filepath.Clean(filepath.Join(fileDir, cleanSource))] { |
There was a problem hiding this comment.
Normalize absolute module sources before stripping
For absolute local sources (including file:///... after StripGetterPrefix), this joins the absolute source to the caller directory, so the key does not match the absolute child dir recorded by EvaluateModule. In that setup the child files are still suppressed and synthetic resource docs are added, but the root module block remains in the OPA payload, so rules with both instantiated-resource and legacy module-call branches can report duplicate findings for the same module.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
In Go, filepath.Join("/some/dir", "/absolute/path") returns /absolute/path, the absolute component wins. So the join produces the correct absolute key and matches calledDirs as expected
…erences like module.a.output in a sibling module's inputs produce known values instead of unknown.
MikaYuoadas
left a comment
There was a problem hiding this comment.
Great feature overall, just pointed a few area of concern that need to be addressed.
| if prev, exists := byAbsPath[abs]; exists && prev != nil && prev.ID != f.ID { | ||
| contextLogger.Warn().Msgf( | ||
| "duplicate terraform file path in scan input %s (file ids %s and %s); using the latter", | ||
| abs, prev.ID, f.ID, | ||
| ) | ||
| } | ||
| byAbsPath[abs] = f | ||
| filesByDir[dir] = append(filesByDir[dir], f) |
There was a problem hiding this comment.
❓ Question: In case of duplicates byAbsPath will use the latter f but filesByDir[dir] slice will have both f previous and current value appended making byAbsPath and filesByDir diverge in the total number of files they hold. Won’t that be an issue?
There was a problem hiding this comment.
On a duplicate abs path ,byAbsPath keeps the latter f while filesByDir[dir] holds both, so both IDs get suppressed but the synthetic doc is attributed only to the latter. In practice both IDs point to identical content so no finding is lost, just attributed to one of the two. What do you think?
| // evalRootDir distinguishes the same child module reached from different roots | ||
| // (different variable bindings) so synthetic documents are not incorrectly deduped. | ||
| extra = append(extra, instantiatedDocs(resources, byAbsPath, repoPath, seen, dir)...) |
There was a problem hiding this comment.
❓ Question: What happens to findings that are found on two different root stack call of the same module’s resource? Won’t they be dedup because they’ll have the same fingerprint? Is it what we want? And if so which resolved value will be shown to the end user?
There was a problem hiding this comment.
…by dropping only their resource blocks, apply document mutations only after evaluation succeeds, and tidy the resolve helper's flag and docs.
MikaYuoadas
left a comment
There was a problem hiding this comment.
LGTM, all my comments were addressed 👍
Tip
This PR is best reviewed commit by commit.
Motivation
Terraform scans previously treated local module calls as call-site blocks only. A rule could inspect a
moduleblock's inputs and map a subset of them back to expected resource attributes viaget_module_equivalent_key, but the scanner never materialized the resources inside a local module with the caller's values. So a misconfiguration that only exists because of what the caller passed in was invisible to any rule that didn't carry a bespoke module-mapping branch.This PR instantiates local modules during scans: it binds caller inputs to the module's variables (→ defaults → unknown), evaluates locals/functions/conditionals, recurses into nested local modules, and emits the resolved resources into the scan payload. Rules then run against real resources (e.g.
aws_s3_bucket) with no module-awareness required.Related Ticket: K9CODESEC-2306
Before / After
Before: the root saw the
module "logs"call-site, andmodules/bucket/main.tfwas scanned standalone withacl = var.aclunresolved. The public ACL was only catchable if the S3 rule had aget_module_equivalent_keybranch, most don't, so it was missed.After: the module is instantiated with
acl = "public-read", materialized asaws_s3_bucket.this, and the normal "S3 bucket must not be public" rule fires on the resolved value. The called body file is suppressed and the local call-site stripped, so there is exactly one finding.Implementation
pkg/parser/terraform/tfeval(module-instance tree: input binding, variable defaults, locals fixed-point, nested local modules with depth + cycle guards, output up-propagation).pkg/engine/module_resolve.go+Inspect: instantiate local modules, inject synthetic resource documents, suppress called bodies, strip instantiated local call-sites.Things reviewers should know
__DD_TFEVAL_UNKNOWN__) so attribute keys stay present. This preventsMissingAttributefalse-positives from treating an unresolved expression as absent. Trade-off: value-equality rules see the sentinel rather than a concrete value (i.e. this swaps potential MissingAttribute FPs for potential IncorrectValue FPs).recover()(a bad module logs and is skipped, scan continues); if every root fails to evaluate, no suppression/stripping is applied.*.tfvars/*.auto.tfvarsfeed module inputs; root-module resources themselves are unchanged (still scanned via the standard converter).Out of scope (follow-ups)
count/for_eachexpansion to N>1: only the disable case (count = 0,for_each = {}) is handled; otherwise a block is materialized once, andcount.index/each.keyresolve to unknown. Incoming in the next PR.x = aws_y.z.id, data sources): resolve to unknown. Incoming in the next PR.get_module_equivalent_key: kept running in parallel; removal is a later PR once instantiated coverage is proven.Followup ticket
Blast radius
All Terraform scans: local modules are evaluated and merged into the OPA payload. Remote and registry modules are unchanged.
Testing / QA
go test ./pkg/parser/terraform/tfeval/... ./pkg/engine/...test-rules(authoritative): confirm no regressions and net-new coverage on local-module fixtures; watch the unknown-placeholder FP rate.modulecall-sites are unchanged and there are no duplicate findings (stripped call-site + instantiated resource).I submit this contribution under the Apache-2.0 license.