Skip to content

[K9CODESEC-2306] Resolve local Terraform modules during IaC scans#192

Merged
whitemerch merged 4 commits into
mainfrom
chakib.hamie/local_module_evaluation
Jun 18, 2026
Merged

[K9CODESEC-2306] Resolve local Terraform modules during IaC scans#192
whitemerch merged 4 commits into
mainfrom
chakib.hamie/local_module_evaluation

Conversation

@whitemerch

@whitemerch whitemerch commented Jun 17, 2026

Copy link
Copy Markdown
Contributor

Tip

This PR is best reviewed commit by commit.

Motivation

Terraform scans previously treated local module calls as call-site blocks only. A rule could inspect a module block's inputs and map a subset of them back to expected resource attributes via get_module_equivalent_key, but the scanner never materialized the resources inside a local module with the caller's values. So a misconfiguration that only exists because of what the caller passed in was invisible to any rule that didn't carry a bespoke module-mapping branch.

This PR instantiates local modules during scans: it binds caller inputs to the module's variables (→ defaults → unknown), evaluates locals/functions/conditionals, recurses into nested local modules, and emits the resolved resources into the scan payload. Rules then run against real resources (e.g. aws_s3_bucket) with no module-awareness required.

Related Ticket: K9CODESEC-2306

Before / After

# main.tf (root)
module "logs" {
  source      = "./modules/bucket"
  bucket_name = "app-logs"
  acl         = "public-read"   # insecure, set by the caller
}
# modules/bucket/main.tf
variable "bucket_name" {}
variable "acl" { default = "private" }

resource "aws_s3_bucket" "this" {
  bucket = var.bucket_name
  acl    = var.acl
}

Before: the root saw the module "logs" call-site, and modules/bucket/main.tf was scanned standalone with acl = var.acl unresolved. The public ACL was only catchable if the S3 rule had a get_module_equivalent_key branch, most don't, so it was missed.

After: the module is instantiated with acl = "public-read", materialized as aws_s3_bucket.this, and the normal "S3 bucket must not be public" rule fires on the resolved value. The called body file is suppressed and the local call-site stripped, so there is exactly one finding.

Implementation

  • New evaluator package pkg/parser/terraform/tfeval (module-instance tree: input binding, variable defaults, locals fixed-point, nested local modules with depth + cycle guards, output up-propagation).
  • Engine wiring in pkg/engine/module_resolve.go + Inspect: instantiate local modules, inject synthetic resource documents, suppress called bodies, strip instantiated local call-sites.

Things reviewers should know

  • Unknown values use a placeholder (__DD_TFEVAL_UNKNOWN__) so attribute keys stay present. This prevents MissingAttribute false-positives from treating an unresolved expression as absent. Trade-off: value-equality rules see the sentinel rather than a concrete value (i.e. this swaps potential MissingAttribute FPs for potential IncorrectValue FPs).
  • Finding attribution shifts for local modules. Findings now report at the module's resource definition (file + line) with the call chain, not at the call site. This changes findingId, which matters for suppression and ticket continuity in hosted scans.
  • Dedup model: called module body files are suppressed; only instantiated local call-sites are stripped from root docs; remote/registry call-sites are left intact so existing Rego branches still fire; uncalled modules are still scanned standalone. Call-sites are only stripped when the module contributed synthetic docs (never remove coverage without a replacement).
  • Resilience: evaluation is wrapped in recover() (a bad module logs and is skipped, scan continues); if every root fails to evaluate, no suppression/stripping is applied.
  • Root *.tfvars / *.auto.tfvars feed module inputs; root-module resources themselves are unchanged (still scanned via the standard converter).

Out of scope (follow-ups)

  • Remote / registry modules: unchanged; still treated as call-site blocks. It will come in the future.
  • count / for_each expansion to N>1: only the disable case (count = 0, for_each = {}) is handled; otherwise a block is materialized once, and count.index / each.key resolve to unknown. Incoming in the next PR.
  • Cross-resource references (x = aws_y.z.id, data sources): resolve to unknown. Incoming in the next PR.
  • Multiple calls to the same module directory / per-instance attribution: distinct instances can collapse to one synthetic doc (same defining file/line).
  • Retiring get_module_equivalent_key: kept running in parallel; removal is a later PR once instantiated coverage is proven.

Followup ticket

Blast radius

All Terraform scans: local modules are evaluated and merged into the OPA payload. Remote and registry modules are unchanged.

Testing / QA

  1. go test ./pkg/parser/terraform/tfeval/... ./pkg/engine/...
  2. test-rules (authoritative): confirm no regressions and net-new coverage on local-module fixtures; watch the unknown-placeholder FP rate.
  3. Scan a fixture that calls a local module and confirm findings target instantiated resources with caller-resolved attributes; confirm remote/registry module call-sites are unchanged and there are no duplicate findings (stripped call-site + instantiated resource).

I submit this contribution under the Apache-2.0 license.

Comment thread pkg/engine/inspector.go Fixed
@datadog-prod-us1-4

datadog-prod-us1-4 Bot commented Jun 17, 2026

Copy link
Copy Markdown

🎯 Code Coverage (details)
Patch Coverage: 73.05%
Overall Coverage: 49.12% (+0.79%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 8588f33 | Docs | Datadog PR Page | Give us feedback!

@whitemerch
whitemerch force-pushed the chakib.hamie/local_module_evaluation branch 2 times, most recently from 2e7a195 to ab8066f Compare June 17, 2026 07:46
chatgpt-codex-connector[bot]

This comment was marked as resolved.

@whitemerch
whitemerch force-pushed the chakib.hamie/local_module_evaluation branch 2 times, most recently from aa80166 to 8411105 Compare June 17, 2026 09:54
chatgpt-codex-connector[bot]

This comment was marked as resolved.

@whitemerch
whitemerch force-pushed the chakib.hamie/local_module_evaluation branch from 8411105 to ea8f6d5 Compare June 17, 2026 10:12
…ed module resources merge into the OPA input without double-counting.
@whitemerch
whitemerch force-pushed the chakib.hamie/local_module_evaluation branch from ea8f6d5 to 3d10e81 Compare June 17, 2026 10:22
@whitemerch
whitemerch marked this pull request as ready for review June 17, 2026 11:17
@whitemerch
whitemerch requested a review from a team as a code owner June 17, 2026 11:17

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3d10e81bad

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

if !tfmodules.LooksLikeLocalModuleSource(cleanSource) {
continue
}
if calledDirs[filepath.Clean(filepath.Join(fileDir, cleanSource))] {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Normalize absolute module sources before stripping

For absolute local sources (including file:///... after StripGetterPrefix), this joins the absolute source to the caller directory, so the key does not match the absolute child dir recorded by EvaluateModule. In that setup the child files are still suppressed and synthetic resource docs are added, but the root module block remains in the OPA payload, so rules with both instantiated-resource and legacy module-call branches can report duplicate findings for the same module.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In Go, filepath.Join("/some/dir", "/absolute/path") returns /absolute/path, the absolute component wins. So the join produces the correct absolute key and matches calledDirs as expected

Comment thread pkg/parser/terraform/tfeval/evaluator.go
Comment thread pkg/parser/terraform/tfeval/parse.go
@whitemerch whitemerch changed the title Resolve local Terraform modules during IaC scans [K9CODESEC-2306] Resolve local Terraform modules during IaC scans Jun 17, 2026
…erences like module.a.output in a sibling module's inputs produce known values instead of unknown.

@MikaYuoadas MikaYuoadas left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great feature overall, just pointed a few area of concern that need to be addressed.

Comment thread pkg/engine/module_resolve.go Outdated
Comment thread pkg/engine/module_resolve.go Outdated
Comment on lines +239 to +246
if prev, exists := byAbsPath[abs]; exists && prev != nil && prev.ID != f.ID {
contextLogger.Warn().Msgf(
"duplicate terraform file path in scan input %s (file ids %s and %s); using the latter",
abs, prev.ID, f.ID,
)
}
byAbsPath[abs] = f
filesByDir[dir] = append(filesByDir[dir], f)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Question: In case of duplicates byAbsPath will use the latter f but filesByDir[dir] slice will have both f previous and current value appended making byAbsPath and filesByDir diverge in the total number of files they hold. Won’t that be an issue?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On a duplicate abs path ,byAbsPath keeps the latter f while filesByDir[dir] holds both, so both IDs get suppressed but the synthetic doc is attributed only to the latter. In practice both IDs point to identical content so no finding is lost, just attributed to one of the two. What do you think?

Comment thread pkg/engine/module_resolve.go Outdated
Comment on lines +111 to +113
// evalRootDir distinguishes the same child module reached from different roots
// (different variable bindings) so synthetic documents are not incorrectly deduped.
extra = append(extra, instantiatedDocs(resources, byAbsPath, repoPath, seen, dir)...)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Question: What happens to findings that are found on two different root stack call of the same module’s resource? Won’t they be dedup because they’ll have the same fingerprint? Is it what we want? And if so which resolved value will be shown to the end user?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment thread pkg/engine/module_resolve.go
…by dropping only their resource blocks, apply document mutations only after evaluation succeeds, and tidy the resolve helper's flag and docs.

@MikaYuoadas MikaYuoadas left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, all my comments were addressed 👍

@whitemerch
whitemerch merged commit e4bae37 into main Jun 18, 2026
19 checks passed
@whitemerch
whitemerch deleted the chakib.hamie/local_module_evaluation branch June 18, 2026 17:14
@whitemerch whitemerch mentioned this pull request Jun 24, 2026
5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants