Skip to content

[Terraform plan] tfplan parser losing track of resources#196

Merged
gh-worker-dd-mergequeue-cf854d[bot] merged 2 commits into
mainfrom
benjamin.chouraqui/fix-tfplan-parser
Jul 1, 2026
Merged

[Terraform plan] tfplan parser losing track of resources#196
gh-worker-dd-mergequeue-cf854d[bot] merged 2 commits into
mainfrom
benjamin.chouraqui/fix-tfplan-parser

Conversation

@ChouraquiBen

@ChouraquiBen ChouraquiBen commented Jun 18, 2026

Copy link
Copy Markdown
Contributor

Motivation

PR #194 enables Terraform plan scanning behind the --x-terraform-plan flag, but the tfplan parser has critical bugs that cause resource loss in multi-module plans. These bugs prevent accurate scanning of real-world Terraform plans with modules, count, or for_each.

Related: #194

Changes

Bug Fixes

  1. Fixed module resource loss: Resources from child modules were overwriting root module resources of the same type

    • Root cause: readModule was reinitializing kp.Resource[resource.Type] for each module, erasing previous resources
    • Solution: Check if map exists before creating (accumulate, don't reinitialize)
    • Disambiguation: Child module resources prefixed with module path (e.g., module.networking.subnet)
    • Backward compatibility: Root module resources keep simple names for existing Rego queries
  2. Fixed count/for_each resource loss: Resources using count or for_each were overwriting each other

    • Root cause: Only resource.Name used as key, ignoring Index field
    • Solution: Format keys as name[0], name[1] for count and name["prod"], name["dev"] for for_each
    • Added formatResourceKeyWithIndex function to handle both numeric and string indexes

Test Coverage

Added 6 comprehensive test cases in tfplan_test.go:

  • Multi-module with same resource type
  • Same resource name in different modules
  • Deeply nested modules (3 levels deep)
  • Multiple sibling modules with same resource types
  • Count creates multiple instances (3 instances)
  • For_each creates multiple instances (3 environments)

Author Checklist

  • I have reviewed my own PR.
  • I have added or updated relevant unit tests where necessary. If no tests are added, I've explained why.
  • All new and existing tests pass.
  • I have tested my changes on staging (if applicable).
  • I have updated any relevant documentation (if applicable).

QA Instruction

Test the fixes:

  1. Module resource accumulation:
go test -v ./pkg/parser/json/... -run "TestJson_parseTFPlan/test_-_multi-module"
go test -v ./pkg/parser/json/... -run "TestJson_parseTFPlan/test_-_same_resource_name"
go test -v ./pkg/parser/json/... -run "TestJson_parseTFPlan/test_-_deeply_nested"
  1. Count/for_each support:
go test -v ./pkg/parser/json/... -run "TestJson_parseTFPlan/test_-_count"
go test -v ./pkg/parser/json/... -run "TestJson_parseTFPlan/test_-_for_each"
  1. Full test suite:
go test ./pkg/parser/json/...

Verify Rego query compatibility:

Existing Rego queries using split(resource, ".")[1] patterns continue to work because root module resources maintain simple names without module prefixes.

Blast Radius

  • Impact: Only affects Terraform plan JSON parsing (currently gated behind --x-terraform-plan experimental flag)
  • Scope: Fixes prevent data loss in scans, enabling accurate security analysis of multi-module Terraform plans
  • Compatibility: Backward compatible with existing Rego queries

Additional Notes

Key Implementation Details

Resource key formats:

  • Root module: resource_name (no prefix for backward compatibility)
  • Child module: module.path.resource_name
  • Count: resource_name[0], resource_name[1], etc.
  • For_each: resource_name["prod"], resource_name["staging"], etc.
  • Combined: module.app.web[0], module.storage.bucket["prod"]

Before these fixes:

{
  "resource": {
    "aws_s3_bucket": {
      "backup": {...}  // Only last module's bucket survives
    }
  }
}

After these fixes:

{
  "resource": {
    "aws_s3_bucket": {
      "main": {...},  // Root module bucket
      "module.storage.backup": {...}  // Child module bucket - both preserved!
    }
  }
}

I submit this contribution under the Apache-2.0 license.

@datadog-official

datadog-official Bot commented Jun 18, 2026

Copy link
Copy Markdown

🎯 Code Coverage (details)
Patch Coverage: 80.95%
Overall Coverage: 50.11% (+0.02%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: f1c70d9 | Docs | Datadog PR Page | Give us feedback!

@ChouraquiBen
ChouraquiBen force-pushed the benjamin.chouraqui/fix-tfplan-parser branch from eb783a6 to 3950501 Compare June 18, 2026 12:30
@ChouraquiBen
ChouraquiBen marked this pull request as ready for review June 18, 2026 14:40
@ChouraquiBen
ChouraquiBen requested a review from a team as a code owner June 18, 2026 14:40

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3950501f4a

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread pkg/analyzer/analyzer.go Outdated
@ChouraquiBen
ChouraquiBen changed the base branch from main to graphite-base/196 June 22, 2026 09:53
@ChouraquiBen
ChouraquiBen force-pushed the benjamin.chouraqui/fix-tfplan-parser branch from 3950501 to ba1d3e1 Compare June 22, 2026 09:53
@ChouraquiBen
ChouraquiBen changed the base branch from graphite-base/196 to benjamin.chouraqui/enable-gated-tfplan-scanning June 22, 2026 09:53

ChouraquiBen commented Jun 22, 2026

Copy link
Copy Markdown
Contributor Author

@ChouraquiBen
ChouraquiBen force-pushed the benjamin.chouraqui/fix-tfplan-parser branch from ba1d3e1 to 63b02f2 Compare June 22, 2026 10:34
@ChouraquiBen
ChouraquiBen force-pushed the benjamin.chouraqui/enable-gated-tfplan-scanning branch from 686a485 to 254bbd2 Compare June 22, 2026 10:34
@ChouraquiBen
ChouraquiBen force-pushed the benjamin.chouraqui/fix-tfplan-parser branch from 63b02f2 to 648ce2d Compare June 25, 2026 09:10
@ChouraquiBen
ChouraquiBen force-pushed the benjamin.chouraqui/enable-gated-tfplan-scanning branch from b7799cf to d23e0c3 Compare June 25, 2026 10:22
@ChouraquiBen
ChouraquiBen force-pushed the benjamin.chouraqui/fix-tfplan-parser branch 2 times, most recently from 6e89b53 to c4aba08 Compare June 25, 2026 10:55
@ChouraquiBen
ChouraquiBen force-pushed the benjamin.chouraqui/enable-gated-tfplan-scanning branch 2 times, most recently from bb73637 to 004184a Compare June 25, 2026 11:05
@ChouraquiBen
ChouraquiBen force-pushed the benjamin.chouraqui/fix-tfplan-parser branch from c4aba08 to 4aeb500 Compare June 25, 2026 11:05
Base automatically changed from benjamin.chouraqui/enable-gated-tfplan-scanning to main June 25, 2026 11:15
@ChouraquiBen
ChouraquiBen force-pushed the benjamin.chouraqui/fix-tfplan-parser branch from 4aeb500 to 1c3e319 Compare June 26, 2026 13:32
@ChouraquiBen ChouraquiBen changed the title [Terraform plan] Fix tfplan parser [Terraform plan] tfplan parser loosing track of resources Jun 29, 2026
@ChouraquiBen ChouraquiBen changed the title [Terraform plan] tfplan parser loosing track of resources [Terraform plan] tfplan parser losing track of resources Jun 30, 2026
@ChouraquiBen
ChouraquiBen force-pushed the benjamin.chouraqui/fix-tfplan-parser branch from 1c3e319 to f1c70d9 Compare June 30, 2026 14:21
@ChouraquiBen

Copy link
Copy Markdown
Contributor Author

Test rules failing is actually a good news: the unexpected finding is legit

@gh-worker-dd-mergequeue-cf854d
gh-worker-dd-mergequeue-cf854d Bot merged commit f67c647 into main Jul 1, 2026
20 of 21 checks passed
@gh-worker-dd-mergequeue-cf854d
gh-worker-dd-mergequeue-cf854d Bot deleted the benjamin.chouraqui/fix-tfplan-parser branch July 1, 2026 13:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants