Skip to content

[K9CODESEC-2947] Add network Terraform module resolvers#248

Merged
whitemerch merged 5 commits into
mainfrom
chakib.hamie/terraform_module_resolver_gogetter
Jul 8, 2026
Merged

[K9CODESEC-2947] Add network Terraform module resolvers#248
whitemerch merged 5 commits into
mainfrom
chakib.hamie/terraform_module_resolver_gogetter

Conversation

@whitemerch

@whitemerch whitemerch commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Tip

This PR is best reviewed commit by commit.

Motivation

#240 added the offline resolver chain; #242 added pinned-git resolution via bare clones. Many real-world Terraform repos still reference registry modules (registry.terraform.io/...) and other remote URLs that require a network fetch to materialize .tf content.

This PR completes the resolver library side of remote module support. Nothing in the scan path invokes these resolvers yet, that wiring is a follow-up.

Related Ticket: K9CODESEC-2947

Changes

Go-getter resolver

Adds GoGetterResolver on top of hashicorp/go-getter for generic remote downloads (git, HTTP, S3, etc.). Includes global and per-host concurrency limits (IAC_MODULE_FETCH_CONCURRENCY, IAC_MODULE_HOST_FETCH_CONCURRENCY), a 200 MiB scan-level download cap(don't ask me why this value), host allowlists, transient-error retries, and integration with the shared on-disk moduleCache (content-addressed by source + resolved version).

Registry HTTP layer

Adds registry discovery, version resolution, and download-URL translation for public and private registries (TF_TOKEN_* auth). Go-getter depends on this layer, registry sources are translated to X-Terraform-Get URLs before download, which is why both land in one PR.

Source routing

Pinned git::...?ref= sources stay on BareGitResolver (go-getter explicitly defers them). Unpinned git::https:// sources keep HTTPS transport instead of being rewritten to SSH. Registry sources with subdirectories (ns/name/provider//subdir) are classified correctly. Pinned registry versions skip network discovery when resolving the cache key, so offline cache hits work.

Author Checklist

  • I have reviewed my own PR.
  • I have added or updated relevant unit tests where necessary.
  • All new and existing tests pass.
  • I have tested my changes on staging (if applicable).
  • I have updated any relevant documentation (if applicable).

QA Instruction

CI should pass.

Impact

Additive only: new code under pkg/parser/terraform/modules/resolver, plus go.mod/go.sum and LICENSE-3rdparty.csv for go-getter transitive deps. No CLI, scan, or engine behavior changes.

Additional Notes

I submit this contribution under the Apache-2.0 license.

@whitemerch
whitemerch requested a review from a team as a code owner July 7, 2026 15:00
@whitemerch
whitemerch marked this pull request as draft July 7, 2026 15:03

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1ad94f2868

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread pkg/parser/terraform/modules/resolver/gogetter.go
Comment thread pkg/parser/terraform/modules/resolver/gogetter.go Outdated
Comment thread pkg/parser/terraform/modules/resolver/registry_cache.go
@datadog-official

datadog-official Bot commented Jul 7, 2026

Copy link
Copy Markdown

🎯 Code Coverage (details)
Patch Coverage: 43.10%
Overall Coverage: 49.73% (+0.07%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 9f3a939 | Docs | Datadog PR Page | Give us feedback!

@whitemerch
whitemerch force-pushed the chakib.hamie/terraform_module_resolver_gogetter branch from b001341 to aa665ba Compare July 7, 2026 15:26
@whitemerch
whitemerch marked this pull request as ready for review July 7, 2026 15:33

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: aa665ba2cb

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread pkg/parser/terraform/modules/resolver/registry.go Outdated
Comment thread pkg/parser/terraform/modules/resolver/registry_cache.go Outdated
@whitemerch
whitemerch merged commit d449a40 into main Jul 8, 2026
19 checks passed
@whitemerch
whitemerch deleted the chakib.hamie/terraform_module_resolver_gogetter branch July 8, 2026 09:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants