Skip to content

Add per-phase CPU/MEM profiling instrumentation behind a --profiling flag#195

Merged
MarshalX merged 2 commits into
mainfrom
ilya.siamionau/perf-metrics
Jun 18, 2026
Merged

Add per-phase CPU/MEM profiling instrumentation behind a --profiling flag#195
MarshalX merged 2 commits into
mainfrom
ilya.siamionau/perf-metrics

Conversation

@MarshalX

@MarshalX MarshalX commented Jun 18, 2026

Copy link
Copy Markdown
Contributor

Motivation

Add KICS-aligned CPU and memory profiling support to help investigate scanner performance and memory footprint by scan phase.

JIRA Ticket K9CODESEC-2331

Changes

  • Added global --profiling CPU|MEM support.
  • Added profiling around major scan phases, including query loading, source preparation, scan execution, analysis, and report generation.
  • Based the implementation on KICS metrics/profiling behavior.
  • Improved on KICS by writing per-phase pprof files to the system temp directory for direct go tool pprof inspection.
  • Improved MEM profiling clarity by logging live heap after each phase, since heap profiles are whole-process snapshots rather than phase-local allocation totals.
  • Updated profiling documentation with go tool pprof examples.

Author Checklist

  • I have reviewed my own PR.
  • I have added or updated relevant unit tests where necessary. If no tests are added, I've explained why.
  • All new and existing tests pass.
  • I have tested my changes on staging (if applicable).
  • I have updated any relevant documentation (if applicable).

QA Instruction

Run a scan with profiling enabled:

./bin/datadog-iac-scanner --profiling CPU scan -p <path>
./bin/datadog-iac-scanner --profiling MEM scan -p <path>

Verify that phase totals are logged and that pprof files are written to the temp directory. For MEM, confirm logs say Total MEM live heap after ... and inspect profiles with go tool pprof.

Blast Radius

Limited to Datadog IaC Scanner CLI profiling behavior and related documentation. Normal scans are unchanged unless --profiling is enabled.

Additional Notes

No new unit tests were added; this change is exercised through CLI profiling runs and produces runtime pprof artifacts/log output.

I submit this contribution under the Apache-2.0 license.

@MarshalX
MarshalX marked this pull request as ready for review June 18, 2026 12:00
@MarshalX
MarshalX requested a review from a team as a code owner June 18, 2026 12:00

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2980924755

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread internal/metrics/mem_metric.go Outdated
@MarshalX
MarshalX force-pushed the ilya.siamionau/perf-metrics branch from 2980924 to 9de0a28 Compare June 18, 2026 12:09
@datadog-official

Copy link
Copy Markdown

🎯 Code Coverage (details)
Patch Coverage: 5.13%
Overall Coverage: 47.99% (-0.34%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 9de0a28 | Docs | Datadog PR Page | Give us feedback!

@MikaYuoadas MikaYuoadas left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍
Just a question, IIUC we store the heap size at each phase boundaries but this means we’ll miss the memory peak inside each phase. Isn’t that what we care the most about to properly size our memory limit on pods and avoid OOMs?

@MarshalX

Copy link
Copy Markdown
Contributor Author

@MikaYuoadas thank you! You are right about miss memory peaks. The near term focus is IDE Integration with long-living HTTP server. We are gonna to use cache across runs. So we are interested in heap growth. But memory peak problem is worth solving with some periodic sampling to track the max seen. Could be tracked as follow-up when needed

@MarshalX
MarshalX merged commit 4277631 into main Jun 18, 2026
19 checks passed
@MarshalX
MarshalX deleted the ilya.siamionau/perf-metrics branch June 18, 2026 14:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants