Skip to content

Optimize privilege-escalation rules by pre-indexing IAM permission sets#221

Merged
whitemerch merged 1 commit into
mainfrom
chakib.hamie/fix_privilege_escalation_rules_perf
Jun 30, 2026
Merged

Optimize privilege-escalation rules by pre-indexing IAM permission sets#221
whitemerch merged 1 commit into
mainfrom
chakib.hamie/fix_privilege_escalation_rules_perf

Conversation

@whitemerch

@whitemerch whitemerch commented Jun 29, 2026

Copy link
Copy Markdown
Contributor

Motivation

The *_unrecommended_permission_policy_scenarios helpers in assets/libraries/common.rego were performing a full input.document[_] scan for every principal (role, user, group) they were called on. With Terraform module instantiation in place, input.document can contain many synthetic module documents, making each helper call O(principals × docs × policies). Because ~64 near-identical privilege-escalation rules call these helpers independently, the total cost was O(64 × principals × docs × policies) per scan, the primary contributor to multi-minute scan times on module-heavy repositories.

Benchmarks on a simulated module-inflated input (3,000 roles, 750 policies across 50 documents) measured 3,216 ms → 45 ms per rule (~71× speedup).

Changes

Rule library (assets/libraries/common.rego)

Three helper functions (role_unrecommended_permission_policy_scenarios, user_unrecommended_permission_policy_scenarios, group_unrecommended_permission_policy_scenarios) are replaced with an indexed design.

Two new helpers support the index:

_to_action_array normalises an Action field (string or array) to an array for uniform iteration.

_policy_wildcard_actions returns the set of lowercased Allow-on-"*" actions for a given policy resource object. OPA memoizes complete-rule calls per unique argument, so json_unmarshal is paid at most once per unique policy within a query.

Three complete set rules (_role_dangerous_pairs, _user_dangerous_pairs, _group_dangerous_pairs) pre-compute [principalName, permission] pairs across both scenarios (inline policies and policy attachments) for all principals at once. OPA evaluates and caches complete rules on first access; all subsequent lookups within the same query are O(1) set membership checks. This is the key difference from an incremental/partial-rule approach, which OPA specializes per bound key and does not cache across callers.

Author Checklist

  • I have reviewed my own PR.
  • I have added or updated relevant unit tests where necessary. If no tests are added, I've explained why.
  • All new and existing tests pass.
  • I have tested my changes on staging (if applicable).
  • I have updated any relevant documentation (if applicable).

QA Instruction

  1. opa test assets/libraries/ — all 324 existing tests must pass (no new tests are needed since the function signatures are unchanged and existing tests cover both scenarios).
  2. go test ./pkg/engine/... -timeout 120s — engine tests must pass.
  3. Optionally benchmark one slow rule before and after:
opa bench \
  -d assets/libraries/common.rego \
  -d assets/libraries/terraform.rego \
  -d <path-to-role_with_privilege_escalation_by_actions_iam_AttachRolePolicy/query.rego> \
  --input <module-heavy-input.json> \
  'data.datadog.DatadogPolicy'

Verify findings count is identical between the patched and unpatched library.

Blast Radius

DataDog IaC Scanner only. The change is limited to three helper functions in assets/libraries/common.rego.

Additional Notes

The ~64 role_with_privilege_escalation_by_actions_*, user_with_privilege_escalation_by_actions_*, and group_with_privilege_escalation_by_actions_* rules in the default rule corpus all benefit from this change without modification.

I submit this contribution under the Apache-2.0 license.

@datadog-prod-us1-5

datadog-prod-us1-5 Bot commented Jun 29, 2026

Copy link
Copy Markdown

🎯 Code Coverage (details)
Patch Coverage: 100.00%
Overall Coverage: 49.95% (+0.00%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 9570712 | Docs | Datadog PR Page | Give us feedback!

@whitemerch
whitemerch force-pushed the chakib.hamie/fix_privilege_escalation_rules_perf branch from 12f61c8 to 05e21d1 Compare June 29, 2026 22:28
@whitemerch
whitemerch marked this pull request as ready for review June 29, 2026 22:32
@whitemerch
whitemerch requested a review from a team as a code owner June 29, 2026 22:32
…rules share one document scan per query instead of repeating it per principal.
@whitemerch
whitemerch force-pushed the chakib.hamie/fix_privilege_escalation_rules_perf branch from 05e21d1 to 9570712 Compare June 30, 2026 08:07

@ChouraquiBen ChouraquiBen left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM


policies := {"aws_iam_role_policy", "aws_iam_user_policy", "aws_iam_group_policy", "aws_iam_policy"}
resourcePolicy := input.document[_].resource[policies[_]][policy]
_to_action_array(x) := x if is_array(x)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Question: That will make queries fail if x is neither an array or a string, is that ok or should there be a default case?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good question, in practice it's fine as-is. IAM Action is only ever a string or array per the AWS policy schema, and if it's something else (e.g. null in a malformed policy), _to_action_array is undefined and that statement is simply skipped in the comprehension, same outcome as having no actions. No query failure, no false positives. Happy to add else := [] if you prefer making that explicit.

@whitemerch
whitemerch merged commit 840ea1d into main Jun 30, 2026
21 checks passed
@whitemerch
whitemerch deleted the chakib.hamie/fix_privilege_escalation_rules_perf branch June 30, 2026 08:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants