[K9CODESEC-2398] Add RunCustomRegoQuery and ValidateCustomRegoQuery#202
Conversation
|
🎯 Code Coverage (details) 🔗 Commit SHA: 3895b1a | Docs | Datadog PR Page | Give us feedback! |
1f4c142 to
4f6c863
Compare
4f6c863 to
879905c
Compare
| Platform: []string{platform}, | ||
| ChangedDefaultQueryPath: true, | ||
| MaxFileSizeFlag: 5, | ||
| QueryExecTimeout: 10, // shorter than the CLI default; custom rules should be fast |
There was a problem hiding this comment.
Question: Is it like the timeout discussed yesterday? The one that made the scanner non-deterministic?
There was a problem hiding this comment.
indeed hahaha, i did this PR a few days ago but took time to test it with the UI (new release => binary => dd-source => deploy), which obviously I forgot about it since the discussion around the timeout happened yesterday afternoon. That's a good comment, thank you Jeff
| return "scan-target.tf" | ||
| case "cloudformation", platformARM: | ||
| return scanTargetJSON | ||
| case "kubernetes", "ansible": |
There was a problem hiding this comment.
must-fix: using 'kubernetes"as platform, the k8s.rego library won't be loaded as it won't be found. Using the k8s platform, the target becomes JSON as described below. More on this line 840.
There was a problem hiding this comment.
Yes i noticed this before you added comments, that's part of why I force-pushed ahha. Sorry for that, now I am deriving it from libraryPlatform
| } | ||
|
|
||
| func libraryFilesystemSource(ctx context.Context, platform string) *source.FilesystemSource { | ||
| return source.NewFilesystemSource(ctx, []string{"."}, []string{platform}, []string{""}, source.LibrariesDefaultBasePath, false) |
There was a problem hiding this comment.
The platform could be normalized here using a map or something so that platform can be kubernetes and links to k8s
There was a problem hiding this comment.
yes using libraryPlatform now
| if !ok || varName == resultVar { | ||
| // "result is unsafe" is always a cascade of other unsafe vars; skip it. | ||
| continue |
There was a problem hiding this comment.
Is it not still skipping? It is expected behaviour?
879905c to
4dd260f
Compare
00139b9 to
3895b1a
Compare
Motivation
Custom rule authors writing Rego policies need to know immediately whether their code is valid, not after a failed deployment. This PR adds two new exported functions,
ValidateCustomRegoQueryandRunCustomRegoQuery, that give callers IDE-quality diagnostics and evaluation results for custom Rego rules without running a full scan.Related Ticket: K9CODESEC-2398
Changes
Validation pipeline
ValidateCustomRegoQueryruns three phases in a single call and returns all errors simultaneously, so authors see the full picture at once rather than fixing one error at a time:.documentreferences (missinginputprefix), missing package declarations, missing commas inside objects, and unbalanced parentheses each get a precise line/column location and a human-readable explanation instead of OPA's internal error text.datadog, that aDatadogPolicyrule is present, that all required result fields (documentId,resourceType,resourceName,searchKey) are assigned, thatsprintfcall matches its format string, and that any aliased library has a corresponding import statement.If the module is missing its
packagedeclaration, validation recovers by temporarily prependingpackage datadog, runs the static checks on the recovered AST, and shifts error line numbers back so locations still point at the user's original code.Evaluation
RunCustomRegoQuerywrites the Rego content and the IaC file to temporary files with the correct platform-specific extension (.tf,.yaml,.json, …), runs the scan engine in-memory, and returns findings with precise resource locations alongside any runtime errors.CLI
Two new subcommands are registered under
custom:datadog-iac-scanner custom validate --platform <platform> --rego <base64>— outputs JSON{"errors": [...]}.datadog-iac-scanner custom evaluate --platform <platform> --rego <base64> --file <base64>— runs validation first; if errors are found they are returned without scanning. Otherwise outputs JSON{"findings": [...], "errors": [...]}.Both commands accept base64-encoded inputs so binary-safe content can be passed from any calling process without shell escaping concerns.
Note
I have a branch with an UI and a server, let me know if you need it to better test.
Author Checklist
QA Instruction
go test ./pkg/scan/ -run TestValidateCustomRegoQuery -v— all scenarios (missing package, wrong package, missing rule, missing result fields, missing imports, missing commas, unbalanced parens, missinginput, undefined variables) should pass.go build ./cmd/scanner/...resourceTypefrom the result object) and re-run validate — expect amissing_result_fielderror with the correct line number.Blast Radius
This PR only adds new exported functions and CLI subcommands. No existing scan logic, query loading, or result reporting paths are modified. The only change to existing files is wiring the new
customcommand into the CLI entry point and a minor fix to pass the IaC file path through the scan client.Additional Notes
I submit this contribution under the Apache-2.0 license.