Skip to content

Introduce ddwaf_(context|subcontext)_multieval to evaluate multiple batches in sequence#494

Merged
Anilm3 merged 4 commits into
masterfrom
anilm3/multieval
Jun 12, 2026
Merged

Introduce ddwaf_(context|subcontext)_multieval to evaluate multiple batches in sequence#494
Anilm3 merged 4 commits into
masterfrom
anilm3/multieval

Conversation

@Anilm3

@Anilm3 Anilm3 commented Jun 9, 2026

Copy link
Copy Markdown
Collaborator

Overview

This PR adds two new public entrypoints, ddwaf_context_multieval and ddwaf_subcontext_multieval, that evaluate multiple input batches in sequence within a single call and return one combined result. Functionally each entrypoint behaves like its _eval counterpart, except that data is an array of maps where every element is treated as a separate input batch and evaluated in order.

To support this, insertions to the object store now result in the batch being added to a queue, which then has to be explicitly applied before it's ready to be evaluated:

  • insert_batch / insert_batches enqueue input (a single map, or an array of maps) without applying it.
  • next_batch pops and applies the next queued batch, marking its targets as new.
  • flush_input_queue drains any batches left unevaluated (e.g. after a timeout) on every exit path, applying them as "existing" targets targets so they carry over to subsequent calls and resets the new-target set for the next evaluation.

The evaluation engine now loops over batches, running pre-processors, filters, rules, and post-processors per batch, accumulating events/actions/attributes into the single result.

A new field has been added to the result object:

  • evaluated: an unsigned integer reporting how many batches were fully evaluated. In the normal case this equals the number of non-empty batches; on timeout/error during batch i (0-based, counting non-empty batches) it equals I, i.e. the index of the batch where the problem occurred. Empty batches are skipped and don't count.

This field is present on the regular _eval results too (the result schema is shared), where it's simply 1 for a non-empty evaluated batch or 0 otherwise.

Files to review (src/)

  • src/object_store.hpp / src/object_store.cpp: core change: the batch queue, insert_batch/insert_batches/insert_target/next_batch/flush_input_queue/enqueue_batch/apply_batch, and the insert_and_apply test helper. Worth the closest look for the queue lifetime and new-target semantics.
  • src/evaluation_engine.cpp: the per-batch evaluation loop, flush_input_queue on exit, and the evaluated counter.
  • src/evaluation_engine.hpp: insert_batch/insert_batches forwarding and the next_batch/insert_and_apply test internals.
  • src/interface.cpp: the two new public functions plus the array-vs-map dispatch added to ddwaf_context_eval / ddwaf_subcontext_eval.
  • src/context.hpp: context/subcontext forwarding for the new store methods and test internals.
  • src/serializer.hpp / src/serializer.cpp: the evaluated field added to result_components and initialise_result_object.
  • src/processor/base.hpp: processor output switched from insert to insert_target.

@Anilm3
Anilm3 requested a review from a team as a code owner June 9, 2026 19:02
@datadog-datadog-prod-us1-2

datadog-datadog-prod-us1-2 Bot commented Jun 9, 2026

Copy link
Copy Markdown

🎯 Code Coverage (details)
Patch Coverage: 79.39%
Overall Coverage: 84.79% (-0.14%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 93d87db | Docs | Datadog PR Page | Give us feedback!

@codecov-commenter

codecov-commenter commented Jun 9, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 79.38596% with 47 lines in your changes missing coverage. Please review.
✅ Project coverage is 84.78%. Comparing base (e71d0da) to head (93d87db).

Files with missing lines Patch % Lines
src/interface.cpp 64.00% 15 Missing and 3 partials ⚠️
src/input_batch_queue.hpp 81.57% 0 Missing and 7 partials ⚠️
src/context.hpp 82.75% 5 Missing ⚠️
src/processor/base.hpp 16.66% 0 Missing and 5 partials ⚠️
src/evaluation_engine.hpp 76.47% 2 Missing and 2 partials ⚠️
src/object_store.hpp 80.00% 0 Missing and 3 partials ⚠️
src/attribute_collector.cpp 75.00% 1 Missing and 1 partial ⚠️
src/evaluation_engine.cpp 95.34% 0 Missing and 2 partials ⚠️
src/attribute_collector.hpp 85.71% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #494      +/-   ##
==========================================
+ Coverage   84.71%   84.78%   +0.07%     
==========================================
  Files         190      191       +1     
  Lines        9670     9341     -329     
  Branches     4186     4184       -2     
==========================================
- Hits         8192     7920     -272     
+ Misses        577      529      -48     
+ Partials      901      892       -9     
Flag Coverage Δ
waf_test 84.78% <79.38%> (+0.07%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@github-actions

github-actions Bot commented Jun 9, 2026

Copy link
Copy Markdown

Dynamic Artifact Size Comparison 📦

Artifact Previous Release This PR Difference
darwin-arm64::libddwaf.dylib 2004032 2020576 0.82%
darwin-universal::libddwaf.dylib 4215872 4248800 0.78%
darwin-x86_64::libddwaf.dylib 2195712 2208040 0.56%
linux-aarch64::libddwaf.so 2378224 2388576 0.43%
linux-armv7::libddwaf.so 2046280 2056416 0.49%
linux-i386::libddwaf.so 2300828 2312916 0.52%
linux-x86_64::libddwaf.so 2571120 2581392 0.39%
windows-arm64::ddwaf.dll 6686208 6731264 0.67%
windows-win32::ddwaf.dll 3315712 3330560 0.44%
windows-x64::ddwaf.dll 4045824 4061184 0.37%

Static Artifact Size Comparison 📦

Artifact Previous Release This PR Difference
darwin-arm64::libddwaf.a 92215128 92531280 0.34%
darwin-arm64::libddwaf.a.stripped 4707464 4715960 0.18%
darwin-universal::libddwaf.a 185945888 186577952 0.33%
darwin-universal::libddwaf.a.stripped 10023864 10041984 0.18%
darwin-x86_64::libddwaf.a 93730712 94046624 0.33%
darwin-x86_64::libddwaf.a.stripped 5316352 5325976 0.18%
linux-aarch64::libddwaf.a 75161592 75455044 0.39%
linux-aarch64::libddwaf.a.stripped 12162860 12184616 0.17%
linux-armv7::libddwaf.a 66399138 66674290 0.41%
linux-armv7::libddwaf.a.stripped 11168358 11189614 0.19%
linux-i386::libddwaf.a 64591428 64858588 0.41%
linux-i386::libddwaf.a.stripped 9661240 9676972 0.16%
linux-x86_64::libddwaf.a 75652672 75942550 0.38%
linux-x86_64::libddwaf.a.stripped 11966124 11985474 0.16%
windows-arm64::ddwaf.lib 16410 16902 2.99%
windows-arm64::ddwaf_static.lib 134528158 135254230 0.53%
windows-win32::ddwaf.lib 16726 17228 3.00%
windows-win32::ddwaf_static.lib 47564214 47779918 0.45%
windows-x64::ddwaf.lib 16410 16902 2.99%
windows-x64::ddwaf_static.lib 55369970 55616586 0.44%

@Anilm3
Anilm3 force-pushed the anilm3/multieval branch 4 times, most recently from df4a13d to 47ed904 Compare June 10, 2026 08:58
@DataDog DataDog deleted a comment from chatgpt-codex-connector Bot Jun 10, 2026
@Anilm3

Anilm3 commented Jun 10, 2026

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 47ed904d18

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread src/object_store.cpp Outdated
@Anilm3
Anilm3 force-pushed the anilm3/multieval branch from 47ed904 to 41ad8b7 Compare June 10, 2026 10:27

@christophe-papazian christophe-papazian left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The serializer now always writes evaluated into the result object, but schema/result.json still has additionalProperties: false and doesn't list it. Worth updating the schema, or will this silently break consumers doing strict validation?

Comment thread src/evaluation_engine.cpp
Comment thread tests/integration/interface/context/multieval/test.cpp Outdated
Comment thread tests/integration/interface/context/multieval/test.cpp Outdated
Comment thread tests/unit/evaluation_engine_test.cpp Outdated
Comment thread src/evaluation_engine.cpp Outdated
Comment thread src/object_store.hpp Outdated
Comment thread src/evaluation_engine.cpp Outdated
Comment thread src/evaluation_engine.cpp Outdated
Comment thread src/object_store.cpp Outdated
@Anilm3

Anilm3 commented Jun 11, 2026

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 9b1dbc72cf

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread src/evaluation_engine.cpp Outdated
Comment thread src/evaluation_engine.cpp
@Anilm3
Anilm3 force-pushed the anilm3/multieval branch from 9b1dbc7 to 682d543 Compare June 11, 2026 21:17
@Anilm3
Anilm3 enabled auto-merge (squash) June 12, 2026 11:04
@Anilm3
Anilm3 force-pushed the anilm3/multieval branch from ba03471 to 93d87db Compare June 12, 2026 11:43
@Anilm3
Anilm3 merged commit 9d00e49 into master Jun 12, 2026
56 of 57 checks passed
@Anilm3
Anilm3 deleted the anilm3/multieval branch June 12, 2026 12:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants