Skip to content

[v2] Pass allocator to context and subcontext eval and add new allocators#452

Merged
Anilm3 merged 13 commits into
anilm3/v2from
anilm3/eval-data-allocator
Sep 18, 2025
Merged

[v2] Pass allocator to context and subcontext eval and add new allocators#452
Anilm3 merged 13 commits into
anilm3/v2from
anilm3/eval-data-allocator

Conversation

@Anilm3

@Anilm3 Anilm3 commented Sep 13, 2025

Copy link
Copy Markdown
Collaborator

This PR adds the following:

  • A new set of allocators beyond the default one: monotonic, synchronized pool, unsynchronized pool and user allocator. The former 3 are existing C++ memory resources which have been backported in libcxx-compat. The latter is a wrapper for custom allocation/deallocation functions, this can be used to create other types of allocators, e.g. counting allocators.
  • Allocators can now be passed to ddwaf_context_eval and ddwaf_subcontext_eval so that the memory provided can be automatically reclaimed using the correct allocator. Additionally, nullptr can be provided to prevent memory from being reclaimed altogether.

Excluding any testing and fixes, this PR concludes the changes required for allocator support. The next PRs will include the following changes:

  • More tests for allocators and subcontexts, as well as any related fixes and improvements.
  • The removal of DDWAF_MATCH due to its confusing and outdated semantics.
  • Updating the logger interface to avoid having to include ddwaf.h inside C++ code (excluding interface.cpp).

@Anilm3
Anilm3 force-pushed the anilm3/eval-data-allocator branch from eaed9a9 to 1c4e8c1 Compare September 13, 2025 22:23
@Anilm3
Anilm3 force-pushed the anilm3/eval-data-allocator branch from 1c4e8c1 to f50f59c Compare September 13, 2025 22:29
@codecov-commenter

codecov-commenter commented Sep 13, 2025

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 48.83721% with 44 lines in your changes missing coverage. Please review.
✅ Project coverage is 84.84%. Comparing base (d01b784) to head (95f32cf).

Files with missing lines Patch % Lines
src/interface.cpp 33.33% 28 Missing and 10 partials ⚠️
src/object.hpp 25.00% 0 Missing and 3 partials ⚠️
src/user_resource.hpp 85.71% 1 Missing and 2 partials ⚠️
Additional details and impacted files
@@              Coverage Diff              @@
##           anilm3/v2     #452      +/-   ##
=============================================
- Coverage      84.94%   84.84%   -0.10%     
=============================================
  Files            189      190       +1     
  Lines           9715     9787      +72     
  Branches        4234     4262      +28     
=============================================
+ Hits            8252     8304      +52     
- Misses           607      613       +6     
- Partials         856      870      +14     
Flag Coverage Δ
waf_test 84.84% <48.83%> (-0.10%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@pr-commenter

pr-commenter Bot commented Sep 13, 2025

Copy link
Copy Markdown

Benchmarks clang

Benchmark execution time: 2025-09-18 18:59:58

Comparing candidate commit 95f32cf in PR branch anilm3/eval-data-allocator with baseline commit d01b784 in branch anilm3/v2.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 1 metrics, 0 unstable metrics.

@pr-commenter

pr-commenter Bot commented Sep 13, 2025

Copy link
Copy Markdown

Benchmarks gcc

Benchmark execution time: 2025-09-18 18:59:54

Comparing candidate commit 95f32cf in PR branch anilm3/eval-data-allocator with baseline commit d01b784 in branch anilm3/v2.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 1 metrics, 0 unstable metrics.

@github-actions

github-actions Bot commented Sep 13, 2025

Copy link
Copy Markdown

Artifact Size Comparison 📦

Artifact Previous Release This PR Difference
darwin-arm64::libddwaf.a 89560336 94536536 0.05%
darwin-arm64::libddwaf.a.stripped 4545616 4871648 0.07%
darwin-arm64::libddwaf.dylib 1960784 2033072 0.03%
darwin-universal::libddwaf.a 180532808 190705944 0.05%
darwin-universal::libddwaf.a.stripped 9690824 10379720 0.07%
darwin-universal::libddwaf.dylib 4139856 4277680 0.03%
darwin-x86_64::libddwaf.a 90972424 96169360 0.05%
darwin-x86_64::libddwaf.a.stripped 5145160 5508024 0.07%
darwin-x86_64::libddwaf.dylib 2147776 2222176 0.03%
linux-aarch64::libddwaf.a 72647358 75790348 0.04%
linux-aarch64::libddwaf.a.stripped 11778162 12257752 0.04%
linux-aarch64::libddwaf.so 2453368 2493368 0.01%
linux-armv7::libddwaf.a 64279684 66947352 0.04%
linux-armv7::libddwaf.a.stripped 10783356 11239600 0.04%
linux-armv7::libddwaf.so 2138996 2152608 0.00%
linux-i386::libddwaf.a 62427430 65055468 0.04%
linux-i386::libddwaf.a.stripped 9321362 9727024 0.04%
linux-i386::libddwaf.so 2382908 2415500 0.01%
linux-x86_64::libddwaf.a 73129494 76271778 0.04%
linux-x86_64::libddwaf.a.stripped 11599138 12052398 0.03%
linux-x86_64::libddwaf.so 2649632 2686864 0.01%
windows-arm64::ddwaf.dll 4769280 4903936 0.02%
windows-arm64::ddwaf.lib 11698 16410 0.40%
windows-arm64::ddwaf_static.lib 57431616 60214920 0.04%
windows-win32::ddwaf.dll 3356160 3466752 0.03%
windows-win32::ddwaf.lib 11922 16726 0.40%
windows-win32::ddwaf_static.lib 49113786 51798778 0.05%
windows-x64::ddwaf.dll 4088832 4186112 0.02%
windows-x64::ddwaf.lib 11698 16410 0.40%
windows-x64::ddwaf_static.lib 56948354 59733342 0.04%

@pr-commenter

pr-commenter Bot commented Sep 13, 2025

Copy link
Copy Markdown

Benchmarks clang-pgo

Benchmark execution time: 2025-09-18 12:51:03

Comparing candidate commit db21366 in PR branch anilm3/eval-data-allocator with baseline commit d6e707c in branch anilm3/v2.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 1 metrics, 0 unstable metrics.

@Anilm3
Anilm3 force-pushed the anilm3/eval-data-allocator branch from e926c7f to e1adffc Compare September 14, 2025 10:35
@Anilm3
Anilm3 force-pushed the anilm3/eval-data-allocator branch from d58f31b to 2bcc91b Compare September 14, 2025 16:59
@Anilm3
Anilm3 force-pushed the anilm3/eval-data-allocator branch from f6f45ef to 6752038 Compare September 14, 2025 19:23
@Anilm3
Anilm3 force-pushed the anilm3/eval-data-allocator branch from 88cd85b to f45b7ff Compare September 15, 2025 09:06
@Anilm3

Anilm3 commented Sep 15, 2025

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Keep it up!

About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you open a pull request for review, mark a draft as ready, or comment "@codex review". If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex fix this CI failure" or "@codex address that feedback".

@Anilm3
Anilm3 force-pushed the anilm3/eval-data-allocator branch 7 times, most recently from a0af9ef to 11d6e68 Compare September 16, 2025 20:48
@Anilm3
Anilm3 force-pushed the anilm3/eval-data-allocator branch from 11d6e68 to 6c4db5c Compare September 16, 2025 20:57
@Anilm3
Anilm3 force-pushed the anilm3/eval-data-allocator branch 7 times, most recently from 4e5340a to 9dfa4af Compare September 17, 2025 12:00
@Anilm3
Anilm3 force-pushed the anilm3/eval-data-allocator branch 5 times, most recently from b45b10a to db21366 Compare September 18, 2025 12:14
@Anilm3 Anilm3 changed the title Pass allocator to context and subcontext eval [v2] Pass allocator to context and subcontext eval Sep 18, 2025
@Anilm3
Anilm3 force-pushed the anilm3/eval-data-allocator branch 2 times, most recently from 522f2d3 to c528eb5 Compare September 18, 2025 18:06
@Anilm3
Anilm3 marked this pull request as ready for review September 18, 2025 18:09
@Anilm3
Anilm3 requested a review from a team as a code owner September 18, 2025 18:09

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Codex Review: Here are some suggestions.

Reply with @codex fix comments to fix any unresolved comments.

About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you open a pull request for review, mark a draft as ready, or comment "@codex review". If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex fix this CI failure" or "@codex address that feedback".

Comment thread src/interface.cpp Outdated
@Anilm3
Anilm3 force-pushed the anilm3/eval-data-allocator branch from c528eb5 to 5de1ecc Compare September 18, 2025 18:21
@Anilm3
Anilm3 force-pushed the anilm3/eval-data-allocator branch from b6610eb to 95f32cf Compare September 18, 2025 18:38
@Anilm3 Anilm3 changed the title [v2] Pass allocator to context and subcontext eval [v2] Pass allocator to context and subcontext eval and add new allocators Sep 18, 2025
@Anilm3
Anilm3 enabled auto-merge (squash) September 18, 2025 18:49
@Anilm3
Anilm3 disabled auto-merge September 18, 2025 19:07
@Anilm3
Anilm3 merged commit 977b7b0 into anilm3/v2 Sep 18, 2025
53 of 54 checks passed
@Anilm3
Anilm3 deleted the anilm3/eval-data-allocator branch September 18, 2025 19:07
Anilm3 added a commit that referenced this pull request Oct 1, 2025
* Object view: read only abstraction to ddwaf_object (#341)
* [v2] Remove mingw builds (#381)
* Writable objects: owned and borrowed object and object limits removal (#378, #382)
* [v2] Refactor and improve object types (#387)
* [v2] Update unit tests to use new abstractions (#389)
* [v2] Update `raw_configuration` type to use `object_view` (#390)
* [v2] Remove remaining uses of ddwaf_object in `src` and `tests/unit` (#391)
* [v2] JWT Decoding Processor (#401)
* [v2] First iteration of object layout changes (#394)
* Split context data insertion from evaluation (#407)
* [v2] Second iteration of object layout changes (#408)
* [v2] Add new fingerprint and object view tests (#414)
* Reenable attribute collector unit test (#415)
* [v2] Container view types (#413)
* Exclude assertions from coverage (#416)
* [v2] Use allocators internally instead of malloc/free and stop generating zero-terminated strings (#418)
* Add memory resource to owned and borrowed objects (#428)
* [v2] Propagate allocators from context (#420)
* [v2] Update interface and expose allocators (#427)
* Refactor evaluation stages out of the context (#442)
* Subcontext: replace ephemerals with a new scope with user-defined lifetime derived from the context (#443)
* Validator: Add support for testing subcontexts and attributes (#451)
* [v2] Pass allocator to context and subcontext eval and add new allocators (#452)
* Update logger to avoid dependencies on ddwaf.h (#453)
* [v2] Return DDWAF_MATCH when there are events, attributes or actions (#455)
* [v2] Cleanup: remove exclusion namespace and some redundant references (#456)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants