[v2] Update interface and expose allocators by Anilm3 · Pull Request #427 · DataDog/libddwaf

Anilm3 · 2025-07-04T13:41:03Z

This PR introduces the new object interface with the following changes:

Setters are now called ddwaf_object_set_* to make their purpose clear.
Container setters now require an initial capacity.
All memory-allocating functions now require passing the allocator, currently the only allocator available is provided through ddwaf_get_default_allocator(). User and internal allocators will be provided in the next PR.
ddwaf_object_is_* functions have been introduced to better assert for types.
The context now also requires an output allocator which will be used for allocation of output objects.
Container insertion is now done in reverse, i.e. the insertion function returns the next available slot. This applies to both maps and arrays.

Remaining work (Next PRs):

~~Update objects to use allocators more effectively.~~
~~Check for allocator compatibility where relevant.~~
~~Propagate allocator from the context.~~
~~Have a clear distinction between input , output and internal allocators.~~
~~Remove from_mutable_buffer use case on cow_string~~
Update interface:

~~Pass input / ouput allocators to context_init / context_eval.~~

~~Pass allocators to memory-allocating ddwaf_object functions.~~
Provide input allocator to the context
Provide functionality for a user allocator.
Expose internal memory resources (monotonic and unsynchronized).

pr-commenter · 2025-07-04T14:02:34Z

Benchmarks clang

Benchmark execution time: 2025-07-09 17:32:06

Comparing candidate commit 4ff9ed3 in PR branch anilm3/v2-allocators-on-interface with baseline commit dbadc60 in branch anilm3/v2.

Found 0 performance improvements and 1 performance regressions! Performance is the same for 0 metrics, 0 unstable metrics.

scenario:global-benchmark.random.clang

🟥 execution_time [+38.101ms; +38.114ms] or [+16.097%; +16.102%]

pr-commenter · 2025-07-04T14:02:44Z

Benchmarks gcc

Benchmark execution time: 2025-07-09 17:33:30

Comparing candidate commit 4ff9ed3 in PR branch anilm3/v2-allocators-on-interface with baseline commit dbadc60 in branch anilm3/v2.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 1 metrics, 0 unstable metrics.

pr-commenter · 2025-07-04T14:16:48Z

Benchmarks clang-pgo

Benchmark execution time: 2025-07-09 15:27:41

Comparing candidate commit c699c1d in PR branch anilm3/v2-allocators-on-interface with baseline commit dbadc60 in branch anilm3/v2.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 1 metrics, 0 unstable metrics.

codecov-commenter · 2025-07-04T14:41:09Z

Codecov Report

❌ Patch coverage is 51.76471% with 82 lines in your changes missing coverage. Please review.
✅ Project coverage is 85.18%. Comparing base (dbadc60) to head (4ff9ed3).

Files with missing lines	Patch %	Lines
src/interface.cpp	47.61%	47 Missing and 8 partials ⚠️
src/object.hpp	58.73%	1 Missing and 25 partials ⚠️
src/waf.hpp	50.00%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@              Coverage Diff              @@
##           anilm3/v2     #427      +/-   ##
=============================================
- Coverage      85.53%   85.18%   -0.36%     
=============================================
  Files            178      178              
  Lines           9179     9211      +32     
  Branches        3983     3991       +8     
=============================================
- Hits            7851     7846       -5     
- Misses           491      529      +38     
+ Partials         837      836       -1

Flag	Coverage Δ
waf_test	`85.18% <51.76%> (-0.36%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

github-actions · 2025-07-04T15:27:10Z

Artifact Size Comparison 📦

Artifact	Previous Release	This PR	Difference
darwin-arm64::libddwaf.a	83315344	86526256	0.03%
darwin-arm64::libddwaf.a.stripped	4352184	4605088	0.05%
darwin-arm64::libddwaf.dylib	1919536	1952592	0.01%
darwin-universal::libddwaf.a	167911160	174525360	0.03%
darwin-universal::libddwaf.a.stripped	9268592	9803184	0.05%
darwin-universal::libddwaf.dylib	4033072	4098896	0.01%
darwin-x86_64::libddwaf.a	84595768	87999056	0.04%
darwin-x86_64::libddwaf.a.stripped	4916360	5198048	0.05%
darwin-x86_64::libddwaf.dylib	2091048	2123864	0.01%
linux-aarch64::libddwaf.a	68876962	70886688	0.02%
linux-aarch64::libddwaf.a.stripped	11720766	12041204	0.02%
linux-aarch64::libddwaf.so	2429048	2450352	0.00%
linux-armv7::libddwaf.a	60213732	61853612	0.02%
linux-armv7::libddwaf.a.stripped	10612116	10885240	0.02%
linux-armv7::libddwaf.so	2103320	2104260	0.00%
linux-i386::libddwaf.a	58414994	60066336	0.02%
linux-i386::libddwaf.a.stripped	9009642	9267288	0.02%
linux-i386::libddwaf.so	2302460	2324344	0.00%
linux-x86_64::libddwaf.a	68977050	70980882	0.02%
linux-x86_64::libddwaf.a.stripped	11389686	11688662	0.02%
linux-x86_64::libddwaf.so	2602216	2618312	0.00%
windows-win32::ddwaf.dll	3292160	3358720	0.02%
windows-win32::ddwaf.lib	11684	13888	0.18%
windows-win32::ddwaf_static.lib	46934870	48703968	0.03%
windows-x64::ddwaf.dll	4021248	4071936	0.01%
windows-x64::ddwaf.lib	11464	13626	0.18%
windows-x64::ddwaf_static.lib	54410908	56195560	0.03%

Copilot

Pull Request Overview

This PR systematically updates the DDWAF interface to expose and propagate allocator usage across object creation, mutation, and destruction, and replaces legacy APIs with their allocator‐aware counterparts.

Swapped ddwaf_object_free (and map/array/string initializers) for ddwaf_object_destroy and ddwaf_object_set_*/ddwaf_object_insert_* calls requiring an allocator.
Updated all ddwaf_context_init calls to accept a ddwaf_allocator pointer.
Changed all direct field accessors and legacy getters (e.g., .type, ddwaf_object_length) to ddwaf_object_get_type, ddwaf_object_get_length, and ddwaf_object_get_size.

Reviewed Changes

Copilot reviewed 59 out of 60 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
validator/utils.cpp	Refactored YAML→`ddwaf_object` conversion to use allocator APIs.
validator/runner.cpp	Updated context init and result object destruction with allocators.
tools/waf_runner.cpp	Replaced map initializers and `free` calls with allocator-backed APIs.
tools/verify_ruleset.cpp	Swapped `ddwaf_object_free` for `ddwaf_object_destroy` using allocator.
tools/verify_rule.cpp	Rewrote `convertRuleToRuleset` to build maps/arrays with allocators.
tools/ip_match_benchmark.cpp	Updated object creation/destruction calls to use default allocator.
tools/infer_schema_bench.cpp	Replaced `free` calls with allocator-based `destroy`.
tools/infer_schema.cpp	Migrated map/string initializers and getters to allocator APIs.
tools/common/utils.cpp	Mirrored YAML→`ddwaf_object` refactor in common utils.
tests/…	Updated all tests to use `ddwaf_object_destroy`, allocator-based setters, and new getters.

Comments suppressed due to low confidence (1)

tools/verify_rule.cpp:20

[nitpick] The local pointer version shadows the earlier ddwaf_object version variable name. Consider renaming to version_obj or version_key for clarity.

    auto *version = ddwaf_object_insert_key(&root, STRL("version"), alloc);

* Object view: read only abstraction to ddwaf_object (#341) * [v2] Remove mingw builds (#381) * Writable objects: owned and borrowed object and object limits removal (#378, #382) * [v2] Refactor and improve object types (#387) * [v2] Update unit tests to use new abstractions (#389) * [v2] Update `raw_configuration` type to use `object_view` (#390) * [v2] Remove remaining uses of ddwaf_object in `src` and `tests/unit` (#391) * [v2] JWT Decoding Processor (#401) * [v2] First iteration of object layout changes (#394) * Split context data insertion from evaluation (#407) * [v2] Second iteration of object layout changes (#408) * [v2] Add new fingerprint and object view tests (#414) * Reenable attribute collector unit test (#415) * [v2] Container view types (#413) * Exclude assertions from coverage (#416) * [v2] Use allocators internally instead of malloc/free and stop generating zero-terminated strings (#418) * Add memory resource to owned and borrowed objects (#428) * [v2] Propagate allocators from context (#420) * [v2] Update interface and expose allocators (#427) * Refactor evaluation stages out of the context (#442) * Subcontext: replace ephemerals with a new scope with user-defined lifetime derived from the context (#443) * Validator: Add support for testing subcontexts and attributes (#451) * [v2] Pass allocator to context and subcontext eval and add new allocators (#452) * Update logger to avoid dependencies on ddwaf.h (#453) * [v2] Return DDWAF_MATCH when there are events, attributes or actions (#455) * [v2] Cleanup: remove exclusion namespace and some redundant references (#456)

Anilm3 force-pushed the anilm3/v2-propagate-allocators-from-context branch from 7381270 to ac33899 Compare July 4, 2025 14:13

Anilm3 force-pushed the anilm3/v2-allocators-on-interface branch 2 times, most recently from d2b10ed to cfea1e5 Compare July 4, 2025 14:31

Anilm3 force-pushed the anilm3/v2-allocators-on-interface branch 10 times, most recently from b5a7b69 to 906577d Compare July 7, 2025 17:30

Anilm3 force-pushed the anilm3/v2-propagate-allocators-from-context branch 2 times, most recently from f63853b to 01232e3 Compare July 8, 2025 13:46

Anilm3 force-pushed the anilm3/v2-allocators-on-interface branch from 08f1bbe to d4c4c4a Compare July 8, 2025 13:48

Base automatically changed from anilm3/v2-propagate-allocators-from-context to anilm3/v2 July 9, 2025 12:54

Anilm3 force-pushed the anilm3/v2-allocators-on-interface branch from 49b6371 to f12d0a6 Compare July 9, 2025 12:56

Anilm3 requested a review from Copilot July 9, 2025 13:16

Copilot AI reviewed Jul 9, 2025

View reviewed changes

Comment thread tools/waf_runner.cpp Outdated

Comment thread tools/waf_runner.cpp Outdated

Anilm3 force-pushed the anilm3/v2-allocators-on-interface branch from f12d0a6 to 9f960b4 Compare July 9, 2025 14:20

Update interface and expose allocators

c699c1d

Anilm3 force-pushed the anilm3/v2-allocators-on-interface branch from 9f960b4 to c699c1d Compare July 9, 2025 14:49

cataphract approved these changes Jul 9, 2025

View reviewed changes

Comment thread src/interface.cpp

Comment thread src/interface.cpp Outdated

Comment thread src/interface.cpp Outdated

Comment thread src/interface.cpp

Address review comments

4ff9ed3

Anilm3 marked this pull request as ready for review July 9, 2025 17:24

Anilm3 requested a review from a team as a code owner July 9, 2025 17:24

Anilm3 enabled auto-merge (squash) July 9, 2025 17:25

Anilm3 merged commit 33f7915 into anilm3/v2 Jul 9, 2025
52 of 53 checks passed

Anilm3 deleted the anilm3/v2-allocators-on-interface branch July 9, 2025 17:37

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[v2] Update interface and expose allocators#427

[v2] Update interface and expose allocators#427
Anilm3 merged 2 commits into
anilm3/v2from
anilm3/v2-allocators-on-interface

Anilm3 commented Jul 4, 2025 •

edited

Loading

Uh oh!

pr-commenter Bot commented Jul 4, 2025 •

edited

Loading

Uh oh!

pr-commenter Bot commented Jul 4, 2025 •

edited

Loading

Uh oh!

pr-commenter Bot commented Jul 4, 2025 •

edited

Loading

Uh oh!

codecov-commenter commented Jul 4, 2025 •

edited

Loading

Uh oh!

github-actions Bot commented Jul 4, 2025 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

Anilm3 commented Jul 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pr-commenter Bot commented Jul 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks clang

scenario:global-benchmark.random.clang

Uh oh!

pr-commenter Bot commented Jul 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks gcc

Uh oh!

pr-commenter Bot commented Jul 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks clang-pgo

Uh oh!

codecov-commenter commented Jul 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

github-actions Bot commented Jul 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Artifact Size Comparison 📦

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Anilm3 commented Jul 4, 2025 •

edited

Loading

pr-commenter Bot commented Jul 4, 2025 •

edited

Loading

pr-commenter Bot commented Jul 4, 2025 •

edited

Loading

pr-commenter Bot commented Jul 4, 2025 •

edited

Loading

codecov-commenter commented Jul 4, 2025 •

edited

Loading

github-actions Bot commented Jul 4, 2025 •

edited

Loading