Skip to content

[v2] Propagate allocators from context#420

Merged
Anilm3 merged 4 commits into
anilm3/v2from
anilm3/v2-propagate-allocators-from-context
Jul 9, 2025
Merged

[v2] Propagate allocators from context#420
Anilm3 merged 4 commits into
anilm3/v2from
anilm3/v2-propagate-allocators-from-context

Conversation

@Anilm3

@Anilm3 Anilm3 commented Jun 23, 2025

Copy link
Copy Markdown
Collaborator

This PR completes #419 by propagating the relevant allocator (memory resource) to all classes and methods which generate an output for the caller. The allocator remains unexposed through the interface as that will be handled in the next PR.

This PR also introduces nonnull_ptr, a zero-cost class holding a pointer which can't be nullptr, this is based on gsl::not_null.

Remaining work (Next PRs):

  • Update objects to use allocators more effectively.
  • Check for allocator compatibility where relevant.
  • Propagate allocator from the context.
  • Have a clear distinction between input , output and internal allocators.
  • Remove from_mutable_buffer use case on cow_string
  • Update interface:
    • Pass input / ouput allocators to context_init / context_eval.
    • Pass allocators to memory-allocating ddwaf_object functions.

@Anilm3 Anilm3 changed the title Propagate allocators from context [v2] Propagate allocators from context Jun 23, 2025
@codecov-commenter

codecov-commenter commented Jun 23, 2025

Copy link
Copy Markdown

Codecov Report

Attention: Patch coverage is 79.01639% with 64 lines in your changes missing coverage. Please review.

Project coverage is 85.53%. Comparing base (b5a5c61) to head (651b1ee).

Files with missing lines Patch % Lines
src/object.hpp 56.98% 2 Missing and 38 partials ⚠️
src/dynamic_string.hpp 88.63% 1 Missing and 4 partials ⚠️
src/processor/base.hpp 42.85% 0 Missing and 4 partials ⚠️
src/processor/extract_schema.cpp 83.33% 0 Missing and 3 partials ⚠️
src/attribute_collector.hpp 50.00% 0 Missing and 2 partials ⚠️
src/context.hpp 75.00% 0 Missing and 2 partials ⚠️
src/cow_string.hpp 89.47% 0 Missing and 2 partials ⚠️
src/serializer.cpp 94.28% 1 Missing and 1 partial ⚠️
src/interface.cpp 75.00% 1 Missing ⚠️
src/pointer.hpp 92.85% 0 Missing and 1 partial ⚠️
... and 2 more
Additional details and impacted files
@@              Coverage Diff              @@
##           anilm3/v2     #420      +/-   ##
=============================================
- Coverage      85.57%   85.53%   -0.05%     
=============================================
  Files            177      178       +1     
  Lines           9138     9179      +41     
  Branches        3940     3983      +43     
=============================================
+ Hits            7820     7851      +31     
+ Misses           497      491       -6     
- Partials         821      837      +16     
Flag Coverage Δ
waf_test 85.53% <79.01%> (-0.05%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@pr-commenter

pr-commenter Bot commented Jun 23, 2025

Copy link
Copy Markdown

Benchmarks clang

Benchmark execution time: 2025-07-09 11:54:17

Comparing candidate commit 651b1ee in PR branch anilm3/v2-propagate-allocators-from-context with baseline commit b5a5c61 in branch anilm3/v2.

Found 1 performance improvements and 0 performance regressions! Performance is the same for 0 metrics, 0 unstable metrics.

scenario:global-benchmark.random.clang

  • 🟩 execution_time [-42.086ms; -42.042ms] or [-15.089%; -15.074%]

@pr-commenter

pr-commenter Bot commented Jun 23, 2025

Copy link
Copy Markdown

Benchmarks gcc

Benchmark execution time: 2025-07-09 11:55:53

Comparing candidate commit 651b1ee in PR branch anilm3/v2-propagate-allocators-from-context with baseline commit b5a5c61 in branch anilm3/v2.

Found 1 performance improvements and 0 performance regressions! Performance is the same for 0 metrics, 0 unstable metrics.

scenario:global-benchmark.random.gcc

  • 🟩 execution_time [-10.096ms; -10.040ms] or [-3.660%; -3.640%]

@pr-commenter

pr-commenter Bot commented Jun 23, 2025

Copy link
Copy Markdown

Benchmarks clang-pgo

Benchmark execution time: 2025-07-09 12:10:10

Comparing candidate commit 651b1ee in PR branch anilm3/v2-propagate-allocators-from-context with baseline commit b5a5c61 in branch anilm3/v2.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 1 metrics, 0 unstable metrics.

@Anilm3
Anilm3 requested a review from Copilot June 24, 2025 08:38

This comment was marked as outdated.

@github-actions

github-actions Bot commented Jul 2, 2025

Copy link
Copy Markdown

Artifact Size Comparison 📦

Artifact Previous Release This PR Difference
darwin-arm64::libddwaf.a 83315344 86471432 0.03%
darwin-arm64::libddwaf.a.stripped 4352184 4573576 0.05%
darwin-arm64::libddwaf.dylib 1919536 1951552 0.01%
darwin-universal::libddwaf.a 167911160 174411648 0.03%
darwin-universal::libddwaf.a.stripped 9268592 9732712 0.05%
darwin-universal::libddwaf.dylib 4033072 4097856 0.01%
darwin-x86_64::libddwaf.a 84595768 87940168 0.03%
darwin-x86_64::libddwaf.a.stripped 4916360 5159088 0.04%
darwin-x86_64::libddwaf.dylib 2091048 2122816 0.01%
linux-aarch64::libddwaf.a 68876962 70794072 0.02%
linux-aarch64::libddwaf.a.stripped 11720766 11971652 0.02%
linux-aarch64::libddwaf.so 2429048 2448392 0.00%
linux-armv7::libddwaf.a 60213732 61777252 0.02%
linux-armv7::libddwaf.a.stripped 10612116 10827984 0.02%
linux-armv7::libddwaf.so 2103320 2103248 0.00%
linux-i386::libddwaf.a 58414994 59997814 0.02%
linux-i386::libddwaf.a.stripped 9009642 9215966 0.02%
linux-i386::libddwaf.so 2302460 2322388 0.00%
linux-x86_64::libddwaf.a 68977050 70896810 0.02%
linux-x86_64::libddwaf.a.stripped 11389686 11624110 0.02%
linux-x86_64::libddwaf.so 2602216 2616520 0.00%
windows-win32::ddwaf.dll 3292160 3357184 0.01%
windows-win32::ddwaf.lib 11684 11938 0.02%
windows-win32::ddwaf_static.lib 46934870 48609086 0.03%
windows-x64::ddwaf.dll 4021248 4068864 0.01%
windows-x64::ddwaf.lib 11464 11712 0.02%
windows-x64::ddwaf_static.lib 54410908 56102306 0.03%

@Anilm3
Anilm3 force-pushed the anilm3/v2-propagate-allocators-from-context branch from 8f48385 to d06b96b Compare July 3, 2025 08:31
@Anilm3
Anilm3 requested a review from Copilot July 3, 2025 13:06

This comment was marked as outdated.

@Anilm3
Anilm3 force-pushed the anilm3/v2-propagate-allocators-from-context branch from d06b96b to 7381270 Compare July 3, 2025 13:15
@Anilm3
Anilm3 force-pushed the anilm3/v2-propagate-allocators branch from dd9fed2 to 154cbfe Compare July 4, 2025 14:07
@Anilm3
Anilm3 force-pushed the anilm3/v2-propagate-allocators-from-context branch from 7381270 to ac33899 Compare July 4, 2025 14:13
@Anilm3
Anilm3 force-pushed the anilm3/v2-propagate-allocators-from-context branch 2 times, most recently from 8548fad to f63853b Compare July 8, 2025 13:43
Base automatically changed from anilm3/v2-propagate-allocators to anilm3/v2 July 8, 2025 13:44
@Anilm3
Anilm3 force-pushed the anilm3/v2-propagate-allocators-from-context branch from f63853b to 01232e3 Compare July 8, 2025 13:46
Comment thread src/attribute_collector.hpp Outdated
Comment thread src/dynamic_string.hpp Outdated
Comment thread src/dynamic_string.hpp
Comment thread src/dynamic_string.hpp Outdated
Comment thread src/interface.cpp
}

to_ref(object) = owned_object{string, length}.move();
to_ref(object) = owned_object{string, static_cast<uint32_t>(length)}.move();

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should bound check length. same in other points in this file.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't do it here because the function will be removed and the new one will use uint32_t rather than size_t.

@Anilm3
Anilm3 requested review from cataphract and Copilot July 8, 2025 21:40

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR propagates a user-specified memory resource (allocator) throughout all API calls that produce output objects, without exposing allocators in the public interface yet. It also introduces a zero-cost nonnull_ptr wrapper to guarantee non-null resource pointers.

  • Added nonnull_ptr<memory_resource> parameters to constructors and eval_impl methods across all processors and serializers
  • Updated result_serializer to store the allocator instance and made initialise_result_object non-static
  • Revised unit tests to pass the default allocator into every call path producing owned objects

Reviewed Changes

Copilot reviewed 40 out of 40 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
tests/unit/*.cpp Updated all tests to call eval/initialise_result_object with a default allocator
src/serializer.hpp/cpp result_serializer now holds an allocator; serialization routines use it when building maps and arrays
src/processor/*.hpp/cpp All eval_impl signatures now accept nonnull_ptr<memory_resource> before the deadline parameter
src/object.hpp / object_store code Cloning, builders, and destroy helpers all updated to carry allocators via nonnull_ptr
src/dynamic_string.hpp/cpp dynamic_string constructors and to_object take an allocator; reserve/resize APIs updated
src/cow_string.hpp replace_buffer and move now carry allocator throughnonnull_ptr
src/utf8.cpp Integrated allocator usage into normalize_string, guarding zero-length allocations
src/json_utils.hpp/cpp JSON parser handler now accepts allocator and passes it to object construction
src/utils.hpp Cleaned up unused includes and added literal case-insensitive comparison helpers
Comments suppressed due to low confidence (1)

tests/unit/object_test.cpp:930

  • [nitpick] The ObjectWithNullAllocator test has been commented out; consider re-enabling or replacing it to ensure allocator logic is fully exercised.
// TODO Reevaluate once allocators are exposed through the interface

Comment thread src/utf8.cpp
@Anilm3
Anilm3 merged commit dbadc60 into anilm3/v2 Jul 9, 2025
53 checks passed
@Anilm3
Anilm3 deleted the anilm3/v2-propagate-allocators-from-context branch July 9, 2025 12:54
Anilm3 added a commit that referenced this pull request Oct 1, 2025
* Object view: read only abstraction to ddwaf_object (#341)
* [v2] Remove mingw builds (#381)
* Writable objects: owned and borrowed object and object limits removal (#378, #382)
* [v2] Refactor and improve object types (#387)
* [v2] Update unit tests to use new abstractions (#389)
* [v2] Update `raw_configuration` type to use `object_view` (#390)
* [v2] Remove remaining uses of ddwaf_object in `src` and `tests/unit` (#391)
* [v2] JWT Decoding Processor (#401)
* [v2] First iteration of object layout changes (#394)
* Split context data insertion from evaluation (#407)
* [v2] Second iteration of object layout changes (#408)
* [v2] Add new fingerprint and object view tests (#414)
* Reenable attribute collector unit test (#415)
* [v2] Container view types (#413)
* Exclude assertions from coverage (#416)
* [v2] Use allocators internally instead of malloc/free and stop generating zero-terminated strings (#418)
* Add memory resource to owned and borrowed objects (#428)
* [v2] Propagate allocators from context (#420)
* [v2] Update interface and expose allocators (#427)
* Refactor evaluation stages out of the context (#442)
* Subcontext: replace ephemerals with a new scope with user-defined lifetime derived from the context (#443)
* Validator: Add support for testing subcontexts and attributes (#451)
* [v2] Pass allocator to context and subcontext eval and add new allocators (#452)
* Update logger to avoid dependencies on ddwaf.h (#453)
* [v2] Return DDWAF_MATCH when there are events, attributes or actions (#455)
* [v2] Cleanup: remove exclusion namespace and some redundant references (#456)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants