Skip to content

[v2] Writable objects: owned and borrowed object#378

Merged
Anilm3 merged 129 commits into
anilm3/v2from
anilm3/writable_objects
Mar 20, 2025
Merged

[v2] Writable objects: owned and borrowed object#378
Anilm3 merged 129 commits into
anilm3/v2from
anilm3/writable_objects

Conversation

@Anilm3

@Anilm3 Anilm3 commented Mar 9, 2025

Copy link
Copy Markdown
Collaborator

This PR introduces two new object types:

  • owned_object: as the name suggests, this is an object which owns its own memory. This abstraction ensures that the memory associated with an object is freed on destruction and moved appropriately.
  • borrowed_object: an object type which points to memory owned by a different object, this allows accessing elements from an owned_object without the risk of accidental double-free.

In addition, some code from object_view has been refactored to ensure that owned_object and borrowed_object are both writable and readable. More of object_view will be refactored in the next few PRs.

Some quirks on this PR:

  • The public functions ddwaf_object_* still contain a separate implementation, these will be updated to use the new abstractions in a following PR. This code has been moved from object.cpp to interface.cpp without changes.
  • owned_object and borrowed_object currently don't check for free function compatibility. While incorrect, this shouldn't be an issue at the moment and free functions are being replaced by allocators in a subsequent PR.

Remaining work (future PRs):

  • Replace all uses of ddwaf_object within tests with owned_object, borrowed_object and object_view.
  • Completely refactor object_view and object_converter.
  • Remove raw_configuration in favour of object_view or use object_view behind the scenes.
  • Update ddwaf_object_* interface to use new types.
  • Improve context method to avoid ddwaf_object arguments or optional_ref.
  • Add allocator support and check for allocator compatibility on emplace and emplace_back.
  • Remove all uses of malloc / free

@Anilm3 Anilm3 changed the title [WIP] Writable objects: owned and borrowed object Writable objects: owned and borrowed object Mar 16, 2025
@Anilm3
Anilm3 marked this pull request as ready for review March 16, 2025 15:55
@Anilm3
Anilm3 requested a review from a team as a code owner March 16, 2025 15:55
@Anilm3 Anilm3 changed the title Writable objects: owned and borrowed object [v2] Writable objects: owned and borrowed object Mar 16, 2025
@DataDog DataDog deleted a comment from pr-commenter Bot Mar 16, 2025
ddwaf_object_array_add(&array, ddwaf_object_stringl(&tmp, arg.data(), arg.size()));
}
auto root = owned_object::make_map();
root.emplace("server.request.query", owned_object::make_string(param));

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

perhaps you could add some syntax sugar like:

#include <string_view>

struct kv;
struct dw_obj {
    char _c{};

    dw_obj(dw_obj &&) = default;
    dw_obj(const dw_obj &) = delete;
    dw_obj &operator=(dw_obj &&) = default;
    dw_obj &operator=(const dw_obj &) = delete;
    ~dw_obj() = default;

    dw_obj(int i) {} // NOLINT
    dw_obj(std::string_view) {} // NOLINT
    template<typename Str> requires std::convertible_to<Str, std::string_view>
    dw_obj(Str) {} // NOLINT
    dw_obj(std::initializer_list<kv>) {} // NOLINT
    dw_obj(std::initializer_list<dw_obj>) {} // NOLINT

    protected:
    dw_obj() = default;
};
struct kv : std::pair<std::string_view, dw_obj> {
    using std::pair<std::string_view, dw_obj>::pair;
    kv(const std::string_view& sv, dw_obj&& obj) : std::pair<std::string_view, dw_obj>{
        sv, std::move(obj)
    } {} // NOLINT
};

void test() {
    dw_obj map{
        kv{"foo", {kv{"bar", "foo"}}},
        kv{
            "xxx",
            {
                1,
                3,
                "yyy", 
                {{"zzz"}},
                {kv{"aaa", "bbb"}}}}};
}

compiler explorer

Empty arrays/maps need special handling; you could add constants, or you could add dw_obj_array, dw_obj_map objects etc. You could use a similar model to that of nginx: they derive dw_obj, they have may_alias, and you can (behind an optional for safe conversions) convert borrowed dw_obj into dw_obj_map/etc. and unconditionally convert in the other direction.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay I had some thoughts along these lines as well, I'll have a look at what you've done and see if I can introduce anything here as well in the next PR (before I update all tests....)

Comment thread src/object.hpp Outdated
Comment thread src/object.hpp Outdated
Comment thread src/object.hpp Outdated
Comment thread src/object.hpp Outdated
Comment thread src/object.hpp
Comment thread src/object.hpp
Comment thread src/object.hpp
Comment thread src/object_view.hpp
Comment thread src/scanner.hpp Outdated
@Anilm3
Anilm3 requested a review from cataphract March 17, 2025 17:23
@pr-commenter

pr-commenter Bot commented Mar 19, 2025

Copy link
Copy Markdown

Benchmarks clang-pgo

Benchmark execution time: 2025-03-19 12:12:32

Comparing candidate commit c6d9fbf in PR branch anilm3/writable_objects with baseline commit f3f23ee in branch anilm3/v2.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 1 metrics, 0 unstable metrics.

Comment thread src/condition/scalar_condition.cpp Outdated
Comment thread src/condition/scalar_condition.cpp
Comment thread src/configuration/common/raw_configuration.hpp Outdated
Comment thread src/configuration/common/raw_configuration.hpp Outdated
Comment thread src/interface.cpp
Comment thread src/interface.cpp
Comment thread src/iterator.hpp
Comment thread src/object_view.hpp
@Anilm3
Anilm3 merged commit 6f1e0d9 into anilm3/v2 Mar 20, 2025
@Anilm3
Anilm3 deleted the anilm3/writable_objects branch March 20, 2025 16:39
Anilm3 added a commit that referenced this pull request Oct 1, 2025
* Object view: read only abstraction to ddwaf_object (#341)
* [v2] Remove mingw builds (#381)
* Writable objects: owned and borrowed object and object limits removal (#378, #382)
* [v2] Refactor and improve object types (#387)
* [v2] Update unit tests to use new abstractions (#389)
* [v2] Update `raw_configuration` type to use `object_view` (#390)
* [v2] Remove remaining uses of ddwaf_object in `src` and `tests/unit` (#391)
* [v2] JWT Decoding Processor (#401)
* [v2] First iteration of object layout changes (#394)
* Split context data insertion from evaluation (#407)
* [v2] Second iteration of object layout changes (#408)
* [v2] Add new fingerprint and object view tests (#414)
* Reenable attribute collector unit test (#415)
* [v2] Container view types (#413)
* Exclude assertions from coverage (#416)
* [v2] Use allocators internally instead of malloc/free and stop generating zero-terminated strings (#418)
* Add memory resource to owned and borrowed objects (#428)
* [v2] Propagate allocators from context (#420)
* [v2] Update interface and expose allocators (#427)
* Refactor evaluation stages out of the context (#442)
* Subcontext: replace ephemerals with a new scope with user-defined lifetime derived from the context (#443)
* Validator: Add support for testing subcontexts and attributes (#451)
* [v2] Pass allocator to context and subcontext eval and add new allocators (#452)
* Update logger to avoid dependencies on ddwaf.h (#453)
* [v2] Return DDWAF_MATCH when there are events, attributes or actions (#455)
* [v2] Cleanup: remove exclusion namespace and some redundant references (#456)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants