Skip to content

Support multiple subcontexts per context#468

Merged
Anilm3 merged 10 commits into
masterfrom
anilm3/concurrent-subcontext
Oct 22, 2025
Merged

Support multiple subcontexts per context#468
Anilm3 merged 10 commits into
masterfrom
anilm3/concurrent-subcontext

Conversation

@Anilm3

@Anilm3 Anilm3 commented Oct 10, 2025 •
edited
Loading

Copy link
Copy Markdown
Collaborator

This PR changes the way subcontexts are used and created, they are now a "copy" of the context, allowing the creation and use of many subcontexts at the same time, although not in a thread-safe manner without user-provided synchronisation. This iteration has some caveats:

  • The context still owns the data provided to it, so the subcontext just copies the targets but points to the context data.
  • The cache is fully copied, but in this iteration:
    • Matches performed with context data are not written back to the context cache.
    • Unevaluated primitives may be evaluated within the subcontext even if their outcome relates to the context.

All of the logic to distinguish contexts and subcontexts internally has been removed for simplification, albeit some of it may come back in the future to address the caveats above.

Remaining work (to be addressed in a future PR):

  • Evaluation within the context must be forbidden if a subcontext exists to avoid inconsistencies, this hasn't been addressed here.
  • If a context is destroyed while there is a subcontext alive, the user-provided data is invalidated, this hasn't been addressed either.
  • There are many unit tests for contexts and subcontexts which are no longer relevant, these have to be removed.
  • There are no new tests to validate the "concurrency-without-thread-safety" aspect as the PR is already too large.

Related Jiras: APPSEC-59671

@pr-commenter

pr-commenter Bot commented Oct 10, 2025 •
edited
Loading

Copy link
Copy Markdown

Benchmarks gcc

Benchmark execution time: 2025-10-21 15:21:26

Comparing candidate commit 15be03e in PR branch anilm3/concurrent-subcontext with baseline commit 4d91b6b in branch master.

Found 0 performance improvements and 1 performance regressions! Performance is the same for 0 metrics, 0 unstable metrics.

scenario:global-benchmark.random.gcc

  • 🟥 execution_time [+16.006ms; +16.016ms] or [+6.230%; +6.234%]

@pr-commenter

pr-commenter Bot commented Oct 10, 2025 •
edited
Loading

Copy link
Copy Markdown

Benchmarks clang

Benchmark execution time: 2025-10-21 15:21:00

Comparing candidate commit 15be03e in PR branch anilm3/concurrent-subcontext with baseline commit 4d91b6b in branch master.

Found 1 performance improvements and 0 performance regressions! Performance is the same for 0 metrics, 0 unstable metrics.

scenario:global-benchmark.random.clang

  • 🟩 execution_time [-55.728ms; -55.715ms] or [-18.851%; -18.847%]

@pr-commenter

pr-commenter Bot commented Oct 10, 2025 •
edited
Loading

Copy link
Copy Markdown

Benchmarks clang-pgo

Benchmark execution time: 2025-10-21 15:35:51

Comparing candidate commit 15be03e in PR branch anilm3/concurrent-subcontext with baseline commit 4d91b6b in branch master.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 1 metrics, 0 unstable metrics.

@Anilm3 Anilm3 changed the title Support multiple subcontexts per-context Support multiple subcontexts per context Oct 13, 2025
@Anilm3
Anilm3 force-pushed the anilm3/concurrent-subcontext branch from 9d7e8d7 to a53b45c Compare October 13, 2025 19:34
@codecov-commenter

codecov-commenter commented Oct 13, 2025 •
edited
Loading

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 88.37209% with 30 lines in your changes missing coverage. Please review.
✅ Project coverage is 84.97%. Comparing base (4d91b6b) to head (15be03e).
⚠️ Report is 19 commits behind head on master.

Files with missing lines Patch % Lines
src/evaluation_engine.hpp 63.15% 0 Missing and 7 partials ⚠️
src/context.hpp 88.57% 1 Missing and 3 partials ⚠️
src/processor/base.hpp 69.23% 1 Missing and 3 partials ⚠️
src/attribute_collector.cpp 66.66% 1 Missing and 2 partials ⚠️
src/evaluation_engine.cpp 87.50% 0 Missing and 3 partials ⚠️
src/object_store.hpp 81.25% 0 Missing and 3 partials ⚠️
src/argument_retriever.hpp 0.00% 0 Missing and 2 partials ⚠️
src/exclusion/common.hpp 90.47% 0 Missing and 2 partials ⚠️
src/attribute_collector.hpp 83.33% 1 Missing ⚠️
src/expression.hpp 66.66% 0 Missing and 1 partial ⚠️
Additional details and impacted files 
@@            Coverage Diff             @@
##           master     #468      +/-   ##
==========================================
+ Coverage   84.91%   84.97%   +0.05%     
==========================================
  Files         190      189       -1     
  Lines        9820     9716     -104     
  Branches     4293     4223      -70     
==========================================
- Hits         8339     8256      -83     
+ Misses        612      602      -10     
+ Partials      869      858      -11
Flag Coverage Δ
waf_test 84.97% <88.37%> (+0.05%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@Anilm3
Anilm3 force-pushed the anilm3/concurrent-subcontext branch from a82571d to 1d5a34b Compare October 14, 2025 09:27
@Anilm3
Anilm3 force-pushed the anilm3/concurrent-subcontext branch from 1d5a34b to c4572a3 Compare October 14, 2025 09:49
@github-actions

github-actions Bot commented Oct 14, 2025 •
edited
Loading

Copy link
Copy Markdown

Artifact Size Comparison 📦

Artifact Previous Release This PR Difference
darwin-arm64::libddwaf.a 90005656 93374392 0.03%
darwin-arm64::libddwaf.a.stripped 4579560 4920392 0.07%
darwin-arm64::libddwaf.dylib 1963024 2048736 0.04%
darwin-universal::libddwaf.a 181419976 188419432 0.03%
darwin-universal::libddwaf.a.stripped 9765384 10492280 0.07%
darwin-universal::libddwaf.dylib 4142096 4309728 0.04%
darwin-x86_64::libddwaf.a 91414272 95044992 0.03%
darwin-x86_64::libddwaf.a.stripped 5185776 5571840 0.07%
darwin-x86_64::libddwaf.dylib 2150896 2237824 0.04%
linux-aarch64::libddwaf.a 73159494 76117586 0.04%
linux-aarch64::libddwaf.a.stripped 11852226 12387390 0.04%
linux-aarch64::libddwaf.so 2463216 2513896 0.02%
linux-armv7::libddwaf.a 64719098 67268620 0.03%
linux-armv7::libddwaf.a.stripped 10852762 11374276 0.04%
linux-armv7::libddwaf.so 2148652 2171356 0.01%
linux-i386::libddwaf.a 62842618 65338596 0.03%
linux-i386::libddwaf.a.stripped 9379954 9834100 0.04%
linux-i386::libddwaf.so 2392676 2435616 0.01%
linux-x86_64::libddwaf.a 73645934 76619888 0.04%
linux-x86_64::libddwaf.a.stripped 11671234 12181236 0.04%
linux-x86_64::libddwaf.so 2660048 2711680 0.01%
windows-arm64::ddwaf.dll 4788224 4940800 0.03%
windows-arm64::ddwaf.lib 11698 16410 0.40%
windows-arm64::ddwaf_static.lib 57913408 60488906 0.04%
windows-win32::ddwaf.dll 3368448 3478528 0.03%
windows-win32::ddwaf.lib 11922 16726 0.40%
windows-win32::ddwaf_static.lib 49534122 51805770 0.04%
windows-x64::ddwaf.dll 4101632 4207616 0.02%
windows-x64::ddwaf.lib 11698 16410 0.40%
windows-x64::ddwaf_static.lib 57409488 59922948 0.04%

@Anilm3
Anilm3 force-pushed the anilm3/concurrent-subcontext branch from b2ece19 to fba2627 Compare October 14, 2025 18:32
@Anilm3
Anilm3 force-pushed the anilm3/concurrent-subcontext branch 3 times, most recently from efd9443 to 607c244 Compare October 15, 2025 09:14
@Anilm3
Anilm3 force-pushed the anilm3/concurrent-subcontext branch from 607c244 to 4cbbd85 Compare October 15, 2025 09:54
@Anilm3
Anilm3 marked this pull request as ready for review October 15, 2025 10:52
@Anilm3
Anilm3 requested a review from a team as a code owner October 15, 2025 10:52
chatgpt-codex-connector[bot]
chatgpt-codex-connector Bot reviewed Oct 15, 2025
View reviewed changes

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting

Comment thread src/evaluation_engine.hpp Outdated
@Anilm3
* Propagate pending attributes to subcontext
aa69efc 
* Share user data with subcontext
* Prevent context evaluation while subcontexts are active
* Update evaluation engine tests to use context / subcontext instead
@Anilm3

Anilm3 commented Oct 15, 2025

Copy link
Copy Markdown
Collaborator Author

@codex review

chatgpt-codex-connector[bot]
chatgpt-codex-connector Bot reviewed Oct 15, 2025
View reviewed changes

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting

Comment thread src/processor/base.hpp
@Anilm3
Anilm3 force-pushed the anilm3/concurrent-subcontext branch from e869b45 to 809aa37 Compare October 20, 2025 09:37
@Anilm3
Anilm3 force-pushed the anilm3/concurrent-subcontext branch from 809aa37 to ee72a8b Compare October 20, 2025 10:18
@Anilm3
Anilm3 force-pushed the anilm3/concurrent-subcontext branch 2 times, most recently from 7cdfc0b to aaf16ce Compare October 21, 2025 10:23
@Anilm3
Anilm3 force-pushed the anilm3/concurrent-subcontext branch from aaf16ce to 15be03e Compare October 21, 2025 14:59
@Anilm3
Anilm3 merged commit ff2357a into master Oct 22, 2025
56 checks passed
@Anilm3
Anilm3 deleted the anilm3/concurrent-subcontext branch October 22, 2025 09:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@chatgpt-codex-connector chatgpt-codex-connector[bot] chatgpt-codex-connector[bot] left review comments
@cataphract cataphract cataphract approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@Anilm3 @codecov-commenter @cataphract