[Debugger] Bound root filter capture expressions#8780
Merged
Conversation
dudikeleti
requested review from
Copilot and
jpbempel
and removed request for
a team
June 11, 2026 10:55
This comment has been minimized.
This comment has been minimized.
Contributor
There was a problem hiding this comment.
Pull request overview
This PR updates the debugger expression evaluator and snapshot serializer so that root-level capture expressions of the form filter(...) are evaluated with a bounded enumeration (respecting MaxCollectionSize) rather than materializing full filtered collections before serialization. It introduces a bounded result wrapper to carry truncation and dictionary-shape metadata through to snapshot JSON, and adds an optimization that flattens direct root filter chains into a single bounded pass with a combined predicate.
Changes:
- Add bounded evaluation for root capture
filter(...)expressions, returningBoundedCaptureCollectionResult<T>(implementsIBoundedCaptureCollectionResult) withWasTruncatedand dictionary-shape metadata. - Update snapshot serialization to recognize bounded results and emit
notCapturedReason: "collectionSize"when the bounded evaluation truncated the capture. - Add/adjust unit tests for bounded filter capture behavior and serializer output, including root filter-chain flattening.
Reviewed changes
Copilot reviewed 13 out of 13 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| tracer/test/Datadog.Trace.Tests/Debugger/DebuggerSnapshotCreatorTests.cs | Adds serializer tests for bounded truncation reason; updates helper to pass wasTruncated into serializer internals. |
| tracer/test/Datadog.Trace.Tests/Debugger/DebuggerExpressionLanguageTests.cs | Adds evaluator tests for bounded root filter(...), dictionary metadata preservation, chain flattening, and exact semantics for len(filter(...)). |
| tracer/src/Datadog.Trace/Debugger/Snapshots/DebuggerSnapshotSerializer.cs | Prefers collectionSize not-captured reason when bounded results indicate truncation. |
| tracer/src/Datadog.Trace/Debugger/Snapshots/DebuggerSnapshotSerializer.CollectionDescriptors.cs | Plumbs bounded result metadata into SupportedEnumerableInfo (count, dictionary-shape, truncation). |
| tracer/src/Datadog.Trace/Debugger/Expressions/ProbeExpressionParser.cs | Adds ParserContext + capture-limit plumbing and a dedicated ParseCaptureExpression(...) entrypoint. |
| tracer/src/Datadog.Trace/Debugger/Expressions/ProbeExpressionParser.Collection.cs | Implements bounded root-filter parsing, filter-chain flattening, and predicate combination. |
| tracer/src/Datadog.Trace/Debugger/Expressions/ProbeExpressionEvaluator.cs | Uses ParseCaptureExpression(...) for capture expressions so root-level filter(...) can be bounded. |
| tracer/src/Datadog.Trace/Debugger/Expressions/ParserContext.cs | Introduces parsing context enum to enable capture-expression-specific behavior. |
| tracer/src/Datadog.Trace/Debugger/Expressions/ParameterReplacingVisitor.cs | Adds helper ExpressionVisitor for combining filter predicates safely. |
| tracer/src/Datadog.Trace/Debugger/Expressions/IBoundedCaptureCollectionResult.cs | Defines interface for bounded collection results (count, truncation, dictionary-shape). |
| tracer/src/Datadog.Trace/Debugger/Expressions/FilterExpression.cs | Adds internal expression node used to defer/filter-chain flattening. |
| tracer/src/Datadog.Trace/Debugger/Expressions/FilterEvaluationHelpers.cs | Adds bounded filtering helper that stops after limit + 1 matching items. |
| tracer/src/Datadog.Trace/Debugger/Expressions/BoundedCaptureCollectionResult.cs | Implements bounded collection wrapper used by root capture filter(...). |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 9f18516b89
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
dudikeleti
marked this pull request as draft
June 11, 2026 11:08
BenchmarksBenchmark execution time: 2026-06-17 13:25:55 Comparing candidate commit 2d250e5 in PR branch Found 0 performance improvements and 0 performance regressions! Performance is the same for 72 metrics, 0 unstable metrics, 60 known flaky benchmarks, 66 flaky benchmarks without significant changes.
|
dudikeleti
force-pushed
the
dudik/bounded-capture-filters
branch
from
June 11, 2026 13:58
9f18516 to
9ae0a02
Compare
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 14 out of 14 changed files in this pull request and generated 2 comments.
Comments suppressed due to low confidence (1)
tracer/src/Datadog.Trace/Debugger/Expressions/ProbeExpressionParser.Collection.cs:94
ReturnDefaultValueExpression()is called when the source evaluates toUndefinedValueType, but the result is not returned. This is a no-op (the return expression is discarded) and can lead to misleading follow-up errors (e.g., "Source must be an array...") instead of short-circuiting to the default return value.
source = ParseTree(reader, parameters, outerItParameter);
if (source.Type == ProbeExpressionParserHelper.UndefinedValueType)
{
ReturnDefaultValueExpression();
}
Execution-Time Benchmarks Report ⏱️Execution-time results for samples comparing This PR (8780) and master. ✅ No regressions detected - check the details below Full Metrics ComparisonFakeDbCommand
HttpMessageHandler
Comparison explanationExecution-time benchmarks measure the whole time it takes to execute a program, and are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are highlighted in **red**. The following thresholds were used for comparing the execution times:
Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard. Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph). Duration chartsFakeDbCommand (.NET Framework 4.8)gantt
title Execution time (ms) FakeDbCommand (.NET Framework 4.8)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8780) - mean (74ms) : 72, 77
master - mean (73ms) : 71, 75
section Bailout
This PR (8780) - mean (78ms) : 76, 80
master - mean (79ms) : 75, 82
section CallTarget+Inlining+NGEN
This PR (8780) - mean (1,099ms) : 1040, 1157
master - mean (1,093ms) : 1051, 1135
FakeDbCommand (.NET Core 3.1)gantt
title Execution time (ms) FakeDbCommand (.NET Core 3.1)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8780) - mean (116ms) : 110, 122
master - mean (115ms) : 109, 122
section Bailout
This PR (8780) - mean (115ms) : 112, 118
master - mean (117ms) : 112, 122
section CallTarget+Inlining+NGEN
This PR (8780) - mean (793ms) : 766, 820
master - mean (788ms) : 769, 808
FakeDbCommand (.NET 6)gantt
title Execution time (ms) FakeDbCommand (.NET 6)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8780) - mean (101ms) : 98, 104
master - mean (104ms) : 99, 108
section Bailout
This PR (8780) - mean (104ms) : 100, 109
master - mean (103ms) : 100, 107
section CallTarget+Inlining+NGEN
This PR (8780) - mean (950ms) : 910, 989
master - mean (953ms) : 916, 990
FakeDbCommand (.NET 8)gantt
title Execution time (ms) FakeDbCommand (.NET 8)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8780) - mean (101ms) : 96, 106
master - mean (99ms) : 96, 103
section Bailout
This PR (8780) - mean (101ms) : 98, 103
master - mean (103ms) : 99, 106
section CallTarget+Inlining+NGEN
This PR (8780) - mean (823ms) : 789, 858
master - mean (819ms) : 778, 861
HttpMessageHandler (.NET Framework 4.8)gantt
title Execution time (ms) HttpMessageHandler (.NET Framework 4.8)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8780) - mean (199ms) : 196, 203
master - mean (199ms) : 193, 206
section Bailout
This PR (8780) - mean (203ms) : 200, 206
master - mean (204ms) : 199, 208
section CallTarget+Inlining+NGEN
This PR (8780) - mean (1,202ms) : 1156, 1248
master - mean (1,192ms) : 1156, 1228
HttpMessageHandler (.NET Core 3.1)gantt
title Execution time (ms) HttpMessageHandler (.NET Core 3.1)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8780) - mean (286ms) : 281, 291
master - mean (284ms) : 277, 291
section Bailout
This PR (8780) - mean (287ms) : 283, 292
master - mean (289ms) : 284, 294
section CallTarget+Inlining+NGEN
This PR (8780) - mean (967ms) : 950, 984
master - mean (966ms) : 948, 985
HttpMessageHandler (.NET 6)gantt
title Execution time (ms) HttpMessageHandler (.NET 6)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8780) - mean (280ms) : 275, 286
master - mean (278ms) : 272, 285
section Bailout
This PR (8780) - mean (281ms) : 277, 284
master - mean (277ms) : 269, 285
section CallTarget+Inlining+NGEN
This PR (8780) - mean (1,161ms) : 1125, 1197
master - mean (1,157ms) : 1119, 1196
HttpMessageHandler (.NET 8)gantt
title Execution time (ms) HttpMessageHandler (.NET 8)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8780) - mean (278ms) : 273, 283
master - mean (276ms) : 268, 285
section Bailout
This PR (8780) - mean (278ms) : 271, 285
master - mean (277ms) : 273, 281
section CallTarget+Inlining+NGEN
This PR (8780) - mean (1,039ms) : 999, 1079
master - mean (1,035ms) : 995, 1076
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
dudikeleti
marked this pull request as ready for review
June 11, 2026 15:25
dudikeleti
force-pushed
the
dudik/bounded-capture-filters
branch
from
June 12, 2026 07:43
09a1d71 to
609168d
Compare
jpbempel
approved these changes
Jun 12, 2026
dudikeleti
enabled auto-merge (squash)
June 12, 2026 09:39
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary of changes
filter(...)capture expressions.BoundedCaptureCollectionResult<T>, exposing:WasTruncatedIsDictionaryCountnotCapturedReason: "collectionSize"when a bounded filtered capture was truncated.filter(filter(collection, p1), p2)by flattening them into one bounded enumeration with a combined predicate.Reason for change
filter(largeCollection, predicate)and only a small number of items can be serialized anyway.Implementation details
filter(...).filter(collection, predicate).filter(filter(collection, p1), p2).len(filter(collection, predicate))still evaluates the complete filtered collection so the count remains exact.any(filter(collection, p1), p2)andall(filter(...), ...)keep existing exact semantics.index(filter(...), key)keeps existing exact semantics.filter(filter(source, p1), p2)becomes one pass oversourcewithp1 && p2.filter(index(filter(source, p1), 0), p2)keeps the innerindex(filter(...), 0)semantics before bounding the root filter output.MaxCollectionSizematching items.WasTruncated = true.WasTruncated = false.IBoundedCaptureCollectionResult.WasTruncatedis true and serialization completes the bounded result, it emitsnotCapturedReason: "collectionSize"even when the wrapper count is below the normal serializer limit. If serialization is canceled before all bounded result items are written, timeout takes priority.IsDictionaryis true, the result is serialized using dictionary/entry shape instead of normal collection element shape.ProbeExpressionParserHelpercache; cache keys snapshot method signature arrays and precompute their hash for stable, cheap lookups.Test coverage
index(filter(...), 0)under a bounded root filter source preserving materialized filter semantics.index(filter(...), 0).Valueunder a bounded root filter source preserving materialized filter semantics.len(filter(...))preserving exact nested filter semantics.collectionSize.collectionSize.collectionSize.Local test runs:
DebuggerExpressionLanguageTests.ProbeExpressionEvaluator_CaptureExpressionRootFilter|DebuggerExpressionLanguageTests.ProbeExpressionEvaluator_LenOfFilter_KeepsExactSemantics|DebuggerExpressionLanguageTests.FilterEvaluationHelpers_FilterForCapture_StopsAfterLimitAndOneExtraMatch"DebuggerSnapshotCreatorTests.Limits_BoundedCaptureCollectionResult|DebuggerSnapshotCreatorTests.CaptureExpressions_AreWrittenWithoutArgumentsOrLocals"dotnet test "tracer/test/Datadog.Trace.Tests/Datadog.Trace.Tests.csproj" --no-restore --filter "FullyQualifiedName~DebuggerExpressionLanguageTests"Other details