Repository navigation
Description
In what area(s)?
/area runtime
/area workflows
What version of Dapr?
1.17.x (msft fork: daprd 1.17.2-msft.1)
Expected Behavior
When a workflow calls continue_as_new, the new orchestration iteration should start with a fresh trace ID. Each iteration of an eternal/polling workflow should produce an independent trace, allowing distributed tracing backends (Geneva, Azure Monitor, Jaeger, etc.) to display bounded, queryable traces per iteration.
Actual Behavior
The daprd sidecar propagates the same trace context across continue_as_new boundaries. All iterations of an eternal workflow share a single trace ID indefinitely. Over hours/days, this produces a single unbounded trace containing thousands of spans — making it:
- Unusable in tracing UIs — the waterfall view becomes impossibly long
- Unqueryable — you cannot filter to "what happened in this specific polling cycle"
- A potential storage/performance concern — trace backends may not expect traces spanning days/weeks
Evidence
OrchestratorRequestprotobuf has noparentTraceContextfield — trace context propagates via gRPC metadata from the sidecar- The Python
durabletaskworker cannot override this — it receives the trace context, it does not create it - Activities receive
parentTraceContextfromActivityRequestproto, which inherits from the same long-lived trace - Screenshot from Geneva showing a single trace spanning the full
init_customer_workflowsorchestration (5+ minute iterations, sametrace_idacross all):
Trace ID: 527ee9323b3743020b2f045ce4dde4f0
├── orchestration|init_customer_workflows 5.0 mins
│ ├── activity|init_list_cell_namespaces 187 ms
│ ├── activity|update_start_time 361 ms
│ ├── activity|update_start_time 439 ms
│ └── activity|fetch_report_ids 1.3 s
│ └── ddms.request 1.3 s ← our custom span
│ ├── POST (200) 378 ms
│ └── POST (200) 902 ms
└── (next iteration reuses same trace_id...)
Proposed Solution
In durabletask-go, when processing a ORCHESTRATION_STATUS_CONTINUED_AS_NEW completion:
- Generate a new trace ID for the re-dispatched orchestration instance
- Optionally, add a link from the new trace to the previous iteration's trace (preserving correlation without parent-child)
- Set the new
traceParentin gRPC metadata when dispatching the nextOrchestratorRequest
This would give each iteration its own trace while still allowing backends that support trace links to navigate between iterations.
Workaround
Currently adding custom span attributes (iteration_start timestamp) to child spans so we can filter within the mega-trace, but this does not solve the unbounded trace growth problem.
Steps to Reproduce the Problem
- Create an eternal workflow using
ctx.continue_as_new(...) - Enable distributed tracing (OTLP exporter to any backend)
- Let the workflow run 2+ iterations
- Observe that all iterations share the same
trace_id
Related Issues
- [1.17] Workflows: Bidirectional trace propagation gaps between runtime and SDKs #9189 — Bidirectional trace propagation (fixed in 1.17, but does not address
continue_as_new) - Enable distributed trace context to flow to external services that are invoked by a workflow activity #7927 — Trace context not available in activities (fixed, related but different)
Release Note
RELEASE NOTE: ADD Workflow continue_as_new resets trace context, producing bounded per-iteration traces.
Activity
SarthakB11 commentedon Jul 1, 2026
Hi @ayackel, took a look at where the trace context propagates. The carryover happens in
dapr/durabletask-go, not in this repo.On
CONTINUE_AS_NEW,backend/runtimestate/applier.go(in theActionsCAN branch) builds the new generation'sExecutionStartedEventand copies the old context directly:Each run parents its span off that field, so every generation re-joins the original trace. That's the unbounded growth you're seeing.
Fix shape, in durabletask-go: mint a fresh root trace context for the new generation instead of copying the old one. Two options:
Before I open a PR: is durabletask-go the repo you want this in, and do you prefer reset-only or reset-with-link?
ayackel commentedon Jul 1, 2026
Reset-only is fine with me. I don't know that anyone would optionally want unbounded traces .
SarthakB11 commentedon Jul 4, 2026
Opened dapr/durabletask-go#113 (reset-only, per your ack). Each generation now gets a fresh W3C root TraceContext when the workflow was traced, and stays untraced when it was not. Existing
Test_ContinueAsNewend-to-end still green; added a unit test intests/runtimestate_test.gofor both branches.fix: reset ParentTraceContext on ContinueAsNew to bound per-generatio…
fix: reset ParentTraceContext on ContinueAsNew to bound per-generatio…
fix: reset ParentTraceContext on ContinueAsNew to bound per-generatio…
workflows: fresh trace roots on ContinueAsNew (dapr#10321)
workflows: fresh trace roots on ContinueAsNew (#10321)
workflows: fresh trace roots on ContinueAsNew (#10321) (#10324)