.NET: Workflow Middleware to Support Customizing Telemetry

Hey folks,

Love a lot of the functionality of Agent Framework. Right now, I see a lot of use cases we have that could benefit from Workflows. However, I would like to customize the spans, particularly to include inputs/outputs. This makes it easier to construct evals from traces for compound AI systems.

Is there some way to do that today?

Skylar Payne (@skylarbpayne)

Hi Skylar Payne (@skylarbpayne),

Thanks for the interest in Agent Framework workflows. There are several ways to customize telemetry today:

Current Approaches

1. Agent-Level Middleware for Input/Output Telemetry

For agents within workflows (or standalone), you can use middleware to intercept invocations and enrich spans with inputs/outputs:

from agent_framework import AgentMiddleware, AgentRunContext
from opentelemetry import trace

class TelemetryMiddleware(AgentMiddleware):
    async def process(self, context: AgentRunContext, next):
        span = trace.get_current_span()
        # Record inputs
        span.set_attribute("agent.input", str(context.messages[-1].text) if context.messages else "")
        
        await next(context)
        
        # Record outputs
        if context.result:
            span.set_attribute("agent.output", str(context.result.text))

See the middleware samples for patterns including class_based_middleware.py and function_based_middleware.py.

2. Custom Spans Within Executors

Inside any executor handler, you can create custom spans with inputs/outputs:

from agent_framework import Executor, WorkflowContext, handler
from agent_framework.observability import get_tracer

class MyExecutor(Executor):
    @handler
    async def process(self, message: str, ctx: WorkflowContext[str]) -> None:
        tracer = get_tracer()
        with tracer.start_as_current_span("my_custom_operation") as span:
            span.set_attribute("input", message)
            result = do_something(message)
            span.set_attribute("output", result)
            await ctx.send_message(result)

3. Workflow-Level Observability

Workflows automatically emit spans for:

workflow.run - overall workflow execution
executor.process {executor_id} - each executor invocation
message.send - messages between executors

You can access the current span context and add custom attributes/events within executor handlers.

What's Not Available Today

There's currently no dedicated workflow-level middleware hook that intercepts all executor invocations in a centralized way to add custom span attributes. The telemetry enrichment happens either:

At the agent level (via AgentMiddleware)
At the chat client level (via ChatMiddleware)
Within individual executors (manual span creation)

If you need a workflow-scoped middleware that can:

Intercept every executor invocation
Access inputs/outputs for all executors uniformly
Enrich spans with custom attributes for evals

This would be a new feature. Could you share more about your specific use case? For example:

Are you primarily using WorkflowAgent (agent executors) or custom Executor classes?
What specific span attributes would be most valuable for your eval construction?

This would help us understand if the existing middleware patterns can meet your needs or if we should consider a workflow-level middleware abstraction.

When building complex agentic systems, it's helpful to be able to inspect "what happened" at multiple levels. As I "zoom in" (ie looking at a single executor rather than the whole workflow), I need to know what was passed to that executor to meaningfully understand it.

By capture executor level inputs/outputs you also enable a clearer path to constructing evals around different scales of the system (ie at workflow => executor => agent levels). This enables testability which in turn provides quick feedback loops for testing new implementations of a particular component (e.g. what if I want to swap an 'agent' / llm call with a simple classifier and want to evaluate quality / cost trade-off?)

Maybe a more concrete example will help. Let's say we want to build a voice agentLet's say we have a workflow like the following to run customer support chatbot(s):

Given company info about brand voice, desired tone/personality, we dynamically generate a personality prompt section
Given what information we already have about the user, we dynamically generate a personalized greeting
Given the user's query we retrieve potentially relevant help documents or former cases that might be related
All this goes into the main chatbot, which also can search directly for relevant help docs / cases through agentic tool use. through the conversation

I want to build evals around everything here: each individual step, as well as the whole thing.

I wouldn't want to have to explicitly implement this for every single workflow/executor.

For what it's worth, this is "standard out of the box" with most other frameworks that support graphs/workflows. E.g. see LangGraph traces in LangSmith.

Skylar Payne (@skylarbpayne)

Thanks for the detailed explanation, Skylar Payne (@skylarbpayne), this helps clarify the need.

To make sure I understand what you're asking for:

Automatic span enrichment with:

Input: The message payload passed to each executor
Output: What the executor sent downstream (via ctx.send_message) or yielded (via ctx.yield_output)

This would enable:

Full trace inspection at any zoom level without manual instrumentation
Eval construction at workflow, executor, and agent granularity
A/B testing component implementations (agent vs classifier, etc.)

We should add opt-in automatic input/output capture to executor spans. Something like:

# At workflow level
workflow = (
    WorkflowBuilder()
    .add_edge(executor1, executor2)
    .with_telemetry(capture_inputs=True, capture_outputs=True)  # opt-in
    .build()
)

Or via observability settings:

from agent_framework.observability import OBSERVABILITY_SETTINGS

OBSERVABILITY_SETTINGS.CAPTURE_EXECUTOR_IO = True

The implementation would:

Serialize inputs before handler invocation and add as span attribute
Track send_message/yield_output calls and add outputs as span attributes
Support configurable serialization (for custom types, redaction, size limits)

Does this capture what you're looking for? If so, we will track this feature request. Any preferences on the API shape (workflow-level config vs global setting)?

That captures most of it, though I would prefer something that lets me add attributes to the span myself (eg middleware). The reason is that Executors may be parametrized in someway and I might want to capture that in the span.

But capturing inputs/outputs works is a good 80% solution (and I can always stuff the parameters into the input). So yes, I think this captures what I'm looking for.

I think global with workflow level overrides: the reality is that the common case for me will be "all track input/output", but I can also imagine there might be some particularly sensitive use case where we explicitly do not want to track it.

(Also, fwiw I'm using dotnet!)

Peter Ibekwe (@peibekwe)

Peter Ibekwe (@peibekwe) any thoughts on how this would be handled in .NET? Thanks.

feat(gatekeeper): runtime fail-closed enforcement middleware M0-M5 (#76)

Workflows consist of edges and executors.

Edges are responsible for sending messages and executors are responsible to processing the messages. We currently create spans for both. What is missing is capturing the actual payload in the spans for Python.

For .Net, it's already possible to capture input & output: #3467. Please give it a try.

With EnableSensitiveData (shown in the PR Tao linked), you would get I/O capture "for free" but not custom attributes or per-executor parameterization. That's something I believe we would have to design and implement.

I am working on the Python workflow-telemetry parity described in this thread, following the .NET approach from #3467.

Planned scope:

keep workflow payload capture disabled by default;
when sensitive-data telemetry is explicitly enabled, capture executor input/output and message.send content;
use the existing safe serialization path for heterogeneous workflow payloads;
add focused unit tests for both redacted and opt-in behavior.

I will open a draft PR and link it here. Please let me know if the Python implementation should use different attribute names from the .NET workflow telemetry tags.

Draft PR opened: #7565. It adds Python workflow/executor/message payload capture behind sensitive-data telemetry, aligns the workflow-specific attributes with .NET #3467, and emits OpenInference input/output attributes for vendor-neutral backend ingestion.

Navigation Menu

Repository navigation

.NET: Workflow Middleware to Support Customizing Telemetry #3075

Current Approaches

1. Agent-Level Middleware for Input/Output Telemetry

2. Custom Spans Within Executors

3. Workflow-Level Observability

What's Not Available Today

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

.NET: Workflow Middleware to Support Customizing Telemetry #3075

Description

Activity

moonbox3 commented on Jan 5, 2026

Current Approaches

1. Agent-Level Middleware for Input/Output Telemetry

2. Custom Spans Within Executors

3. Workflow-Level Observability

What's Not Available Today

skylarbpayne commented on Jan 5, 2026

moonbox3 commented on Jan 6, 2026

skylarbpayne commented on Jan 6, 2026

moonbox3 commented on Jun 1, 2026

TaoChenOSU commented on Jul 7, 2026

peibekwe commented on Jul 7, 2026

slcnx commented on Aug 7, 2026

slcnx commented on Aug 7, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions