Skip to content

Python: Support response_format in GitHubCopilotAgent - #9810

Merged
Eduard van Valkenburg (eavanvalkenburg) merged 2 commits into
microsoft:mainfrom
droideronline:feature/github-copilot-structured-output
Nov 11, 2026
Merged

Eduard van Valkenburg (eavanvalkenburg) merged 2 commits into
microsoft:mainfrom
droideronline:feature/github-copilot-structured-output

Conversation

@droideronline

@droideronline Dineshsuriya D (droideronline) commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

Motivation & Context

GitHubCopilotAgent ignored response_format, so structured output was unavailable even though the Copilot SDK supports it (github/copilot-sdk#2590, in SDK v1.0.15 and later, within the existing >=1.0.16 pin).

Description & Review Guide

  • What are the major changes?
    • Added response_format (Pydantic model type or JSON Schema mapping) to GitHubCopilotOptions.
    • The agent forwards it as response_schema to send_and_wait (non-streaming) and send (streaming). Per-run options override default_options.
    • response_format is stripped from the create_session kwargs so it no longer reaches the SDK session factory.
    • AgentResponse.value is parsed from the final text: via response_format= on the non-streaming response and output_format_type= in the streaming finalizer.
  • What is the impact of these changes? Opt-in. Without response_format the SDK receives response_schema=None, which is the existing behavior.
  • What do you want reviewers to focus on? Streaming still yields text deltas, with the value parsed once the stream completes, matching the SDK behavior.

Dependency note: This change depends on the next agent-framework-core version bump. It relies on the shared structured-output parsing fix in core (alias-aware validation of AgentResponse.value), which core 1.22.0 does not contain, and the current agent-framework-github-copilot dependency floor still permits 1.22.0. No dependency change is needed in this PR; the next version bump will raise the floor.

Related Issue

Fixes #9809

AI Assistance

  • No material AI assistance was used.
  • This is an AI-assisted contribution. I reviewed, understood, and verified all submitted content and accept responsibility for it.

AI assistance details: implementation and tests.

Contribution Checklist

  • The code builds clean without any errors or warnings
  • All unit tests pass, and I have added new tests where possible
  • The PR follows the Contribution Guidelines
  • This PR links to an agreed issue with no competing open PR, or the Related Issue section documents a trivial-change or repository-automation exception.
  • This is not a breaking change. If it is a breaking change, add the breaking change label (or add "[BREAKING]" to the title prefix, before or after any language prefix) — a workflow keeps the label and title prefix in sync automatically.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Structured-output parsing can incorrectly use trailing subagent text instead of the root answer, while mapping schemas and aliased models can trigger serialization or validation failures.

3 open issues
What changed in this PR

Adds opt-in structured output support to GitHubCopilotAgent, bringing it closer to other Agent Framework providers and addressing #9809.

Changes:

  • Accepts Pydantic models or JSON Schema mappings through response_format.
  • Forwards schemas to streaming and non-streaming SDK calls, excludes them from session creation, and enables AgentResponse.value parsing.
  • Adds tests for schema forwarding, default options, unstructured requests, and streaming results.
File Description
python/​packages/​github_copilot/​tests/​test_github_copilot_agent.py Adds structured-output tests for both response modes.
python/​packages/​github_copilot/​agent_framework_github_copilot/​_agent.py Adds the option, SDK forwarding, and structured response parsing.

Review effort: Balanced

@droideronline

Copy link
Copy Markdown
Contributor Author

Hi Eduard van Valkenburg (@eavanvalkenburg) , please take a look when you get a chance

@droideronline

Copy link
Copy Markdown
Contributor Author

Eduard van Valkenburg (@eavanvalkenburg) - the pipeline failure is fixed. Package Checks (3.11) failed because pyright could not resolve the from_updates overload in the streaming finalizer: options.get("response_format") was partially unknown. It is now typed as Any (a14d0ce), and pyright, ruff and the github_copilot tests are clean locally.

Could you please retrigger the pipeline when you get a chance? Thanks!

@droideronline

Copy link
Copy Markdown
Contributor Author

Eduard van Valkenburg (@eavanvalkenburg) - I moved the alias parsing into core as you suggested (790ac8e), and fixed the typing failure in the new aliased-model tests (3d1e5c8). Could you please retrigger the pipeline when you get a chance?

If everything is green, would it be possible to get this into the next release? We are waiting on structured output for the GitHub Copilot agent. Thanks!

@eavanvalkenburg

Copy link
Copy Markdown
Member

Dineshsuriya D (@droideronline) Hard-coding by_name=False in shared response parsing overrides models configured with validate_by_name=True or populate_by_name=True. A model with item_count: int = Field(alias="itemCount") previously accepted {"item_count": 3} through AgentResponse.value and ChatResponse.value; this head now raises ValidationError. Please preserve each model’s configured field-name validation while still supporting alias-keyed output, and add a name-keyed regression test.

@droideronline

Copy link
Copy Markdown
Contributor Author

Eduard van Valkenburg (@eavanvalkenburg) - fixed in 6cdf1e0. Core now validates with the model's own configuration first, so name-keyed output (validate_by_name or populate_by_name) is accepted as before. Only if that fails does it retry with aliases only (pydantic 2.11+) to support alias-keyed provider output, and it re-raises the original error if both fail. Added regression tests for name-keyed output and for the original error on invalid output.

@eavanvalkenburg

Copy link
Copy Markdown
Member

Dineshsuriya D (@droideronline) One correctness issue remains in the alias fallback: defaulted aliased fields can silently lose the provider’s value. With validate_by_alias=False and validate_by_name=True, item_count: int = Field(default=0, alias="itemCount") parses {"itemCount": 3} as item_count=0 because initial validation succeeds and the alias retry never runs. I reproduced this through both streaming and non-streaming GitHubCopilotAgent paths. Please enable alias validation while preserving configured field-name validation, and add a defaulted-aliased-field regression for both run modes.

@droideronline

Copy link
Copy Markdown
Contributor Author

Eduard van Valkenburg (@eavanvalkenburg) - good catch, fixed in b2dc326. Core now validates with a single pass using by_alias=True and leaves by_name to the model's own configuration. Aliases are always accepted, so a defaulted aliased field keeps the provider's value instead of silently using its default, and models configured with validate_by_name or populate_by_name still accept name-keyed output. The retry logic is gone. Added defaulted-aliased-field regressions for the core response and for both streaming and non-streaming GitHubCopilotAgent runs.

@eavanvalkenburg

Copy link
Copy Markdown
Member

Dineshsuriya D (@droideronline) Please add a note to the PR description that this change depends on the next agent-framework-core version bump. The current agent-framework-github-copilot dependency floor still permits core 1.22.0, which does not contain the shared parser fix used by this PR. No dependency change is required here; the next version bump will update the floor.

@droideronline

Copy link
Copy Markdown
Contributor Author

Eduard van Valkenburg (@eavanvalkenburg) - added a dependency note to the PR description: this change depends on the next agent-framework-core version bump, and no dependency change is needed here since the bump will raise the floor.

Forward response_format as response_schema to the Copilot SDK send and
send_and_wait calls and keep it out of create_session kwargs. Parse
AgentResponse.value in core with alias validation enabled while preserving
each model's field-name validation.

For streaming, only the root agent's final idle ends the run, aborted and
autopilot idle events are handled like the SDK waiter, and events are
correlated to the request ID returned by send so an earlier request cannot
supply the structured value.
@droideronline
Dineshsuriya D (droideronline) force-pushed the feature/github-copilot-structured-output branch from 955c090 to eb24f70 Compare November 9, 2026 10:40
@droideronline

Copy link
Copy Markdown
Contributor Author

Eduard van Valkenburg (@eavanvalkenburg) - addressed the three streaming comments (request correlation, aborted idle, autopilot idle) and rebased onto the latest main, so the branch is now a single commit. Please take another look and retrigger the pipeline when you get a chance. Thanks!

@droideronline

Copy link
Copy Markdown
Contributor Author

Fixed the failing typing check (Test Typing Checks (3.11), which also tripped merge-gatekeeper). The two new streaming tests set aborted and mode on the idle event data, which ty and mypy reject on the SDK's data union type. They now assign through an Any-typed variable. Test-only change in 397de11; the Copilot tests, ruff and ty are clean locally.

Merged via the queue into microsoft:main with commit b193c1b Nov 11, 2026
50 checks passed

This branch was previously deployed

1 inactive deployment
github-app-auth — 397de11b Deployed Nov 9, 2026 by droideronline via add_label #27395
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

python Usage: [Issues, PRs], Target: Python

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Python: [Feature]: Support response_format (structured output) in GitHubCopilotAgent

3 participants