feat(ui): add UIAdapter.manage_system_prompt + ReinjectSystemPrompt capability#4087
feat(ui): add UIAdapter.manage_system_prompt + ReinjectSystemPrompt capability#4087
UIAdapter.manage_system_prompt + ReinjectSystemPrompt capability#4087Conversation
system_prompt in UI adapters message_history when no model responses existsystem_prompt in UI adapters' message_history when no model responses exist
Docs Preview
|
|
This PR is stale, and will be closed in 7 days if no reply is received. |
…ins SystemPromptParts Include next_message in the system prompt existence check so that when message_history ends with a ModelRequest that already has SystemPromptParts, fresh ones aren't injected again. Co-Authored-By: Claude Opus 4.6 <[email protected]>
…uests, not just the first The code iterates over every ModelRequest in the history and skips injection if any SystemPromptPart is present anywhere, but the doc copy said 'first request'. Align the wording with the actual behavior.
'sys_parts' is an internal implementation term (the list of ModelRequestParts built by UserPromptNode._sys_parts); users don't see it in any public API. Rephrase the auto-injection docs and the UIAdapter.manage_system_prompt docstring to talk about 'system prompt' consistently.
Previous approaches to UIAdapter's 'server mode' system-prompt reinjection were architecturally unsatisfying: a ContextVar with mutable state (subtle leakage into nested tool agents), then generalized auto-injection in UserPromptNode (silently changed the contract of agent.run(message_history=[...]) that the docs explicitly documented). Shift to a capability-based design: - New `ReinjectSystemPrompt` capability in `pydantic_ai.capabilities`, hooked on `before_model_request`. If no SystemPromptPart is present in history, it resolves the agent's configured system_prompt and prepends it at the head of the first ModelRequest. No-op if any sys_prompt is already there. - New public `AbstractAgent.system_prompts(...)` method that resolves configured static strings and runners (static + dynamic) into SystemPromptParts, using a minimal RunContext built from kwargs. Reusable by anyone who needs to produce the same parts outside an agent run. - New `capabilities=` run-time kwarg on `Agent.run` / `iter` / `run_stream` / `run_stream_events` — additive with the agent's configured capabilities, matching the `toolsets=` / `builtin_tools=` pattern. - UIAdapter in `manage_system_prompt='server'` mode adds ReinjectSystemPrompt via the new kwarg. Client mode is unchanged (agent's configured sys_prompt is not injected; add the capability explicitly for fallback behavior). - Shared `resolve_system_prompts` helper moved into `_system_prompt.py`; `UserPromptNode._sys_parts` delegates to it, removing duplication. - UserPromptNode's own injection logic is restored to the pre-PR behavior (only injects on the first turn of a fresh conversation) — no surprise behavior change for `agent.run(message_history=[...])` callers. A `# TODO (v2):` on the capability notes the option of enabling it by default in v2 to close #1646 — but as a deliberate breaking change, not smuggled in.
|
|
||
| On the frontend, AI SDK UI's [`useChat`](https://ai-sdk.dev/docs/reference/ai-sdk-ui/use-chat) hook handles the approval flow. You can use the [`Confirmation`](https://ai-sdk.dev/elements/components/confirmation) component from AI Elements for a pre-built approval UI, or build your own using the hook's `addToolApprovalResponse` function. | ||
|
|
||
| ## System prompts and instructions |
There was a problem hiding this comment.
This uses ## (top-level) with title case ("System Prompts and Instructions"), while the equivalent section in docs/ui/ag-ui.md uses ### System prompts and instructions (sentence case, nested under ## Features). These should be consistent — the AG-UI version is more correct because it's a subsection, not a top-level section. Please change to ### System prompts and instructions to match.
| from pydantic_ai import Agent | ||
| from pydantic_ai.ui.ag_ui import AGUIAdapter | ||
|
|
||
| agent = Agent('openai:gpt-5.2', system_prompt='Be fun!') |
There was a problem hiding this comment.
This example is misleading: it configures system_prompt='Be fun!' on the agent, but then uses manage_system_prompt='client', which means the agent's configured system prompt is not injected — the frontend is expected to send it. A reader who copies this example will end up with an agent whose system_prompt has no effect.
The example title says "client managed system prompt" so the intent is correct, but the system_prompt='Be fun!' creates a false impression that it does something. Consider either:
- Removing
system_prompt=from the agent to make it clear the frontend owns it, or - Switching this to a
'server'mode example that actually demonstrates the configuredsystem_promptbeing reinjected (which is the more common and interesting case for docs)
| from pydantic_ai import Agent | ||
| from pydantic_ai.ui.vercel_ai import VercelAIAdapter | ||
|
|
||
| agent = Agent('openai:gpt-5.2', system_prompt='Be fun!') |
There was a problem hiding this comment.
Same issue as the AG-UI example: system_prompt='Be fun!' has no effect when manage_system_prompt='client' — the frontend is responsible for sending the system prompt. See my comment on the AG-UI equivalent.
| Dynamic runners produce parts with `dynamic_ref` set so they can continue to be | ||
| re-evaluated by the standard agent graph path on subsequent turns. | ||
| """ | ||
| return [] |
There was a problem hiding this comment.
@DouweM New public API: AbstractAgent.system_prompts(). This returns [] by default in the abstract base class, is overridden in Agent to resolve configured prompts, and delegates through WrapperAgent. Called by ReinjectSystemPrompt.before_model_request to resolve the agent's system prompts.
Worth confirming this is the right shape for this method — specifically the signature with deps, message_history, prompt, usage, model_settings kwargs that are used to construct a RunContext inside Agent.system_prompts(). An alternative would be to accept a RunContext directly (which ReinjectSystemPrompt already has), but the current design is more ergonomic for external callers who don't have a RunContext handy.
| request_context: ModelRequestContext, | ||
| ) -> ModelRequestContext: | ||
| messages = request_context.messages | ||
| if _has_system_prompt(messages): |
There was a problem hiding this comment.
@DouweM The _has_system_prompt check (line 69) means this capability is a no-op if any SystemPromptPart exists anywhere in the history. This is intentional for the multi-agent handoff case, but it also means that in 'server' mode, if the frontend manages to sneak a SystemPromptPart through (e.g. via a non-UIAdapter path), the agent's own system prompt won't be reinjected.
In the UIAdapter flow this is fine because frontend system prompts are stripped before the capability runs. But for standalone use (e.g. Agent(..., capabilities=[ReinjectSystemPrompt()])) a user who passes a message_history containing a SystemPromptPart from a different agent will get that agent's system prompt respected instead of the current agent's. The docstring documents this as intentional behavior for "multi-agent handoff", which seems like the right call — just flagging for explicit awareness.
| if _has_system_prompt(messages): | ||
| return request_context | ||
| if ctx.agent is None: | ||
| return request_context # pragma: no cover — ctx.agent is always set during an agent run |
There was a problem hiding this comment.
This pragma: no cover comment says "ctx.agent is always set during an agent run" — but capabilities can theoretically be used outside agent runs. Given that the early return here just silently returns without injecting, it might be worth dropping the pragma and adding a minimal test, or at minimum making the comment more precise: something like # defensive guard — should not happen during a normal agent run.
…snapshot Two small CI fixes for the ReinjectSystemPrompt capability work: 1. The `capabilities=` run-time parameter was added to `AbstractAgent`'s run/run_sync/run_stream/run_stream_events/iter overloads, but not propagated to the DBOS/Prefect/Temporal durable_exec agent overrides. Pyright flagged these as incompatible overrides. 2. `ReinjectSystemPrompt.get_serialization_name` returned snake_case, which broke the `CAPABILITY_TYPES` snapshot and deviated from the other capabilities' PascalCase convention. Remove the override so the default (class name) is used. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
- `AbstractAgent.system_prompts` default `return []`: mark `pragma: no cover` since `Agent` and `WrapperAgent` always override it. - `WrapperAgent.system_prompts`: call through it in `test_wrapper_agent` so the delegation is exercised. - `ReinjectSystemPrompt._prepend_to_first_request`: use `next(...)` instead of an explicit loop so the helper has no uncovered branches (callers already guarantee there is at least one `ModelRequest`). Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
…nstructions) `Agent.run()`'s capability selection previously only checked for `override()` or spec-based capabilities when deciding whether to re-extract per-run capability contributions (tools, instructions, builtin tools, model settings). When a user passed capabilities only via the new run-time `capabilities=` parameter, `effective_capability` was correctly wrapped in a `CombinedCapability`, but `source_cap` fell through to `None`, causing the code to use init-time defaults and silently dropping those capabilities' configuration contributions. Fix by checking `extra_capabilities` (spec + run-time), matching the construction of `effective_capability` above. Add a regression test that passes a toolset-providing capability only via `capabilities=` and asserts the tool is invoked. Caught by Devin review on PR #4087. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
When an agent is constructed without a configured model and receives it at run time via `agent.run(model=...)`, `ReinjectSystemPrompt.before_model_request` needs to call `agent.system_prompts()` to resolve the prompts. Previously `system_prompts()` unconditionally called `self._get_model(None)` to build a `RunContext`, which raised `UserError` in this case — so UI adapters in the default `manage_system_prompt='server'` mode broke for model-less agents. Accept an optional `model` parameter on `system_prompts()` and forward `ctx.model` from the capability's `before_model_request` hook. The parameter propagates through `AbstractAgent` → `Agent` → `WrapperAgent`. Caught by Devin review on PR #4087. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
UIAdapter.manage_system_prompt + ReinjectSystemPrompt capability
- Drop `system_prompt='Be fun!'` from the `'client'` mode examples in `docs/ui/ag-ui.md` and `docs/ui/vercel-ai.md`. In client mode the agent's configured `system_prompt` is intentionally not injected, so the example was silently demonstrating an inert setting. The prose above already covers the `ReinjectSystemPrompt` fallback pattern for callers who want it. - Link `SystemPromptPart` and `ModelRequest` to their API reference pages in the `ReinjectSystemPrompt` section of `docs/capabilities.md` and the two UI docs, per `docs/CLAUDE.md` guidance. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Move UIAdapter's manual system-prompt stripping into the capability itself via an opt-in `replace_existing: bool = False` flag: - `replace_existing=False` (default, unchanged): capability is a no-op if any `SystemPromptPart` is already in history. Multi-agent handoff and user-managed prompts remain authoritative. - `replace_existing=True`: capability strips any existing `SystemPromptPart`s (dropping resulting empty `ModelRequest`s) before prepending the agent's configured prompt. UIAdapter `'server'` mode now passes `ReinjectSystemPrompt(replace_existing=True)` and only retains its `manage_system_prompt`-aware UserWarning (no more manual message surgery). The capability becomes a reusable anti-injection primitive for direct-agent users with untrusted histories. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
| def _strip_system_prompts(messages: list[ModelMessage]) -> None: | ||
| kept: list[ModelMessage] = [] | ||
| for msg in messages: | ||
| if isinstance(msg, ModelRequest): | ||
| msg.parts = [p for p in msg.parts if not isinstance(p, SystemPromptPart)] | ||
| if not msg.parts: | ||
| continue | ||
| kept.append(msg) | ||
| messages[:] = kept |
There was a problem hiding this comment.
🚩 Messages can start with ModelResponse after system-prompt-only ModelRequest is stripped
When replace_existing=True and the frontend sends a SystemMessage followed by an AssistantMessage then a UserMessage, the MessagesBuilder produces [ModelRequest([SystemPromptPart]), ModelResponse([TextPart]), ModelRequest([UserPromptPart])]. After _strip_system_prompts, the first ModelRequest is dropped (empty parts), leaving [ModelResponse(...), ModelRequest(...)]. The agent then prepends the system prompt to the second ModelRequest, resulting in messages starting with a ModelResponse. The agent graph only validates that messages end with a ModelRequest (_agent_graph.py:822), so this passes. The test test_frontend_system_prompt_only_request_dropped confirms this behavior works with TestModel, but some model providers may not handle messages starting with ModelResponse well. This is a known edge case of the stripping logic rather than a bug.
Was this helpful? React with 👍 or 👎 to provide feedback.
…tSystemPrompt `request_context.messages` is a shallow copy of `ctx.state.message_history`, so `ModelRequest` objects are shared between the capability's view and the user's input `message_history=[...]` list. The previous implementation mutated `msg.parts = [...]` on shared objects, leaking edits back into the caller's list (they'd see `SystemPromptPart`s stripped from their own `ModelRequest` instances). Use `dataclasses.replace` to create new `ModelRequest` objects when we need to change their parts, and swap them into the copy list without touching the originals. Adds a regression test exercising `replace_existing=True` with a caller-owned `ModelRequest` containing a `SystemPromptPart`. Caught by Devin review on PR #4087. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
…100% coverage `next((i, m) for i, m in enumerate(messages) if isinstance(m, ModelRequest))` replaces the for/if loop, eliminating the two uncovered loop-exit branches that appeared when the helper switched from mutating `msg.parts` to using `dataclasses.replace` via indexed assignment. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
| infer_name=infer_name, | ||
| toolsets=toolsets, | ||
| builtin_tools=builtin_tools, | ||
| capabilities=capabilities, | ||
| ) |
There was a problem hiding this comment.
📝 Info: Server mode strips frontend system prompts via capability, not at warning site
In run_stream_native at lines 282-297, the warning about frontend system prompts is emitted but the frontend_messages (still containing SystemPromptParts) are passed through to message_history. The actual stripping is deferred to the ReinjectSystemPrompt(replace_existing=True) capability which fires later in before_model_request. This is intentional — the warning is an early diagnostic, while the capability handles the mutation in the correct lifecycle hook where ctx.state.message_history can be properly synced back (at _agent_graph.py:835). Readers unfamiliar with the design might expect stripping to happen at the warning site.
Was this helpful? React with 👍 or 👎 to provide feedback.
The new `AbstractAgent` method returns `list[SystemPromptPart]`, not a list of strings. `system_prompt_parts` makes that explicit and leaves the unqualified `system_prompt` name free for future additions (for example, a matching `instruction_parts()` that resolves the agent's configured [`instructions`][pydantic_ai.Agent.instructions] into structured parts — the `@agent.instructions` decorator already occupies the singular name). Also adds `test_reinject_system_prompt_capability_reaches_model_and_all_messages` using `FunctionModel` to capture what the model actually received, asserting both that the injected `SystemPromptPart` reaches the model AND that it persists in `result.all_messages()`. This directly verifies that the hook's mutations to `request_context.messages` are synced back to canonical state via `_agent_graph.py:835`, and addresses the ambiguity flagged by Devin. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
system_promptis ignored when using UI adapter (AG-UI, Vercel AI) orto_web#3315Summary
Two changes, both composing around a new capability.
manage_system_promptonUIAdapterNew
manage_system_prompt: Literal['server', 'client'] = 'server'on [UIAdapter][pydantic_ai.ui.UIAdapter] (and thehandle_ag_ui_request/run_ag_uiconvenience functions, andVercelAIAdapter).'server'(default): anySystemPromptPartsent by the frontend is stripped with a warning, since a malicious client could otherwise inject arbitrary instructions via crafted API requests. The agent's ownsystem_promptis then reinjected at the head of the first request via the [ReinjectSystemPrompt][pydantic_ai.capabilities.ReinjectSystemPrompt] capability (added automatically in this mode).'client': frontendSystemPromptParts are preserved as-is. The agent's configuredsystem_promptis not injected — the caller is fully responsible for sending it on every turn if desired. To opt into the fallback behavior anyway, addReinjectSystemPromptto the agent's capabilities.If you want per-request guidance that doesn't need to live in message history, prefer
instructionsoversystem_prompt.ReinjectSystemPromptcapabilityNew capability at [
pydantic_ai.capabilities.ReinjectSystemPrompt][pydantic_ai.capabilities.ReinjectSystemPrompt]. Ensures the agent's configuredsystem_promptis at the head of the firstModelRequeston every model request. If anySystemPromptPartis already present anywhere in the history (preserved from a prior run, handed off from another agent, or sent by a client), the capability is a no-op.Useful for any scenario where
message_historycomes from a source that doesn't round-trip system prompts — UI frontends, database persistence layers, conversation compaction pipelines. Community users have asked for this repeatedly on #1646; shipping it as an opt-in capability avoids breaking the documented contract ofagent.run(message_history=[...])while giving people a single one-line opt-in.Supporting infrastructure:
Agent.system_prompts(...)method on [AbstractAgent][pydantic_ai.agent.AbstractAgent] that resolves configured static strings + runner functions intoSystemPromptParts, reusable by anyone who needs to resolve system prompts outside an agent run.capabilities=run-time kwarg onAgent.run/Agent.iter/Agent.run_stream/Agent.run_stream_events— additive with the agent's configured capabilities (matches thetoolsets=/builtin_tools=pattern). Used internally byUIAdapterto opt into the capability in server mode.A
# TODO (v2):comment marks the capability for potential default inclusion in v2 — which would close #1646 directly, at the cost of a documented behavior change toagent.run(message_history=[...]).Behavior change (security-relevant)
Frontend-originated
SystemPromptParts are now stripped by default in UI adapter flows and a warning is emitted. Rationale: a malicious client could otherwise inject arbitrary instructions via crafted API requests. The agent's configuredsystem_promptis authoritative in this default mode and is reinjected when missing.Callers who intentionally let the frontend own the system prompt must opt in with
manage_system_prompt='client'.Pre-Review Checklist
Pre-Merge Checklist