Skip to content

[Task][AI] Expose a bounded normalized request view before governance #8255

Closed

Description

@Aias00

Problem and ownership

Parent: #8055. Reviewed master 8046953ca has protocol decoding/adaptation inside AiProxyEngine.prepare, after independently ordered prompt, token and safety plugins. Those components do not yet share one explicit bounded request interpretation and post-transformation invalidation contract.

Define the smallest reusable public contract for supported AI governance stages. This is a design-first integration task under the existing protocol work (#7382/#7351), not another protocol stack or plugin/workflow graph engine.

Scope

  • Start with an explicitly documented Chat/Responses coverage matrix. Reuse native protocol decoding and preserve opaque/unsupported fields according to existing passthrough semantics; unsupported inspection must remain visible.
  • Expose the bounded structured view and required governance metadata before its consumers execute. Define who reads/caches/replays the body once, byte/depth/collection limits, immutable ownership and terminal cleanup. Do not put request content in the body-free AgentRequestContext, trace/audit events or global maps.
  • Define original wire representation versus effective post-prompt/post-transform view. Every consumer states which view it uses; mutation creates/invalidates a scoped generation so a token estimate cannot silently use pre-template text while routing/safety see a different request.
  • Separate deterministic prompt decoration ([Feature][AI] Add versioned prompt templates with validation, rollout and rollback #8051) from transformations that invoke a model. Keep protocol fields, tool calls, structured output and modality content intact; do not flatten arbitrary structured content into text.
  • Specify phase/ordering and compatibility validation through the existing plugin chain and Admin sorts. A misordered/unsupported consumer cannot report successful inspection. Metadata such as candidate model information may be resolved before model-specific estimation; final selection still checks budget/capabilities.
  • Reuse [Feature] Share an end-to-end request deadline across plugins and upstream attempts #7931/[Task][AI Proxy] Apply a shared request deadline across AI attempts, backoff and fallback #8121 deadline propagation and existing authenticated identity. Gateways use synchronized configuration/caches, not Admin DB reads.

Acceptance

  • Review a contract and source-to-consumer matrix showing parser/view ownership, supported protocols, actual phase and release boundaries.
  • A real prompt -> safety -> budget-inspection -> proxy fixture observes the same effective request generation; modifying prompt variables invalidates prior interpretations predictably.
  • Tests cover missing/malformed/oversized input, role/tool preservation, split buffers, non-matching traffic and supported/unsupported representations.
  • One request-body subscription and bounded retained memory are demonstrated; no full-body copies/reparses are added without justified measured need.
  • Concurrent requests, rule updates, failure/cancellation and streaming response lifetime cannot leak data or mutate another request's snapshot.
  • Existing Proxy-only and disabled/unmatched governance behavior remains compatible; deterministic before/after measurements state ordinary-path overhead.

Related boundaries

#7382 owns native protocol contracts; #8060 owns transactional configuration revisions; #8010 owns HTTP plugin-graph role validation; #7950 owns body-free auditing; #8113 owns AI workflow composition. Reuse these contracts rather than creating competing ownership.

Activity

  1. konlue commented on Nov 8, 2026

    @konlue
    Contributor

    /assign

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions