Skip to content

[ISSUE #7871] Separate response adaptation from upstream request model - #7876

Merged
Aias00 merged 1 commit into
apache:masterfrom
hengyuss:refactor/separate_reponse_adapter_from_upsteam_request
Oct 21, 2026
Merged

Aias00 merged 1 commit into
apache:masterfrom
hengyuss:refactor/separate_reponse_adapter_from_upsteam_request

Conversation

@hengyuss

Copy link
Copy Markdown
Contributor

Part of #7871

Summary

  • introduce PreparedAiRequest to carry the upstream request and its request-scoped response adapter
  • keep AiUpstreamRequest limited to transport request data and preserve compatibility for existing providers with an identity adapter
  • apply provider, request-scoped, and client protocol response conversions in an explicit order
  • cover legacy providers, OpenAI-compatible conversion, and orchestration ordering

Tests

./mvnw -pl shenyu-plugin/shenyu-plugin-ai/shenyu-plugin-ai-proxy -am -Dtest=AiProxyProviderFactoryTest,OpenAiProviderOptionsTest,AiProxyEngineOrchestrationTest -Dsurefire.failIfNoSpecifiedTests=false test -Drat.skip=true -Djacoco.skip=true

Tests run: 67, Failures: 0, Errors: 0, Skipped: 0. Checkstyle passed.

Make sure that:

  • You have read the contribution guidelines.
  • You submit test cases (unit or integration tests) that back your changes.
  • Your local test passed ./mvnw clean install -Dmaven.javadoc.skip=true.

@Aias00 Aias00 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Clean separation, and it fixes a real modelling problem: the response adapter was a mutable field on AiUpstreamRequest, so a request could reach the engine without its adapter and silently be converted by Function.identity(). Approving.

What I verified

1. Both directions stay binary-compatible. The interface default prepareRequest → buildRequest + AiResponseAdapter.identity() keeps every existing provider working (pinned by testExistingProviderReceivesIdentityResponseAdapter), and OpenAi.buildRequest delegating to prepareRequest keeps direct buildRequest callers working. OpenAiCompatible forwards both, so the delegate never loses the adapter.

2. The construction order is now explicit and pinned. provider.adaptResponse → prepareRequest-bound adapter → protocol.encodeResponse used to be one nested expression; AiProxyEngineOrchestrationTest#providerRequestAdapterAndProtocolConvertTheResponseInOrder asserting List.of("provider", "request", "protocol") is exactly the test this kind of reorder needs, and it also checks no conversion happens on the failure path.

3. PreparedAiRequest closes the original failure mode. With Objects.requireNonNull on both components there is no state in which a request exists without an adapter, which is what AiUpstreamRequest.setResponseAdapter allowed.

4. Request-scoped state is still captured correctly. OpenAi.prepareRequest binds adapter and stream into the lambda, and OpenAiProviderOptionsTest exercises both chat and responses protocols through prepareRequest(...).responseAdapter(), so the pair cannot drift from the body that was actually built.

Non-blocking suggestions

  1. The two-method contract invites recursion. prepareRequest is a default that calls buildRequest, while OpenAi implements buildRequest by calling prepareRequest. It is safe today, but a subclass that overrides one and calls the other loops with nothing to warn it at compile time. Cleaner: make prepareRequest the single abstract method and add default AiUpstreamRequest buildRequest(...) { return prepareRequest(...).request(); }, then @Deprecated buildRequest for removal. OpenAi would carry one implementation, and no override could drop the adapter.

  2. buildRequest now builds an adapter and throws it away. That is harmless for OpenAi (the closure is cheap), but as long as buildRequest remains a first-class way to obtain a request, an adapter-aware provider can be used through it and be converted by the identity function. Removing adaptResponse from AiUpstreamRequest turned the old silent bug into a compile error, which is good; deprecating buildRequest in the same spirit would finish the job.

  3. Document the adapter's contract, not just its type. The javadoc explains the default behaviour, but the part a new provider implementer will get wrong is the contract: the adapter must capture the same stream decision used to build the body, must be invocable once per response, and must not mutate the request. OpenAi does this correctly — it is worth stating it so the next provider does too.

  4. Nit: AiResponseAdapter and PreparedAiRequest sit in different packages (conversion vs model) although the record is really part of the conversion contract. Trivial, but the record would read better next to the interface.

  5. Nit: if AiResponseAdapter is only ever going to be "a function from response to response with a null-safe default", a Function<AiUpstreamResponse, AiUpstreamResponse> field defaulted in the record's compact constructor would do the same job with less surface. Keeping the named interface makes sense only if you expect to add more semantics (andThen, capability flags); otherwise it is an indirection without a payload.

CI

pr_build passes, 18 pass / 10 skipping, no failures. The it workflow did not run for this PR (plugin-only change), so no e2e signal — acceptable for a refactor with no runtime behaviour change, but worth a manual gateway smoke test before merge if you have one handy.

@Aias00
Aias00 merged commit 143070d into apache:master Oct 21, 2026
28 checks passed
@Aias00 Aias00 added this to the 2.7.2 milestone Nov 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants