Repository navigation
[ISSUE #7871] Separate response adaptation from upstream request model - #7876
Conversation
Aias00
left a comment
There was a problem hiding this comment.
Clean separation, and it fixes a real modelling problem: the response adapter was a mutable field on AiUpstreamRequest, so a request could reach the engine without its adapter and silently be converted by Function.identity(). Approving.
What I verified
1. Both directions stay binary-compatible. The interface default prepareRequest → buildRequest + AiResponseAdapter.identity() keeps every existing provider working (pinned by testExistingProviderReceivesIdentityResponseAdapter), and OpenAi.buildRequest delegating to prepareRequest keeps direct buildRequest callers working. OpenAiCompatible forwards both, so the delegate never loses the adapter.
2. The construction order is now explicit and pinned. provider.adaptResponse → prepareRequest-bound adapter → protocol.encodeResponse used to be one nested expression; AiProxyEngineOrchestrationTest#providerRequestAdapterAndProtocolConvertTheResponseInOrder asserting List.of("provider", "request", "protocol") is exactly the test this kind of reorder needs, and it also checks no conversion happens on the failure path.
3. PreparedAiRequest closes the original failure mode. With Objects.requireNonNull on both components there is no state in which a request exists without an adapter, which is what AiUpstreamRequest.setResponseAdapter allowed.
4. Request-scoped state is still captured correctly. OpenAi.prepareRequest binds adapter and stream into the lambda, and OpenAiProviderOptionsTest exercises both chat and responses protocols through prepareRequest(...).responseAdapter(), so the pair cannot drift from the body that was actually built.
Non-blocking suggestions
-
The two-method contract invites recursion.
prepareRequestis a default that callsbuildRequest, whileOpenAiimplementsbuildRequestby callingprepareRequest. It is safe today, but a subclass that overrides one and calls the other loops with nothing to warn it at compile time. Cleaner: makeprepareRequestthe single abstract method and adddefault AiUpstreamRequest buildRequest(...) { return prepareRequest(...).request(); }, then@DeprecatedbuildRequestfor removal.OpenAiwould carry one implementation, and no override could drop the adapter. -
buildRequestnow builds an adapter and throws it away. That is harmless forOpenAi(the closure is cheap), but as long asbuildRequestremains a first-class way to obtain a request, an adapter-aware provider can be used through it and be converted by the identity function. RemovingadaptResponsefromAiUpstreamRequestturned the old silent bug into a compile error, which is good; deprecatingbuildRequestin the same spirit would finish the job. -
Document the adapter's contract, not just its type. The javadoc explains the default behaviour, but the part a new provider implementer will get wrong is the contract: the adapter must capture the same
streamdecision used to build the body, must be invocable once per response, and must not mutate the request.OpenAidoes this correctly — it is worth stating it so the next provider does too. -
Nit:
AiResponseAdapterandPreparedAiRequestsit in different packages (conversionvsmodel) although the record is really part of the conversion contract. Trivial, but the record would read better next to the interface. -
Nit: if
AiResponseAdapteris only ever going to be "a function from response to response with a null-safe default", aFunction<AiUpstreamResponse, AiUpstreamResponse>field defaulted in the record's compact constructor would do the same job with less surface. Keeping the named interface makes sense only if you expect to add more semantics (andThen, capability flags); otherwise it is an indirection without a payload.
CI
pr_build passes, 18 pass / 10 skipping, no failures. The it workflow did not run for this PR (plugin-only change), so no e2e signal — acceptable for a refactor with no runtime behaviour change, but worth a manual gateway smoke test before merge if you have one handy.
Part of #7871
Summary
PreparedAiRequestto carry the upstream request and its request-scoped response adapterAiUpstreamRequestlimited to transport request data and preserve compatibility for existing providers with an identity adapterTests
Tests run: 67, Failures: 0, Errors: 0, Skipped: 0. Checkstyle passed.
Make sure that:
./mvnw clean install -Dmaven.javadoc.skip=true.