fix: sort AI proxy upstream request JSON keys#13461
Merged
Merged
Conversation
Encode the AI proxy upstream request body with rapidjson using sort_keys=true so the serialized payload has a deterministic key order, improving provider-side LLM cache hit rate. Empty arrays are preserved via the json array metatable, and encoding falls back to cjson on any error. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
lua-rapidjson enables -march=native by default, which can leak the build host's CPU features into the shipped rapidjson.so and crash on older CPUs. Install it via a helper that strips that flag, and wire the dependency into the Makefile/CI/dev-image build paths. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
dosubot
Bot
added
the
size:L
There was a problem hiding this comment.
Pull request overview
This PR makes AI proxy upstream JSON serialization deterministic (stable key ordering) by switching from core.json.encode to rapidjson.encode(..., { sort_keys = true }), improving upstream LLM prompt-cache hit rates. It also adds the rapidjson dependency and wiring so the C module builds portably (avoiding -march=native), plus tests covering key ordering and fallback behavior.
Changes:
- Encode
ai-transport/http.luaupstream request bodies with rapidjson sorted keys, with a cjson fallback on encoding failure. - Add
rapidjson = 0.7.2-1dependency and a CI/dev install script to strip-march=nativefrom the build. - Add tests asserting sorted keys, empty-array/null preservation, and fallback behavior.
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
apisix/plugins/ai-transport/http.lua |
Switch request body encoding to rapidjson w/ sorted keys and fallback logic. |
apisix-master-0.rockspec |
Add rapidjson dependency. |
ci/install-lua-rapidjson.sh |
Install/patch lua-rapidjson to avoid -march=native portability issues. |
ci/linux_apisix_current_luarocks_runner.sh |
Run the rapidjson install/patch step in CI. |
Makefile |
Run the rapidjson install/patch step during make deps. |
docker/debian-dev/Dockerfile |
Add cmake for building lua-rapidjson in the dev image. |
t/plugin/ai-transport-http.t |
Add test coverage for sorted encoding, preservation semantics, and fallback. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
rapidjson is a CMake/C++ rock, so the clean source-install environments (ubuntu runner and the redhat/centos containers) need cmake and a C++ compiler to build it. Also add the Apache license header to the rapidjson install helper. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
Move the table-to-rapidjson conversion inside the pcall so a conversion error also falls back to cjson instead of failing the request. Make the -march=native removal fail loudly if the line is gone (so a native build can't silently ship), use a portable sed -i, and drop the ambiguous --local when installing into an explicit tree. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
membphis
approved these changes
Jun 1, 2026
shreemaan-abhishek
approved these changes
Jun 2, 2026
AlinsRan
approved these changes
Jun 2, 2026
wistefan
pushed a commit
to wistefan/apisix
that referenced
this pull request
Jun 16, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
When the AI proxy (
ai-transport/http.lua) forwards a request body to the upstream LLM provider, it currently serializes the body withcore.json.encode, which does not guarantee a stable key order. Because many providers key their server-side prompt cache on the exact request bytes, a non-deterministic key order lowers the cache hit rate for otherwise-identical requests.This PR encodes the upstream request body with
rapidjsonusingsort_keys = true, producing a deterministic key order. Details:[](viacore.json.array_mt) andcore.json.nullis mapped to JSONnull, so the payload is semantically unchanged — only key order becomes stable.cjsonon any rapidjson error and logs it, so a serialization edge case can never fail the request.rapidjson = 0.7.2-1as a dependency.A second commit makes the new dependency build portably:
lua-rapidjsonenables-march=nativeby default, which can bake the build host's CPU features into the shippedrapidjson.soand crash on older CPUs.ci/install-lua-rapidjson.shstrips that flag and is wired into the Makefile / CI runner / dev image build.New test:
t/plugin/ai-transport-http.tasserts sorted keys, empty-array preservation, and the cjson fallback path.Checklist
🤖 Generated with Claude Code