Repository navigation
[ISSUE #7563] Document wasm host abi and test a gateway reference plugin. - #7567
Conversation
|
Marking this DRAFT, so I won't approve. The new Before it is ready for merge, please ensure:
Ping me when it is marked ready for a full review. |
|
@Aias00 This is now ready for a full review at 60c7d61. The branch includes master at a89459b, including #7568's optional discovery-deletion callback. The README/ABI document now distinguish the gateway test adapter's binary profile from #7569's separate JSON default bridge; the actual merged host imports and callback names are retained. Export lookup preserves #7568's policy: absent exports are optional, non-function exports are errors. The private The new |
Aias00
left a comment
There was a problem hiding this comment.
Review: document wasm host ABI and test a gateway reference plugin
This is primarily documentation + test-infrastructure, with one small and safe production change. Well done.
Production change (the only non-test code)
WasmLoader.java: the_initializecall previously wrappedinstance.export("_initialize")in a try/catch for Chicory'sInvalidException. It now uses the sharedfindWasmExtern("_initialize").ifPresent(...), andgetWasmExterndelegates to the same private helper. The helper checks export names first, so an absent optional hook stays optional while a present-but-invalid export type still surfaces as an error — exactly the documented policy inABI-v1.md("An export with a matching name but the wrong kind ... is invalid, not an absent optional hook"). Removed the now-unusedExportSectionimport. Correct and more robust.
Test infrastructure
ReferenceWasmPlugin/ReferenceWasmPluginDataHandler: coherent concrete adapters extending the existingAbstractShenyuWasmPlugin/AbstractWasmPluginDataHandlerbase classes, implementing the binary request profile documented inABI-v1.md(10-field frame, version 1, decision 0/1,-1/-2codes, ≤32 KiB body viaDataBufferUtils.join(..., 32768)).synchronizedguards the shared loader instance as the doc mandates.WasmGatewayTest: a real Spring Boot WebFlux gateway bootstrap on a random port that sends actual HTTP requests. It covers the happy allow/deny path, config update/delete (plugin/selector/rule subscribe & unsubscribe), unmatched-request bypass, 60 concurrent requests with argument isolation, missing-execute-export 5xx, optional missing hooks, guest trap / invalid memory / incompatible-result failure + recovery, and assertspendingArguments() == 0after every test (no leaked Java arguments). High-quality integration coverage.WasmLifecycleContractTest: lifecycle contract fixtures for the data handler variants..wasmbinaries are checked in (rebuilt from.watvia theMakefileusing WABT 1.0.42) so Java builds need no Wasm compiler; thepom.xmlrat exclude for*.wasmis correct (binaries can't carry license headers; their.watsources do).
CI
.github/workflows/wasm-fixtures.yml: least-privilegepermissions: contents: read,concurrencywithcancel-in-progress, pins WABT 1.0.42 with a sha256 checksum verification, and installs Rust 1.90.0. Triggers only on the relevant wasm paths (pluspom.xml)..github/scripts/verify-wasm-fixtures.py:verify_watrebuilds WAT fixtures into a temp dir and byte-compares against the committed binaries (never modifies them);verify_rustrebuilds the five Rust guests, runs the WASM Java suites, and restores the original committed bytes in afinallyblock even on failure. Idempotent and safe. Supports--wat-only/--rust-only.- Full module CI green, including
rebuild-and-test,Analyze (java),check-license-header, andrat_check.
Non-blocking note
verify_watglobswasm-bridgeandwasm-gatewayfamilies; onlywasm-gatewayexists in this PR, and the code handles a missing directory gracefully. Fine as-is.
Approving. This meaningfully improves the WASM subsystem's testability and documents a previously-undocumented contract.
wy471x
left a comment
There was a problem hiding this comment.
Reviewed 60c7d618b.
Verification (JDK 21):
shenyu-plugin-wasm-basesuite 43/43 green, including the newWasmGatewayTest(9 tests: boots the gateway starter on a random port with a registered WASM plugin and drives real HTTP requests) andWasmLifecycleContractTest(4);shenyu-plugin-wasm-apibuilds in the same reactor; checkstyle 0 violations andapache-rat:check0 unapproved on both modules.- Not reproduced on my side: the WAT byte-identity check and the rebuilt Rust guest runs. This machine has no
wat2wasm/ WABT 1.0.42 and no Rust 1.90.0 toolchain, so everything the new workflow adds there is reviewed by reading only. - The
WasmLoaderchange is behaviour-preserving for the found case and tidier for the absent one:_initializenow goes through the samefindWasmExternused bygetWasmExtern, so an absent export returnsOptional.empty()instead of relying on a caughtInvalidException. - The module is already registered in
shenyu-plugin/pom.xml; the RAT exclusion is scoped to this module and the.watsources carry license headers, so RAT stays clean.
Findings:
- Coordination with #7569 (still open, same author): the earlier review asked for a single lookup policy, landed here as a shared helper. This helper matches on export name and then calls
instance.export(name), so a non-function export still throwsInvalidException, while #7569 returnsOptional.empty()for that case. Only one of the two can be the contract - please state which, and rebase the other. Concrete edge this creates today: a guest exporting a global or memory named_initializeused to be caught and ignored (debug log) and would now propagate out of the loader constructor. - The new CI job rebuilds WAT fixtures and Rust guests for any PR touching
pom.xml,shenyu-plugin-wasm-api/**orshenyu-plugin-wasm-base/**, with a 30 minute timeout and no cargo / WABT caching.cargo build --offlineworks only because all fiverust-*fixtures have zero[dependencies]; the first future dependency turns that step red on a fresh runner. Either drop--offlineor cache.cargo/registry. verify-wasm-fixtures.pywalks the families["wasm-bridge", "wasm-gateway"]undershenyu-plugin-wasm-base/src/test, but onlywasm-gatewayexists -wasm-bridgeis skipped silently byif not directory.exists(): continue. Either drop the name or assert that an expected family was found.verify_rust()rewrites tracked fixtures in the working tree and restores them infinally, which is fine for CI but leavessrc/test/resources/*.wasmmodified if a local run is aborted. A line in the README saying to run it from a clean checkout would cover that.- Pinning WABT 1.0.42 by sha256 and Rust 1.90.0 is good practice;
actions/checkout@v3matches most workflows in this repo.
Nit: findWasmExtern has no javadoc while getWasmExtern does, and the name-first-then-export ordering is the entire point of the helper - one line of javadoc would keep the next reader from simplifying it back to a bare instance.export(name).
Fixes #7563.
Document host ABI v1 and an explicit gateway reference profile. A Spring Boot gateway test registers a Java adapter and a real Wasm guest, then verifies HTTP requests, matched selector/rule DTOs, guest decisions, chain continuation, updates/deletions, concurrency and failure isolation. WAT sources and resource-name mappings are included.
The branch includes master at
a89459bf2, including #7568's discovery-deletion callback. It preserves the merged export policy: missing exports are optional; wrong export kinds fail. The private lookup helper also validates_initialize, fixing the catch-and-ignore behavior demonstrated by the invalid-initializer regression. The gateway test's binary profile is distinct from the JSON default bridge in #7569.The
wasm-fixturesworkflow pins WABT 1.0.42 and Rust 1.90.0, verifies every available WAT fixture byte-for-byte, rebuilds the five Rust guests and runs the Java suites against them. Original resource bytes are restored even after a failed build/test. Legacy compiler versions were not recorded, so Rust reproduction is verified by behavior; existing TinyGo fixtures also run in the suites.Validation on
60c7d618b:The gateway uses the normal gateway starter on a random HTTP port. Configuration enters through the gateway-side subscriber; Admin persistence and remote sync transports are outside this test's scope.