Skip to content

[ISSUE #7563] Document wasm host abi and test a gateway reference plugin. - #7567

Merged
Aias00 merged 3 commits into
apache:masterfrom
yangjj-iso:fix/7563-wasm-abi-gateway
Oct 13, 2026
Merged

Aias00 merged 3 commits into
apache:masterfrom
yangjj-iso:fix/7563-wasm-abi-gateway

Conversation

@yangjj-iso

@yangjj-iso yangjj-iso commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #7563.

Document host ABI v1 and an explicit gateway reference profile. A Spring Boot gateway test registers a Java adapter and a real Wasm guest, then verifies HTTP requests, matched selector/rule DTOs, guest decisions, chain continuation, updates/deletions, concurrency and failure isolation. WAT sources and resource-name mappings are included.

The branch includes master at a89459bf2, including #7568's discovery-deletion callback. It preserves the merged export policy: missing exports are optional; wrong export kinds fail. The private lookup helper also validates _initialize, fixing the catch-and-ignore behavior demonstrated by the invalid-initializer regression. The gateway test's binary profile is distinct from the JSON default bridge in #7569.

The wasm-fixtures workflow pins WABT 1.0.42 and Rust 1.90.0, verifies every available WAT fixture byte-for-byte, rebuilds the five Rust guests and runs the Java suites against them. Original resource bytes are restored even after a failed build/test. Legacy compiler versions were not recorded, so Rust reproduction is verified by behavior; existing TinyGo fixtures also run in the suites.

Validation on 60c7d618b:

  • 47 WASM tests pass against the freshly rebuilt Rust guests, including the gateway/lifecycle tests; zero failures/errors/skips.
  • All 11 WAT binaries reproduce exactly; the verifier also rejects a corrupted binary.
  • Checkstyle, Apache RAT and the CI-pinned license checker pass.
  • No current merge conflicts with master. GitHub CI must validate the updated head.
python3 .github/scripts/verify-wasm-fixtures.py
./mvnw -B -ntp -pl shenyu-plugin/shenyu-plugin-wasm-api,shenyu-plugin/shenyu-plugin-wasm-base apache-rat:check

The gateway uses the normal gateway starter on a random HTTP port. Configuration enters through the gateway-side subscriber; Admin persistence and remote sync transports are outside this test's scope.

  • Read the contribution guidelines.
  • Add regression coverage.
  • Full-repository install (validation scoped to affected modules and dependencies).

@Aias00

Aias00 commented Oct 12, 2026

Copy link
Copy Markdown
Contributor

Marking this DRAFT, so I won't approve. The new shenyu-plugin-wasm-base/README.md documents the WASM host ABI (request/config/result bridge and the discovery callback contract) and the test guests (go-discovery-handler, rust-discovery-handler) are updated, which is good foundational documentation for the WASM plugin series.

Before it is ready for merge, please ensure:

  1. The README and the actual exported symbols stay in sync as the sibling PRs ([ISSUE #7562] Forward wasm discovery upstream deletion events. #7568, [ISSUE #7561] Add reusable request, configuration and result bridge for wasm base plugins. #7569) land — they evolve together, so merge order matters.
  2. The referenced wat2wasm / cargo fixture-regeneration steps are captured in CI so the committed .wasm binaries remain reproducible.
  3. Any interface methods referenced by the docs (e.g. DiscoveryUpstreamDataHandler.removeDiscoveryUpstreamData, the bridge get_args/put_result contract) are present on the merged base.

Ping me when it is marked ready for a full review.

@yangjj-iso
yangjj-iso marked this pull request as ready for review October 13, 2026 01:33
@yangjj-iso

Copy link
Copy Markdown
Contributor Author

@Aias00 This is now ready for a full review at 60c7d61.

The branch includes master at a89459b, including #7568's optional discovery-deletion callback. The README/ABI document now distinguish the gateway test adapter's binary profile from #7569's separate JSON default bridge; the actual merged host imports and callback names are retained. Export lookup preserves #7568's policy: absent exports are optional, non-function exports are errors. The private findWasmExtern helper also validates _initialize; the existing invalid-initializer regression fails with the old catch-and-ignore behavior and passes with the helper.

The new wasm-fixtures CI workflow pins WABT 1.0.42 and Rust 1.90.0. Locally it reproduced all 11 gateway/failure WAT binaries byte-for-byte, rebuilt all five Rust guests and ran the WASM suites against those freshly built resources: 47 tests passed with no failures/errors/skips. The script restores original resources even on failure; legacy Rust binaries are validated by behavior because their original compiler versions were not recorded. Checkstyle, Apache RAT and the pinned license checker pass. New CI is running on this head.

@Aias00 Aias00 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review: document wasm host ABI and test a gateway reference plugin

This is primarily documentation + test-infrastructure, with one small and safe production change. Well done.

Production change (the only non-test code)

  • WasmLoader.java: the _initialize call previously wrapped instance.export("_initialize") in a try/catch for Chicory's InvalidException. It now uses the shared findWasmExtern("_initialize").ifPresent(...), and getWasmExtern delegates to the same private helper. The helper checks export names first, so an absent optional hook stays optional while a present-but-invalid export type still surfaces as an error — exactly the documented policy in ABI-v1.md ("An export with a matching name but the wrong kind ... is invalid, not an absent optional hook"). Removed the now-unused ExportSection import. Correct and more robust.

Test infrastructure

  • ReferenceWasmPlugin / ReferenceWasmPluginDataHandler: coherent concrete adapters extending the existing AbstractShenyuWasmPlugin / AbstractWasmPluginDataHandler base classes, implementing the binary request profile documented in ABI-v1.md (10-field frame, version 1, decision 0/1, -1/-2 codes, ≤32 KiB body via DataBufferUtils.join(..., 32768)). synchronized guards the shared loader instance as the doc mandates.
  • WasmGatewayTest: a real Spring Boot WebFlux gateway bootstrap on a random port that sends actual HTTP requests. It covers the happy allow/deny path, config update/delete (plugin/selector/rule subscribe & unsubscribe), unmatched-request bypass, 60 concurrent requests with argument isolation, missing-execute-export 5xx, optional missing hooks, guest trap / invalid memory / incompatible-result failure + recovery, and asserts pendingArguments() == 0 after every test (no leaked Java arguments). High-quality integration coverage.
  • WasmLifecycleContractTest: lifecycle contract fixtures for the data handler variants.
  • .wasm binaries are checked in (rebuilt from .wat via the Makefile using WABT 1.0.42) so Java builds need no Wasm compiler; the pom.xml rat exclude for *.wasm is correct (binaries can't carry license headers; their .wat sources do).

CI

  • .github/workflows/wasm-fixtures.yml: least-privilege permissions: contents: read, concurrency with cancel-in-progress, pins WABT 1.0.42 with a sha256 checksum verification, and installs Rust 1.90.0. Triggers only on the relevant wasm paths (plus pom.xml).
  • .github/scripts/verify-wasm-fixtures.py: verify_wat rebuilds WAT fixtures into a temp dir and byte-compares against the committed binaries (never modifies them); verify_rust rebuilds the five Rust guests, runs the WASM Java suites, and restores the original committed bytes in a finally block even on failure. Idempotent and safe. Supports --wat-only/--rust-only.
  • Full module CI green, including rebuild-and-test, Analyze (java), check-license-header, and rat_check.

Non-blocking note

  • verify_wat globs wasm-bridge and wasm-gateway families; only wasm-gateway exists in this PR, and the code handles a missing directory gracefully. Fine as-is.

Approving. This meaningfully improves the WASM subsystem's testability and documents a previously-undocumented contract.

@Aias00
Aias00 merged commit 93cff09 into apache:master Oct 13, 2026
28 checks passed

@wy471x wy471x left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 60c7d618b.

Verification (JDK 21):

  • shenyu-plugin-wasm-base suite 43/43 green, including the new WasmGatewayTest (9 tests: boots the gateway starter on a random port with a registered WASM plugin and drives real HTTP requests) and WasmLifecycleContractTest (4); shenyu-plugin-wasm-api builds in the same reactor; checkstyle 0 violations and apache-rat:check 0 unapproved on both modules.
  • Not reproduced on my side: the WAT byte-identity check and the rebuilt Rust guest runs. This machine has no wat2wasm / WABT 1.0.42 and no Rust 1.90.0 toolchain, so everything the new workflow adds there is reviewed by reading only.
  • The WasmLoader change is behaviour-preserving for the found case and tidier for the absent one: _initialize now goes through the same findWasmExtern used by getWasmExtern, so an absent export returns Optional.empty() instead of relying on a caught InvalidException.
  • The module is already registered in shenyu-plugin/pom.xml; the RAT exclusion is scoped to this module and the .wat sources carry license headers, so RAT stays clean.

Findings:

  • Coordination with #7569 (still open, same author): the earlier review asked for a single lookup policy, landed here as a shared helper. This helper matches on export name and then calls instance.export(name), so a non-function export still throws InvalidException, while #7569 returns Optional.empty() for that case. Only one of the two can be the contract - please state which, and rebase the other. Concrete edge this creates today: a guest exporting a global or memory named _initialize used to be caught and ignored (debug log) and would now propagate out of the loader constructor.
  • The new CI job rebuilds WAT fixtures and Rust guests for any PR touching pom.xml, shenyu-plugin-wasm-api/** or shenyu-plugin-wasm-base/**, with a 30 minute timeout and no cargo / WABT caching. cargo build --offline works only because all five rust-* fixtures have zero [dependencies]; the first future dependency turns that step red on a fresh runner. Either drop --offline or cache .cargo/registry.
  • verify-wasm-fixtures.py walks the families ["wasm-bridge", "wasm-gateway"] under shenyu-plugin-wasm-base/src/test, but only wasm-gateway exists - wasm-bridge is skipped silently by if not directory.exists(): continue. Either drop the name or assert that an expected family was found.
  • verify_rust() rewrites tracked fixtures in the working tree and restores them in finally, which is fine for CI but leaves src/test/resources/*.wasm modified if a local run is aborted. A line in the README saying to run it from a clean checkout would cover that.
  • Pinning WABT 1.0.42 by sha256 and Rust 1.90.0 is good practice; actions/checkout@v3 matches most workflows in this repo.

Nit: findWasmExtern has no javadoc while getWasmExtern does, and the name-first-then-export ordering is the entire point of the helper - one line of javadoc would keep the next reader from simplifying it back to a bare instance.export(name).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Task] Document the WASM host ABI and add a gateway-level reference plugin test

3 participants