feat(wasm): add opt-in execution deadlines and memory budgets - #7948

Review

Solid piece of work. The hard part of this problem is that a Java thread cannot be forcibly stopped, so "cancel the guest" is not something you can implement — you can only interrupt and then decide what to do with the instance and the worker. This PR makes the right call on all three of those decisions.

What I verified

1. The nested-call short-circuit is what prevents self-deadlock

if (timeoutMillis == 0 || Objects.nonNull(cancellation.get())) {
    return task.call();
}

Without the cancellation.get() != null check, a guest that calls back into an export would submit a task to the same bounded pool and then block waiting for it — with core = max = poolSize and a queue of poolSize, that deadlocks as soon as enough workers are occupied. Running nested calls inline on the worker thread is correct.

2. Interruption-ignoring hosts keep their worker, and that is the right trade-off

// Guest execution is cooperative: interruption-ignoring host calls retain their
// bounded worker until physical exit. Never replace those threads on timeout.

Replacing the thread would let a guest that ignores interrupts create unbounded workers — a slow-motion OOM. Retaining the worker and surfacing RejectedExecutionException as ShenyuWasmException("... capacity exhausted ...") fails closed, which is the correct choice for a gateway. interruptionIgnoringHostCannotCreateReplacementWorkerCapacity pins exactly this, including the calls.get() == 1 assertion that proves the first host call is still occupying its slot.

3. The pool stays balanced across the replacement path

executeOnPool acquires one instance, and every exit path returns exactly one:

• replacement succeeds → old instance is dropped and release(replacement) puts one back
• replacement throws → replaced stays false, so release(acquired) returns the old (still-poisoned) instance, and poisonedInstances still contains it so the next call retries the replacement
• normal path → release(lease)

So freeInstances never loses or gains a slot, and release's offer failing loudly on a full queue is a good corruption guard.

4. Poisoning is scoped to calls that actually went through the deadline path

AtomicBoolean cancelled = cancellation.get();
if (Objects.nonNull(cancelled) && (Objects.nonNull(failure) || cancelled.get() || Thread.currentThread().isInterrupted())) {
    poisonedInstances.add(lease);
}

cancellation is only set on a runLimited worker, so nested inline calls (which never see a cancel signal) do not poison instances on ordinary failures. That preserves the pre-existing behaviour for deployments that have not configured a deadline.

5. Memory ceiling cannot silently raise or lower anything

int maximum = Math.min(declared.maximumPages(), limits.getMaxMemoryPages());
if (declared.initialPages() > maximum || declared.shared()) {
    throw new ShenyuWasmInitException("WASM initial/shared memory is incompatible with the configured memory budget");
}
return new ByteBufferMemory(new MemoryLimits(declared.initialPages(), maximum));

min means the budget can only tighten, never loosen. Refusing when initialPages > maximum instead of clamping avoids the failure mode where a module starts with less memory than it declares and then traps on the first allocation. Refusing shared is right — a shared memory cannot be bounded per instance. And the constructor-level guard requires exactly one module-owned memory, which is the only layout where the budget is actually enforceable.

6. Reactor offload is correctly conditional

if (wasmLoader.hasExecutionDeadline()) {
    return Mono.defer(() -> executeWasm(exchange, chain, selector, rule)).subscribeOn(Schedulers.boundedElastic());
}
return executeWasm(exchange, chain, selector, rule);

Mono.defer plus subscribeOn means the blocking guest call happens on boundedElastic, not on a Netty event loop — which is the whole point of having a deadline, since a timeout you cannot reach because the event loop is blocked is useless. Keeping the old inline path when no deadline is configured means zero behavioural change for existing deployments. Note this keys off timeoutMillis only, not initializationTimeoutMillis, which is right: initialization happens at startup, not per request.

7. Test coverage targets the failure modes, not the happy path

infiniteGuestStopsAndItsInstanceIsReplaced (timeout → instance replaced → next call returns 7, and the memory read is back to 0 so the replacement really is fresh), interruptionCancelsTheActualGuestWorker (asserts the worker exited, with the comment "cancelling only the caller is insufficient" — that is the assertion most people forget), memoryGrowthCannotExceedTheConfiguredBudget (grow 1 → 1, second grow 1 → -1, so the ceiling is enforced by the runtime and not just by the factory), and the .wat sources plus Makefile mean the binaries are reproducible rather than opaque blobs. Good.

Non-blocking

1. fromSystemProperties throws NumberFormatException with no context. Long.parseLong(property(...)) on a malformed -Dshenyu.wasm.max-memory-pages=8m fails deep in construction with a message that names neither the property nor the value. Wrapping it in an IllegalArgumentException that includes the key and value would save operators a lot of time.
2. A module with no memory section is rejected when a budget is configured. module.memorySection().map(section -> section.memoryCount() != 1).orElse(true) treats "no memory at all" the same as "more than one memory". A memory-free module cannot exceed a memory budget by definition, so orElse(false) (or an explicit empty-section check) would be more permissive without weakening the guarantee.
3. runLimited allocates a FutureTask per call on the deadline path. That is per-request garbage when a deadline is configured, but it is opt-in, the allocation is small, and avoiding it would need a hand-rolled task wrapper. Fine to leave; mentioning it only so it is a conscious choice.
4. docs/wasm-execution-limits.md should state the worker-retention behaviour explicitly. The rollback guidance is there, but the fact that an interrupt-ignoring host permanently consumes one worker (and therefore that poolSize effectively caps how many such guests you can have) is the single most surprising operational property of this design.

CI

All 12 checks are QUEUED, so the new wasm tests have not run on CI yet. They are the substance of this PR, so I want to see them green before merge. Approving on the code.

Re-review after becc542f: opt-in, cooperative deadlines, and the memory-free case is handled

The shape of this change is sound for what it claims: nothing is enabled unless shenyu.wasm.* is set, and when it is set the limits are resolved once at construction time and never mutated.

What I verified specifically:

• Default is a no-op. WasmExecutionLimits.fromSystemProperties defaults to 0 / 0 / 0, runLimited short-circuits to a plain task.call() when the timeout is 0, and executionExecutor is not even created unless a deadline is configured. So a deployment that does not opt in keeps the previous execution path exactly.
• Cooperative cancellation, honestly documented. close() cancels active FutureTasks and calls shutdownNow(), and the comment says out loud that interruption-ignoring host calls keep their bounded worker until physical exit and are never replaced. That is the honest statement for a pure-Java runtime; pretending otherwise would be worse.
• Poisoning after a timeout. The ThreadLocal<AtomicBoolean> cancellation is set inside the worker task and read in executeOnPool's finally, so an instance that was interrupted mid-call is not returned to the pool as if it were clean. poisonedInstances plus the replace-on-acquire path (create, initialize, swap, and release the poisoned lease if the replacement fails) is the right recovery, and the replacement happens on the worker rather than on the caller.
• Memory budget. withMemoryFactory clamps the declared maximum to maxMemoryPages, rejects initialPages > maximum and rejects shared() memory; a module that imports memory or declares more than one memory is rejected at load time with a clear message. That is a real budget, not a hint.
• Memory-free guests. Dropping the unconditional instance.memory() == null throw at construction and moving it into getMemory() is the correct trade: a guest without a linear memory is legal, and only callers that actually need it should fail. The new MemoryFreeGuest test pins it.
• Interpreter fallback still works. The catch (LinkageError | RuntimeException) re-throws ShenyuWasmException first, so a construction timeout or a memory-budget rejection is not misread as "compiler unavailable" and retried on the interpreter.

Non-blocking:

1. When a deadline is configured, AbstractWasmPlugin.execute hops to Schedulers.boundedElastic() and runLimited then hops to its own executionExecutor, so each request crosses two thread boundaries, and the blocking future.get sits on a boundedElastic thread. One hop would do: run the export directly on executionExecutor and drop the subscribeOn.
2. poisonedInstances only drops an entry when the instance is re-acquired and replaced successfully. A poisoned instance that is never leased again stays in the set for the loader's lifetime - bounded by pool size, so harmless, but an explicit clear in close() would keep it tidy.
3. docs/wasm-execution-limits.md should call out the third behaviour change explicitly: a guest that previously failed at startup because it had no memory now starts and fails at the first getMemory() call instead. That is a different failure point for plugin authors.
4. Same shared-CI-commit issue as the sibling PRs: this branch carries the cryptor / SDK-route waits. Rebase after #7970 lands.

Approved. The opt-in WASM execution limits are implemented correctly: WasmLoader builds WasmExecutionLimits from (optionally per-plugin) system properties, runs deadline-bound work on an owned executor via runLimited() (future.get(timeoutMillis)), and caps linear memory to min(declared maximum, maxMemoryPages) - rejecting incompatible shared/initial memory and more than one module-owned memory. Offloading to a bounded executor when a deadline is set is the right call since guest execution is blocking. Validation (nonnegative deadlines, pages 0-65536) fails construction on bad config. The real compiler/interpreter guest tests cover deadlines, cancellation, and memory-budget enforcement. Defaults stay disabled, preserving trusted deployments. Good.