Repository navigation
Description
Current behavior
The sensitive-word plugin intends to move CPU-heavy dictionary construction away from event loops, but places subscribeOn(boundedElastic()) after a reactive Redis publisher. That moves subscription, while asynchronous Redis result emissions can still execute the downstream construction on the Redis I/O thread.
Confirmed in Apache ShenYu 2.7.2-SNAPSHOT and upstream master@7cc1818ace246f60a608e44b5fe985dff46908e7. The inspected upstream file is identical to the current working copy.
Code evidence
The dictionary cache-miss/expiry path in SensitiveWordPlugin.dictionary() is:
redisTemplate.opsForSet()
.members(redisKey)
.collectList()
// building the automaton is cpu bound, keep it away from the event loop
.map(loaded -> AhoCorasick.of(merge(loaded, words)))
.subscribeOn(Schedulers.boundedElastic())The later publishOn(boundedElastic()) occurs after construction. It moves subsequent request-body scanning, but cannot move work that has already executed in the preceding map. The rule-word-only fallback at line 219 also constructs an automaton before that final handoff.
Trigger: a missing/expired dictionary followed by asynchronous Redis results; the cost increases with dictionary size. CPU-heavy automaton construction can delay work sharing the Redis event-loop thread.
Minimal operator reproduction
A standalone Reactor 3.6.7 probe used the same operator ordering and a named asynchronous source to simulate the Redis emission boundary:
Flux.just("word")
.delayElements(Duration.ofMillis(10), ioScheduler)
.collectList()
.map(words -> {
System.out.println("BUILD_THREAD=" + Thread.currentThread().getName());
return words;
})
.subscribeOn(Schedulers.boundedElastic())
.publishOn(Schedulers.boundedElastic())
.map(words -> {
System.out.println("SCAN_THREAD=" + Thread.currentThread().getName());
return words;
})
.block(Duration.ofSeconds(5));Observed output:
BUILD_THREAD=simulated-redis-io-1
SCAN_THREAD=boundedElastic-2
This verifies the operator-placement behavior; it is not a live Redis integration test or a throughput benchmark.
Expected behavior / suggested direction
Explicitly schedule automaton construction after asynchronous Redis emissions, for example by placing an appropriate publishOn before the CPU-heavy map, or by deferring construction inside an explicitly scheduled publisher. Cover fallback construction too. Preserve the cached-path handoff needed to keep automaton.search(body) off the request event loop; simply deleting all scheduling would not address the problem.
Suggested validation
- Verify on a named asynchronous source that dictionary construction runs on the intended worker, rather than the source emission thread.
- Exercise miss, refresh, Redis-error fallback, and valid cached-dictionary paths.
- Confirm request-body scanning remains off the event loop.
No production latency/throughput impact has been measured.
Activity
konlue commentedon Oct 18, 2026
/assign
perf(ai): parse the rule words only on a dictionary cache miss (#7829) (
Merge master into fix/sensitive-word-io-thread to fix CI (apache#7830)
konlue commentedon Oct 21, 2026
PTAL. This is fixed on
master: the dictionary-threading change was delivered through #7850 — that PR was stacked on #7844 (both changes touch the same method), so merging it brought this fix in as well, along with its tests (testTheDictionaryIsBuiltOffTheRedisIoThreads,testTheRuleWordsAutomatonBuildIsDeferred).#7844 was then closed as merged-by-another-route since its diff shrank to nothing. Ready to be closed.
Aias00 commentedon Nov 11, 2026
Verified against current master (9dd9a12). Merged PR #7850 (e1f9ea8) adds
publishOn(Schedulers.boundedElastic())beforeAhoCorasick.of(...)inSensitiveWordPlugin, with the regression testtestTheDictionaryIsBuiltOffTheRedisIoThreads. Closing as resolved.