Skip to content

[BUG] Sensitive-word dictionary construction can still run on the Redis I/O thread #7830

Closed

Description

@Aias00

Current behavior

The sensitive-word plugin intends to move CPU-heavy dictionary construction away from event loops, but places subscribeOn(boundedElastic()) after a reactive Redis publisher. That moves subscription, while asynchronous Redis result emissions can still execute the downstream construction on the Redis I/O thread.

Confirmed in Apache ShenYu 2.7.2-SNAPSHOT and upstream master@7cc1818ace246f60a608e44b5fe985dff46908e7. The inspected upstream file is identical to the current working copy.

Code evidence

The dictionary cache-miss/expiry path in SensitiveWordPlugin.dictionary() is:

redisTemplate.opsForSet()
        .members(redisKey)
        .collectList()
        // building the automaton is cpu bound, keep it away from the event loop
        .map(loaded -> AhoCorasick.of(merge(loaded, words)))
        .subscribeOn(Schedulers.boundedElastic())

The later publishOn(boundedElastic()) occurs after construction. It moves subsequent request-body scanning, but cannot move work that has already executed in the preceding map. The rule-word-only fallback at line 219 also constructs an automaton before that final handoff.

Trigger: a missing/expired dictionary followed by asynchronous Redis results; the cost increases with dictionary size. CPU-heavy automaton construction can delay work sharing the Redis event-loop thread.

Minimal operator reproduction

A standalone Reactor 3.6.7 probe used the same operator ordering and a named asynchronous source to simulate the Redis emission boundary:

Flux.just("word")
        .delayElements(Duration.ofMillis(10), ioScheduler)
        .collectList()
        .map(words -> {
            System.out.println("BUILD_THREAD=" + Thread.currentThread().getName());
            return words;
        })
        .subscribeOn(Schedulers.boundedElastic())
        .publishOn(Schedulers.boundedElastic())
        .map(words -> {
            System.out.println("SCAN_THREAD=" + Thread.currentThread().getName());
            return words;
        })
        .block(Duration.ofSeconds(5));

Observed output:

BUILD_THREAD=simulated-redis-io-1
SCAN_THREAD=boundedElastic-2

This verifies the operator-placement behavior; it is not a live Redis integration test or a throughput benchmark.

Expected behavior / suggested direction

Explicitly schedule automaton construction after asynchronous Redis emissions, for example by placing an appropriate publishOn before the CPU-heavy map, or by deferring construction inside an explicitly scheduled publisher. Cover fallback construction too. Preserve the cached-path handoff needed to keep automaton.search(body) off the request event loop; simply deleting all scheduling would not address the problem.

Suggested validation

  • Verify on a named asynchronous source that dictionary construction runs on the intended worker, rather than the source emission thread.
  • Exercise miss, refresh, Redis-error fallback, and valid cached-dictionary paths.
  • Confirm request-body scanning remains off the event loop.

No production latency/throughput impact has been measured.

Activity

  1. konlue commented on Oct 18, 2026

    @konlue
    Contributor

    /assign

  2. konlue commented on Oct 21, 2026

    @konlue
    Contributor

    PTAL. This is fixed on master: the dictionary-threading change was delivered through #7850 — that PR was stacked on #7844 (both changes touch the same method), so merging it brought this fix in as well, along with its tests (testTheDictionaryIsBuiltOffTheRedisIoThreads, testTheRuleWordsAutomatonBuildIsDeferred).

    #7844 was then closed as merged-by-another-route since its diff shrank to nothing. Ready to be closed.

  3. Aias00 commented on Nov 11, 2026

    @Aias00
    ContributorAuthor

    Verified against current master (9dd9a12). Merged PR #7850 (e1f9ea8) adds publishOn(Schedulers.boundedElastic()) before AhoCorasick.of(...) in SensitiveWordPlugin, with the regression test testTheDictionaryIsBuiltOffTheRedisIoThreads. Closing as resolved.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions