Skip to content

[Task][Performance] Avoid reparsing rule words on sensitive-word dictionary cache hits #7829

Closed

Description

@Aias00

Description

SensitiveWordPlugin.dictionary() reparses the configured rule words on every inspected request, even when the cached dictionary is valid and the parsed words will not be used.

Confirmed in Apache ShenYu 2.7.2-SNAPSHOT and upstream master@7cc1818ace246f60a608e44b5fe985dff46908e7. The inspected upstream file is identical to the current working copy.

Code evidence and trigger

In dictionary(), parseWords(handle.getWords()) executes before looking up the cached dictionary and checking its expiry:

List<String> words = parseWords(handle.getWords());
CachedDictionary cached = SensitiveWordPluginDataHandler.DICTIONARIES.get().obtainHandle(dictionaryKey);
if (Objects.nonNull(cached) && !cached.isExpired(handle.getRefreshIntervalSeconds())) {
    automaton = Mono.just(cached.getAutomaton());
}

On that hit path, words is unused. parseWords() splits and trims the configured string, allocates a list, and uses List.contains() for each token. For many distinct words, deduplication is O(n²) in the number of tokens. This work runs before the later scheduler handoff.

Trigger: requests inspected by a configured sensitive-word rule with nonempty words, while its dictionary cache entry remains valid. Repeated requests with unchanged configuration continue doing the same parsing.

Expected behavior / suggested direction

Check the dictionary cache before parsing rule words. Parse only when a dictionary load/rebuild needs them, or keep a parsed representation tied to the rule configuration. Preserve cache invalidation and Redis-failure fallback behavior, including enforcement of configured rule words when no Redis dictionary is available.

Task List

  • Avoid parsing rule words on valid cached-dictionary hits.
  • Preserve miss/expiry loading, rule-change invalidation, stale-dictionary fallback, and rule-word-only fallback.
  • Verify that repeated cache hits do not redo rule-word parsing.
  • Measure allocation/latency changes before claiming a performance gain.

Validation limits

The redundant work and its trigger are confirmed by source inspection. No throughput benchmark was run; the relative impact depends on configured word count and traffic. This is a performance optimization task, not a claim of incorrect moderation results.

Activity

  1. konlue commented on Oct 18, 2026

    @konlue
    Contributor

    /assign

  2. konlue commented on Oct 19, 2026

    @konlue
    Contributor

    Fix delivered stacked on #7844 (same method): parseWords moves into the cache-miss branch, so cache hits no longer repeat the O(n²) split / contains parse. A spy test pins the contract (hits: never invoked; miss: exactly once, rule words still enforced).

    The PR is already open against master, stacked on #7844 — its diff will shrink to just this change once #7844 merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions