Skip to content

[BUG] Gateway ignores failed WebSocket configuration messages without resynchronizing #7316

Closed

Description

@Aias00

Current Behavior

ShenyuWebsocketClient#onMessage catches any RuntimeException raised while parsing or applying a configuration message and logs:

Failed to handle websocket message ..., the message will be ignored

The connection remains open and the client does not request MYSELF, reconnect, mark itself unready, or notify Admin. The ignored message can represent a plugin, selector, rule, auth, metadata, proxy-selector, discovery, or API-key update.

Because the WebSocket protocol has no event sequence, ACK, NACK, or replay cursor, one handler exception permanently diverges gateway cache from Admin state until another update happens to overwrite it or the gateway restarts.

This behavior was introduced as a reader-thread safety improvement after #6843, but reader-thread survival alone does not restore consistency.

Reproduction

  • Bind a subscriber whose update handler throws once.
  • Deliver a valid WebSocket UPDATE message.
  • Verify the exception is caught and ignored while the socket remains open.
  • Verify no full sync is requested and the gateway cache remains stale.
  • Restart/reconnect and verify MYSELF restores the current snapshot.

Expected Behavior

Failure to apply a configuration message must move the client into an explicit desynchronized state and initiate bounded recovery. It must not silently continue as healthy.

Scope

Acceptance Criteria

  • A subscriber/application exception cannot be silently ignored while readiness remains healthy.
  • The client either replays the failed event or completes a full resynchronization.
  • Recovery is bounded and observable.
  • Poison data does not cause an unbounded reconnect storm.
  • Logs/metrics identify the failed group, event type, recovery attempt, and final result without leaking secrets.
  • Tests cover transient handler failure, permanent failure, successful full-sync recovery, and subsequent incremental updates.
  • End-to-end cache state converges to Admin state after a recoverable failure.

Activity

  1. github-actions commented on Sep 26, 2026

    @github-actions
    Contributor

    👋 Hello @yuluo-yx

    A new issue has been opened by @Aias00:

    Please review when you have time. Thank you! 🙏

  2. added a commit that references this issue on Oct 6, 2026
    b2e1a81
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions