Skip to content

api: an Application watch resumed from a WorkloadMonitor version can skip HelmRelease changes of other applications #5324

Open

Description

Where

pkg/registry/apps/application/rest.go, REST.Watch.

Problem

An Application watch merges two backing watches: HelmReleases and WorkloadMonitors. Events from the two streams reach the client in whatever order they arrive, and a WorkloadMonitor-triggered event carries a resourceVersion at least as new as the monitor's. A client-go reflector records the resourceVersion of every event it receives as its resume point, whatever object the event is for, and passes it back when it reconnects.

The backing HelmRelease watch is opened without the client's resourceVersion, and resume is emulated by replaying every release as ADDED and skipping those whose resourceVersion is not above the client's.

Sequence:

  1. Application A's HelmRelease is written at revision 100; the event has not been delivered on the HelmRelease stream yet.
  2. Application B's WorkloadMonitor is written at revision 101, and its event is delivered first.
  3. The client records 101 and the connection drops.
  4. On reconnect the replay of A at 100 is skipped as already seen, and the client keeps A's previous spec until A changes again or the client relists.

Impact

Rare: it needs a disconnect inside the window between the two deliveries. When it happens, the stale object stays in the client's cache with no error.

Possible directions

  • Deliver events in revision order across both streams, holding a monitor event until the HelmRelease stream has progressed past its revision (for example via a progress bookmark from the backing watch).
  • Open the backing watches from the client's resourceVersion instead of replaying and filtering, so the server, not the merge, decides what the client has seen. This alone does not fix the ordering across the two streams.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    triage/needs-triageIndicates an issue needs triage by a maintainer

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions