Skip to content

Conversation

@akshaymankar
Copy link
Member

@akshaymankar akshaymankar commented Jun 26, 2025

https://wearezeta.atlassian.net/browse/WPB-18436

The message count recieved in the queue info gotten as response of declaring the
queue may include some expired messages which have not yet made it to the head
of the queue. This makes the message count mechanism useless to the clients for
detecting the end of initial sync.

This PR allows clients to detect the end of initial sync by inserting a synchronization marker upon connecting to the websocket using a query param. Once they receive this synchronization marker, they will know that all the notifications which existed in the queue before they connected to the websocket have been consumed.

Checklist

  • Add a new entry in an appropriate subdirectory of changelog.d
  • Read and follow the PR guidelines

…tial sync

The message count recieved in the queue info gotten as response of declaring the
queue may include some expired messages which have not yet made it to the head
of the queue. This makes the message count mechanism useless to the clients for
detecting the end of initial sync.

This commit detects end of initial sync based on sending a transient
notification in the queue. If we recieve this notification back, it would mean
that the queue is empty and the initial sync is complete.

The timing of sending this notification must be carefully chosen, it is sent in
one of these two conditions:

  1. If the message count in the queue info is 0, the queue must be empty as the
  expired messages are already considered expired when they reach the head of
  the queue. So its a good time to publish our end-of-initial-sync notification.
  If the queue is indeed empty we'll recieve it and forward it to the client,
  letting it know that it can consider itself up to date.

  2. If number of unacked messages becomes 0 right after recieving an
  acknowledgement and the initial sync hasn't already completed, this could mean
  either the queue is empty or there is some delay in receiving the next message
  from RabbitMQ. Here we publish our end-of-initial-sync notification. If there
  was a delay in receiving the next message from RabbitMQ, the consumer thread
  will increase the count for unacked messages and we'll get another opportunity
  to check this. Otherwise, the consumer thread will recieve this
  end-of-initial-sync notification and forward it to the client.
@akshaymankar akshaymankar requested review from a team as code owners June 26, 2025 13:56
@zebot zebot added the ok-to-test Approved for running tests in CI, overrides not-ok-to-test if both labels exist label Jun 26, 2025
reqId = RequestId defRequestId
in Env {..}

-- Env leh lp (RequestId defRequestId)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove comment

@akshaymankar akshaymankar merged commit 1b84c4a into develop Jul 3, 2025
8 checks passed
@akshaymankar akshaymankar deleted the end-of-initial-sync branch July 3, 2025 11:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ok-to-test Approved for running tests in CI, overrides not-ok-to-test if both labels exist

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants