In Solace messaging, flow control is critical to ensure guaranteed delivery and efficient client-broker interaction. This post breaks down how flow is managed at both the broker and client API levels. Let’s first start with some core concepts.
Core Concepts
Solace Queue Unacknowledged Messages Limit
A broker-side setting that defines how many messages can be delivered to a client before they must be settled (acknowledged with ACK, NACK-REJECT, or NACK-FAILURE).
Example: if the limit is set to 5, the broker delivers only messages 1–5 to the client. Messages 6–10 remain queued until some of the earlier ones are settled.
Subscriber Window Size
A client-side mechanism, similar to TCP’s sliding window protocol.
The client tells the broker how many messages it can accept at once. If the window is 3, the broker delivers only 3 messages and then waits until the client signals that it can accept more.
The queue unacknowledged limit and subscriber window size operate independently.
The unacknowledged limit can be much larger than the window size. This means multiple messages can remain unsettled even as the window is being actively refilled.
Client APIs usually try to keep the window open, unless:
the application explicitly pauses flow (e.g., calling stop()), or
synchronous receive() calls cause the internal queue to fill up.
By default, the internal API queue is sized to match the subscriber window size (with some exceptions, e.g., JCSMP uses window+1).
Flow Control Mechanics
The subscriber window size can grow or shrink dynamically throughout the session.
Each time the client consumes messages from its internal queue, the broker refills the window with more.
Flow control messages must be exchanged within a 30-second timeout. However, the client doesn’t need to resend control updates if the state hasn’t changed.
Ordered Processing Considerations
For strict ordering, one option is to set both the unacknowledged limit and subscriber window size to 1. This ensures single-message processing but comes at a big performance cost.
More commonly, defaults are used with application-level logic to enforce order (e.g., blocking APIs or synchronized callbacks).
Messages are always delivered to the client in queue order, but acknowledgments may happen out of order if the application processes them asynchronously.
In short: the broker limits outstanding unacknowledged messages, the client controls how many it can handle at a time, and together they ensure both reliability and performance.
Thank you for the great, informative post. As mentioned in our private message, I am encountering a behavior that I cannot quite explain which (might be) is related to your post.
I am using the GuaranteedReceiver sample pattern from the Solace Java API GitHub repository which can be found here
The only changes I made to the sample code were:
Changed the QUEUE_NAME.
Added the property to define the window size to 100:
I started the receiver, but as mentioned never called the receiveAsync() function on it. Which results in the consumer screenshotted above. Showcasing the defined window size.
My expectation was that the receiver would locally receive/buffer unacknowledged messages only up to the defined window size (100) or until the “Maximum Delivered Unacknowledged Messages per Flow” limit is hit.
However, neither seems to be the case. I consistently see about ~580 messages buffered. Sometimes slightly fewer, but never more.
Here is the same client with ~500 messages marked as unacknowledged.
I initially thought it might be related to the TCP receive buffers on the system, but that does not seem to be the cause. Maybe someone else has an idea
Note: If I enable the receiveAsync() block but simply do not acknowledge any messages, I receive all 1,000 messages as unacknowledged (as expected).
Hey @jfg1306 ! I suspect this has to do with the back pressure strategy on the client side that leverages the Internal bounded queue in the Java API (Note: you can read more about back pressure strategy here Consuming Direct Messages Using the Solace Java API ) . This is a per-flow receiver queue that holds messages before delivery to application (and hence before reaching the subscriber window size) and has its own capacity limits. The messages accumulate in the internal bounded queue because there’s no consumer draining it
Broker → Transport Layer (window=100) → Internal Bounded Queue (~580) → [BLOCKED - no consumer]
↑
Fills to capacity
Back-pressure applied
Note that the API applies back-pressure and stops accepting more messages once messages accumulate in the internal bounded queue and it reaches capacity. The transport window size and the Java internal buffer have independent mechanisms and serve different purposes
The reason why you receive all 1,000 messages if you enable the receiveAsync() as mentioned, its because the callback actively drains the internal queue, preventing it from filling
In short: the broker limits outstanding unacknowledged messages, the client controls how many it can handle at a time, and together they ensure both reliability and performance.
