zulip: fix event queue registration for multiple monitored streams#14182
zulip: fix event queue registration for multiple monitored streams#14182simpliq-marvin wants to merge 11 commits intoopenclaw:mainfrom
Conversation
Adds a full Zulip channel plugin with: - Event queue polling with exponential backoff - Authenticated upload downloads from /user_uploads/ - Outbound file uploads via /api/v1/user_uploads - Reaction indicators (eyes → check_mark/warning) - Topic directive [[zulip_topic: <topic>]] - HTTP retry with backoff (429/502/503/504) - Full actions API (stream CRUD, user management, reactions) - Channel docs and unit tests Refs: #5163, openclaw#8365, openclaw#9643
Addresses Greptile review feedback: ensure lastEventId stays as a number type when assigned from event response data.
Zulip /register narrows are ANDed, so a narrow containing multiple stream operators matches nothing. For >1 stream, register a broad queue and rely on client-side filtering. Adds a vitest regression test for the narrow/all_public_streams behavior.
| chatType, | ||
| sender: { name: senderName, id: senderId }, | ||
| }); | ||
|
|
||
| const to = kind === "dm" ? `user:${senderId}` : `stream:${streamName || streamId}:${topic}`; |
There was a problem hiding this comment.
Stream replies can break
to is built as stream:${streamName || streamId}:${topic}. When display_recipient isn’t a string, streamName stays empty and this falls back to message.stream_id (numeric). That numeric value then flows through sendMessageZulip → sendZulipStreamMessage and is used as the Zulip API to field, which this implementation treats as a stream name (not an ID). This will cause stream replies to fail / go to the wrong place for events where only stream_id is available.
Prompt To Fix With AI
This is a comment left during a code review.
Path: extensions/zulip/src/zulip/monitor.ts
Line: 542:546
Comment:
**Stream replies can break**
`to` is built as ``stream:${streamName || streamId}:${topic}``. When `display_recipient` isn’t a string, `streamName` stays empty and this falls back to `message.stream_id` (numeric). That numeric value then flows through `sendMessageZulip` → `sendZulipStreamMessage` and is used as the Zulip API `to` field, which this implementation treats as a *stream name* (not an ID). This will cause stream replies to fail / go to the wrong place for events where only `stream_id` is available.
How can I resolve this? If you propose a fix, please make it concise.| if (params.streams?.includes("*")) { | ||
| body.set("all_public_streams", "true"); | ||
| } else if (params.streams && params.streams.length > 0) { | ||
| // IMPORTANT: Zulip narrow filters are ANDed. | ||
| // A narrow like [["stream","a"],["stream","b"]] matches nothing. | ||
| // So: | ||
| // - if exactly one stream is requested, use a stream narrow | ||
| // - if multiple streams are requested, register a broad queue and filter client-side | ||
| // (we already drop messages outside account.streams in monitor.ts). |
There was a problem hiding this comment.
No client-side stream filter
For streams.length > 1, /register intentionally omits narrow and relies on “existing client-side filtering” (“we already drop messages outside account.streams in monitor.ts”). However extensions/zulip/src/zulip/monitor.ts doesn’t filter inbound stream messages against account.streams at all (it only uses account.streams for queue registration). With multiple streams configured, this will process messages from any stream the bot can see, which is a behavior regression vs the intended “monitor only these streams” semantics.
Prompt To Fix With AI
This is a comment left during a code review.
Path: extensions/zulip/src/zulip/client.ts
Line: 281:289
Comment:
**No client-side stream filter**
For `streams.length > 1`, `/register` intentionally omits `narrow` and relies on “existing client-side filtering” (“we already drop messages outside account.streams in monitor.ts”). However `extensions/zulip/src/zulip/monitor.ts` doesn’t filter inbound stream messages against `account.streams` at all (it only uses `account.streams` for queue registration). With multiple streams configured, this will process messages from *any* stream the bot can see, which is a behavior regression vs the intended “monitor only these streams” semantics.
How can I resolve this? If you propose a fix, please make it concise.When monitoring multiple streams we must omit Zulip /register narrow (narrows are ANDed), which results in a broad event queue. Ensure we still respect channels.zulip.streams by filtering inbound stream messages in monitor.ts.
Also resolve missing stream names via /streams/{id} so outbound replies don't fall back to numeric stream ids.
|
Addressed Greptile concerns:
Includes a small unit test for (Commit: 0338bdf) |
|
@greptileai could you re-review? We addressed the two concerns you raised:
Latest commit: 0338bdf |
| if (lower.startsWith("stream:")) { | ||
| const rest = trimmed.slice("stream:".length).trim(); | ||
| if (!rest) { | ||
| throw new Error("Stream name is required for Zulip sends"); | ||
| } | ||
| const [stream, topic] = rest.split(/[:#/]/); | ||
| return { kind: "stream", stream: stream.trim(), topic: topic?.trim() }; | ||
| } |
There was a problem hiding this comment.
Topic parsing truncates
parseZulipTarget() splits stream:NAME[:topic] using rest.split(/[:#/]/) and destructures [stream, topic], so any topic containing : or # (and anything after the first separator) is silently dropped (e.g. stream:general:foo:bar becomes topic foo). This breaks the advertised :topic format for common topic strings and can route messages to the wrong topic.
Prompt To Fix With AI
This is a comment left during a code review.
Path: extensions/zulip/src/zulip/send.ts
Line: 71:78
Comment:
**Topic parsing truncates**
`parseZulipTarget()` splits `stream:NAME[:topic]` using `rest.split(/[:#/]/)` and destructures `[stream, topic]`, so any topic containing `:` or `#` (and anything after the first separator) is silently dropped (e.g. `stream:general:foo:bar` becomes topic `foo`). This breaks the advertised `:topic` format for common topic strings and can route messages to the wrong topic.
How can I resolve this? If you propose a fix, please make it concise.| const mentionRegexes = core.channel.mentions.buildMentionRegexes(cfg, "main"); | ||
| const wasMentioned = | ||
| !isDM && | ||
| (rawText.toLowerCase().includes(`@${botUsername.toLowerCase()}`) || | ||
| core.channel.mentions.matchesMentionPatterns(rawText, mentionRegexes)); |
There was a problem hiding this comment.
Hardcoded agentId for mentions
Mention detection is built with buildMentionRegexes(cfg, "main") before resolveAgentRoute() runs, but mention patterns are agent-scoped (derived from the routed agent’s identity / groupChat.mentionPatterns). If routing selects a non-main agent and requireMention is enabled, messages that correctly mention that agent can be treated as “not mentioned” and dropped.
Prompt To Fix With AI
This is a comment left during a code review.
Path: extensions/zulip/src/zulip/monitor.ts
Line: 379:383
Comment:
**Hardcoded agentId for mentions**
Mention detection is built with `buildMentionRegexes(cfg, "main")` before `resolveAgentRoute()` runs, but mention patterns are agent-scoped (derived from the routed agent’s identity / `groupChat.mentionPatterns`). If routing selects a non-`main` agent and `requireMention` is enabled, messages that correctly mention that agent can be treated as “not mentioned” and dropped.
How can I resolve this? If you propose a fix, please make it concise.Adds a full-featured Zulip channel plugin with: Core Features: - Event queue polling with exponential backoff and auto-recovery - Authenticated upload downloads from /user_uploads/ - Outbound file uploads via /api/v1/user_uploads - Reaction indicators (eyes → check_mark/warning) - Topic directive [[zulip_topic: <topic>]] - HTTP retry with backoff (429/502/503/504) - Full actions API (stream CRUD, user management, reactions) - Channel docs and unit tests Key Improvement Over Existing PRs: - Implements CONCURRENT message processing with staggered start times - Other PRs (openclaw#9643, openclaw#14182) process messages sequentially, causing reply delays and "message dumps" when multiple messages arrive - This implementation starts processing each message immediately with a 200ms stagger for natural conversation flow - Replies arrive as each finishes instead of all at once Technical Details: - Fire-and-forget message processing with per-message error handling - Maintains event cursor and queue ID correctly during concurrent processing - Tested live and confirmed significant UX improvement Refs: openclaw#5163, openclaw#8365, openclaw#9643, openclaw#12183
bfc1ccb to
f92900f
Compare
|
This pull request has been automatically marked as stale due to inactivity. |
Adds a full-featured Zulip channel plugin with: Core Features: - Event queue polling with exponential backoff and auto-recovery - Authenticated upload downloads from /user_uploads/ - Outbound file uploads via /api/v1/user_uploads - Reaction indicators (eyes → check_mark/warning) - Topic directive [[zulip_topic: <topic>]] - HTTP retry with backoff (429/502/503/504) - Full actions API (stream CRUD, user management, reactions) - Channel docs and unit tests Key Improvement Over Existing PRs: - Implements CONCURRENT message processing with staggered start times - Other PRs (openclaw#9643, openclaw#14182) process messages sequentially, causing reply delays and "message dumps" when multiple messages arrive - This implementation starts processing each message immediately with a 200ms stagger for natural conversation flow - Replies arrive as each finishes instead of all at once Technical Details: - Fire-and-forget message processing with per-message error handling - Maintains event cursor and queue ID correctly during concurrent processing - Tested live and confirmed significant UX improvement Refs: openclaw#5163, openclaw#8365, openclaw#9643, openclaw#12183
Adds a full-featured Zulip channel plugin with: Core Features: - Event queue polling with exponential backoff and auto-recovery - Authenticated upload downloads from /user_uploads/ - Outbound file uploads via /api/v1/user_uploads - Reaction indicators (eyes → check_mark/warning) - Topic directive [[zulip_topic: <topic>]] - HTTP retry with backoff (429/502/503/504) - Full actions API (stream CRUD, user management, reactions) - Channel docs and unit tests Key Improvement Over Existing PRs: - Implements CONCURRENT message processing with staggered start times - Other PRs (openclaw#9643, openclaw#14182) process messages sequentially, causing reply delays and "message dumps" when multiple messages arrive - This implementation starts processing each message immediately with a 200ms stagger for natural conversation flow - Replies arrive as each finishes instead of all at once Technical Details: - Fire-and-forget message processing with per-message error handling - Maintains event cursor and queue ID correctly during concurrent processing - Tested live and confirmed significant UX improvement Refs: openclaw#5163, openclaw#8365, openclaw#9643, openclaw#12183
|
This pull request has been automatically marked as stale due to inactivity. |
|
Closing due to inactivity. |
Problem
When
channels.zulip.streamscontains multiple stream names, the Zulip plugin was registering an event queue with anarrowlike:Zulip narrows are ANDed, so this matches nothing (a message cannot be in multiple streams at once). This caused the default/Marvin account to stop receiving messages, while role accounts (1 stream each) continued to work.
Fix
"*": useall_public_streams=truenarrow=[["stream","<name>"]](legacy list format)narrow(broad queue) and rely on existing client-side filteringTests
Adds a Vitest regression test covering the three cases above.
Greptile Overview
Greptile Summary
This PR fixes Zulip event queue registration when multiple monitored streams are configured: it uses
all_public_streams=truefor"*", uses a single-streamnarrowfor exactly one stream (legacy list format), and omitsnarrowfor multiple streams while enforcing the stream allowlist client-side. It also adds new Zulip plugin implementation files (client, monitor, send/probe, config schema, normalization) plus Vitest coverage for queue registration and stream filtering, and updates docs/labeler to include Zulip.The change integrates into the channel plugin system by providing a
zulipPluginwith config + pairing + outbound delivery, a monitor loop using Zulip event queues, and shared helpers for routing/session keys and stream monitoring.Confidence Score: 3/5