Skip to content

zulip: fix event queue registration for multiple monitored streams#14182

Closed
simpliq-marvin wants to merge 11 commits intoopenclaw:mainfrom
simpliq-marvin:marvin/harden-12183-on-feat
Closed

zulip: fix event queue registration for multiple monitored streams#14182
simpliq-marvin wants to merge 11 commits intoopenclaw:mainfrom
simpliq-marvin:marvin/harden-12183-on-feat

Conversation

@simpliq-marvin
Copy link
Copy Markdown

@simpliq-marvin simpliq-marvin commented Feb 11, 2026

Problem

When channels.zulip.streams contains multiple stream names, the Zulip plugin was registering an event queue with a narrow like:

[["stream","build"],["stream","general"],["stream","openclaw"],["stream","research"]]

Zulip narrows are ANDed, so this matches nothing (a message cannot be in multiple streams at once). This caused the default/Marvin account to stop receiving messages, while role accounts (1 stream each) continued to work.

Fix

  • If streams contains "*": use all_public_streams=true
  • If exactly 1 stream: use narrow=[["stream","<name>"]] (legacy list format)
  • If multiple streams: omit narrow (broad queue) and rely on existing client-side filtering

Tests

Adds a Vitest regression test covering the three cases above.

Greptile Overview

Greptile Summary

This PR fixes Zulip event queue registration when multiple monitored streams are configured: it uses all_public_streams=true for "*", uses a single-stream narrow for exactly one stream (legacy list format), and omits narrow for multiple streams while enforcing the stream allowlist client-side. It also adds new Zulip plugin implementation files (client, monitor, send/probe, config schema, normalization) plus Vitest coverage for queue registration and stream filtering, and updates docs/labeler to include Zulip.

The change integrates into the channel plugin system by providing a zulipPlugin with config + pairing + outbound delivery, a monitor loop using Zulip event queues, and shared helpers for routing/session keys and stream monitoring.

Confidence Score: 3/5

  • This PR is close to mergeable but has a couple of logic issues that can affect message routing and inbound gating.
  • Core stream registration fix looks correct and is covered by targeted tests, but there are two verified logic bugs: outbound parsing can truncate topics, and inbound mention detection uses a hardcoded agentId which can drop messages for routed agents when requireMention is enabled.
  • extensions/zulip/src/zulip/send.ts; extensions/zulip/src/zulip/monitor.ts

FtlC-ian and others added 10 commits February 8, 2026 17:48
Adds a full Zulip channel plugin with:
- Event queue polling with exponential backoff
- Authenticated upload downloads from /user_uploads/
- Outbound file uploads via /api/v1/user_uploads
- Reaction indicators (eyes → check_mark/warning)
- Topic directive [[zulip_topic: <topic>]]
- HTTP retry with backoff (429/502/503/504)
- Full actions API (stream CRUD, user management, reactions)
- Channel docs and unit tests

Refs: #5163, openclaw#8365, openclaw#9643
Addresses Greptile review feedback: ensure lastEventId stays
as a number type when assigned from event response data.
Zulip /register narrows are ANDed, so a narrow containing multiple stream operators matches nothing. For >1 stream, register a broad queue and rely on client-side filtering.

Adds a vitest regression test for the narrow/all_public_streams behavior.
@openclaw-barnacle openclaw-barnacle bot added the docs Improvements or additions to documentation label Feb 11, 2026
Copy link
Copy Markdown
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

29 files reviewed, 2 comments

Edit Code Review Agent Settings | Greptile

Comment on lines +542 to +546
chatType,
sender: { name: senderName, id: senderId },
});

const to = kind === "dm" ? `user:${senderId}` : `stream:${streamName || streamId}:${topic}`;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stream replies can break

to is built as stream:${streamName || streamId}:${topic}. When display_recipient isn’t a string, streamName stays empty and this falls back to message.stream_id (numeric). That numeric value then flows through sendMessageZulipsendZulipStreamMessage and is used as the Zulip API to field, which this implementation treats as a stream name (not an ID). This will cause stream replies to fail / go to the wrong place for events where only stream_id is available.

Prompt To Fix With AI
This is a comment left during a code review.
Path: extensions/zulip/src/zulip/monitor.ts
Line: 542:546

Comment:
**Stream replies can break**

`to` is built as ``stream:${streamName || streamId}:${topic}``. When `display_recipient` isn’t a string, `streamName` stays empty and this falls back to `message.stream_id` (numeric). That numeric value then flows through `sendMessageZulip``sendZulipStreamMessage` and is used as the Zulip API `to` field, which this implementation treats as a *stream name* (not an ID). This will cause stream replies to fail / go to the wrong place for events where only `stream_id` is available.

How can I resolve this? If you propose a fix, please make it concise.

Comment on lines +281 to +289
if (params.streams?.includes("*")) {
body.set("all_public_streams", "true");
} else if (params.streams && params.streams.length > 0) {
// IMPORTANT: Zulip narrow filters are ANDed.
// A narrow like [["stream","a"],["stream","b"]] matches nothing.
// So:
// - if exactly one stream is requested, use a stream narrow
// - if multiple streams are requested, register a broad queue and filter client-side
// (we already drop messages outside account.streams in monitor.ts).
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No client-side stream filter

For streams.length > 1, /register intentionally omits narrow and relies on “existing client-side filtering” (“we already drop messages outside account.streams in monitor.ts”). However extensions/zulip/src/zulip/monitor.ts doesn’t filter inbound stream messages against account.streams at all (it only uses account.streams for queue registration). With multiple streams configured, this will process messages from any stream the bot can see, which is a behavior regression vs the intended “monitor only these streams” semantics.

Prompt To Fix With AI
This is a comment left during a code review.
Path: extensions/zulip/src/zulip/client.ts
Line: 281:289

Comment:
**No client-side stream filter**

For `streams.length > 1`, `/register` intentionally omits `narrow` and relies on “existing client-side filtering” (“we already drop messages outside account.streams in monitor.ts”). However `extensions/zulip/src/zulip/monitor.ts` doesn’t filter inbound stream messages against `account.streams` at all (it only uses `account.streams` for queue registration). With multiple streams configured, this will process messages from *any* stream the bot can see, which is a behavior regression vs the intended “monitor only these streams” semantics.

How can I resolve this? If you propose a fix, please make it concise.

When monitoring multiple streams we must omit Zulip /register narrow (narrows are ANDed), which results in a broad event queue. Ensure we still respect channels.zulip.streams by filtering inbound stream messages in monitor.ts.

Also resolve missing stream names via /streams/{id} so outbound replies don't fall back to numeric stream ids.
@simpliq-marvin
Copy link
Copy Markdown
Author

Addressed Greptile concerns:

  1. Broad queue filtering: when monitoring multiple streams we must omit /register narrow (Zulip narrows are ANDed), which creates a broad event queue. monitor.ts now enforces channels.zulip.streams via a isMonitoredStream(...) check for inbound stream messages.

  2. Numeric stream_id reply targets: when display_recipient is absent we now resolve the stream name via /streams/{id} (cached) so outbound replies prefer stream:<name>: rather than falling back to the numeric id.

Includes a small unit test for isMonitoredStream.

(Commit: 0338bdf)

@simpliq-marvin
Copy link
Copy Markdown
Author

@greptileai could you re-review? We addressed the two concerns you raised:

  • enforce monitored-stream filtering when registering a broad event queue (multi-stream case)
  • resolve missing stream names via /streams/{id} (cached) so replies don't fall back to numeric stream_id

Latest commit: 0338bdf

Copy link
Copy Markdown
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

30 files reviewed, 2 comments

Edit Code Review Agent Settings | Greptile

Comment on lines +71 to +78
if (lower.startsWith("stream:")) {
const rest = trimmed.slice("stream:".length).trim();
if (!rest) {
throw new Error("Stream name is required for Zulip sends");
}
const [stream, topic] = rest.split(/[:#/]/);
return { kind: "stream", stream: stream.trim(), topic: topic?.trim() };
}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Topic parsing truncates

parseZulipTarget() splits stream:NAME[:topic] using rest.split(/[:#/]/) and destructures [stream, topic], so any topic containing : or # (and anything after the first separator) is silently dropped (e.g. stream:general:foo:bar becomes topic foo). This breaks the advertised :topic format for common topic strings and can route messages to the wrong topic.

Prompt To Fix With AI
This is a comment left during a code review.
Path: extensions/zulip/src/zulip/send.ts
Line: 71:78

Comment:
**Topic parsing truncates**

`parseZulipTarget()` splits `stream:NAME[:topic]` using `rest.split(/[:#/]/)` and destructures `[stream, topic]`, so any topic containing `:` or `#` (and anything after the first separator) is silently dropped (e.g. `stream:general:foo:bar` becomes topic `foo`). This breaks the advertised `:topic` format for common topic strings and can route messages to the wrong topic.

How can I resolve this? If you propose a fix, please make it concise.

Comment on lines +379 to +383
const mentionRegexes = core.channel.mentions.buildMentionRegexes(cfg, "main");
const wasMentioned =
!isDM &&
(rawText.toLowerCase().includes(`@${botUsername.toLowerCase()}`) ||
core.channel.mentions.matchesMentionPatterns(rawText, mentionRegexes));
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hardcoded agentId for mentions

Mention detection is built with buildMentionRegexes(cfg, "main") before resolveAgentRoute() runs, but mention patterns are agent-scoped (derived from the routed agent’s identity / groupChat.mentionPatterns). If routing selects a non-main agent and requireMention is enabled, messages that correctly mention that agent can be treated as “not mentioned” and dropped.

Prompt To Fix With AI
This is a comment left during a code review.
Path: extensions/zulip/src/zulip/monitor.ts
Line: 379:383

Comment:
**Hardcoded agentId for mentions**

Mention detection is built with `buildMentionRegexes(cfg, "main")` before `resolveAgentRoute()` runs, but mention patterns are agent-scoped (derived from the routed agent’s identity / `groupChat.mentionPatterns`). If routing selects a non-`main` agent and `requireMention` is enabled, messages that correctly mention that agent can be treated as “not mentioned” and dropped.

How can I resolve this? If you propose a fix, please make it concise.

FtlC-ian added a commit to FtlC-ian/openclaw that referenced this pull request Feb 12, 2026
Adds a full-featured Zulip channel plugin with:

Core Features:
- Event queue polling with exponential backoff and auto-recovery
- Authenticated upload downloads from /user_uploads/
- Outbound file uploads via /api/v1/user_uploads
- Reaction indicators (eyes → check_mark/warning)
- Topic directive [[zulip_topic: <topic>]]
- HTTP retry with backoff (429/502/503/504)
- Full actions API (stream CRUD, user management, reactions)
- Channel docs and unit tests

Key Improvement Over Existing PRs:
- Implements CONCURRENT message processing with staggered start times
- Other PRs (openclaw#9643, openclaw#14182) process messages sequentially, causing
  reply delays and "message dumps" when multiple messages arrive
- This implementation starts processing each message immediately with
  a 200ms stagger for natural conversation flow
- Replies arrive as each finishes instead of all at once

Technical Details:
- Fire-and-forget message processing with per-message error handling
- Maintains event cursor and queue ID correctly during concurrent processing
- Tested live and confirmed significant UX improvement

Refs: openclaw#5163, openclaw#8365, openclaw#9643, openclaw#12183
@openclaw-barnacle
Copy link
Copy Markdown

This pull request has been automatically marked as stale due to inactivity.
Please add updates or it will be closed.

@openclaw-barnacle openclaw-barnacle bot added stale Marked as stale due to inactivity and removed stale Marked as stale due to inactivity labels Feb 21, 2026
simpliq-marvin pushed a commit to simpliq-marvin/openclaw that referenced this pull request Feb 26, 2026
Adds a full-featured Zulip channel plugin with:

Core Features:
- Event queue polling with exponential backoff and auto-recovery
- Authenticated upload downloads from /user_uploads/
- Outbound file uploads via /api/v1/user_uploads
- Reaction indicators (eyes → check_mark/warning)
- Topic directive [[zulip_topic: <topic>]]
- HTTP retry with backoff (429/502/503/504)
- Full actions API (stream CRUD, user management, reactions)
- Channel docs and unit tests

Key Improvement Over Existing PRs:
- Implements CONCURRENT message processing with staggered start times
- Other PRs (openclaw#9643, openclaw#14182) process messages sequentially, causing
  reply delays and "message dumps" when multiple messages arrive
- This implementation starts processing each message immediately with
  a 200ms stagger for natural conversation flow
- Replies arrive as each finishes instead of all at once

Technical Details:
- Fire-and-forget message processing with per-message error handling
- Maintains event cursor and queue ID correctly during concurrent processing
- Tested live and confirmed significant UX improvement

Refs: openclaw#5163, openclaw#8365, openclaw#9643, openclaw#12183
FtlC-ian added a commit to FtlC-ian/openclaw that referenced this pull request Feb 27, 2026
Adds a full-featured Zulip channel plugin with:

Core Features:
- Event queue polling with exponential backoff and auto-recovery
- Authenticated upload downloads from /user_uploads/
- Outbound file uploads via /api/v1/user_uploads
- Reaction indicators (eyes → check_mark/warning)
- Topic directive [[zulip_topic: <topic>]]
- HTTP retry with backoff (429/502/503/504)
- Full actions API (stream CRUD, user management, reactions)
- Channel docs and unit tests

Key Improvement Over Existing PRs:
- Implements CONCURRENT message processing with staggered start times
- Other PRs (openclaw#9643, openclaw#14182) process messages sequentially, causing
  reply delays and "message dumps" when multiple messages arrive
- This implementation starts processing each message immediately with
  a 200ms stagger for natural conversation flow
- Replies arrive as each finishes instead of all at once

Technical Details:
- Fire-and-forget message processing with per-message error handling
- Maintains event cursor and queue ID correctly during concurrent processing
- Tested live and confirmed significant UX improvement

Refs: openclaw#5163, openclaw#8365, openclaw#9643, openclaw#12183
@openclaw-barnacle
Copy link
Copy Markdown

This pull request has been automatically marked as stale due to inactivity.
Please add updates or it will be closed.

@openclaw-barnacle openclaw-barnacle bot added the stale Marked as stale due to inactivity label Mar 12, 2026
@openclaw-barnacle
Copy link
Copy Markdown

Closing due to inactivity.
If you believe this PR should be revived, post in #pr-thunderdome-dangerzone on Discord to talk to a maintainer.
That channel is the escape hatch for high-quality PRs that get auto-closed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs Improvements or additions to documentation stale Marked as stale due to inactivity

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants