Skip to content

fix(message-format): honor caller --limit for read, pins and search#79593

Closed
pahuchi-joe wants to merge 1 commit into
openclaw:mainfrom
pahuchi-joe:i71452-message-list-pagination
Closed

fix(message-format): honor caller --limit for read, pins and search#79593
pahuchi-joe wants to merge 1 commit into
openclaw:mainfrom
pahuchi-joe:i71452-message-list-pagination

Conversation

@pahuchi-joe

Copy link
Copy Markdown
Contributor

Summary

  • Problem: renderMessageList in src/commands/message-format.ts truncated CLI output with a hardcoded messages.slice(0, 25). The CLI exposed --limit for message read, message pins, and Discord message search, and providers honored the upstream limit, but the rendered table still capped at 25 rows. Provider pagination signals (hasMore, nextBatch, @odata.nextLink) were also discarded, and Discord's read action returns no hasMore field at all so a full page gave no signal that older history existed.
  • Why it matters: callers requesting --limit 50 saw 25 rows with no indication more existed. The maintainer review on feat(message): list chat / list messages should support pagination instead of hardcoded 25 limit #71452 (steipete + clawsweeper) explicitly asked for this contract fix.
  • What changed: renderMessageList now takes displayLimit from FormatOpts and returns { lines, total, displayed, limit }. formatMessageCliText accepts an optional { displayLimit }. The CLI parses --limit via the existing parsePositiveIntOrUndefined helper. A new renderPaginationHint emits a "Showing X of Y; raise --limit to see more" line when the payload had more rows than were rendered, "More results available beyond this page; use --json for the raw cursor" when the payload exposes hasMore / nextBatch / @odata.nextLink, and a heuristic "Reached --limit (N); raise it to fetch older history if any" when the provider returned exactly the requested limit and no other signal is available.
  • What did NOT change (scope boundary): no provider-side changes. MS Teams list-pins still ignores caller limit upstream, MS Teams channel-list still has no exposed cursor, and Feishu pagination is untouched. The default 25-row cap is preserved when --limit is absent. JSON payloads are unchanged. The hint deliberately avoids prescribing --before / --after because those flags only exist on message read (not pins/search) and take message ids rather than the raw cursor tokens returned by Matrix/Graph.

Change Type (select all)

  • Bug fix
  • Feature
  • Refactor required for the fix
  • Docs
  • Security hardening
  • Chore/infra

Scope (select all touched areas)

  • Gateway / orchestration
  • Skills / tool execution
  • Auth / tokens
  • Memory / storage
  • Integrations
  • API / contracts
  • UI / DX
  • CI/CD / infra

Linked Issue/PR

Real behavior proof (required for external PRs)

  • Behavior or issue addressed: message read --limit N, message pins --limit N, and Discord message search --limit N truncated to 25 rows; full Discord pages gave no hint that older history existed.
  • Real environment tested: local OpenClaw checkout on Linux WSL2, Node 22, pnpm 10, against a Discord channel with thousands of historical messages.
  • Exact steps or command run after this patch:
    • pnpm build
    • pnpm openclaw message read --target channel:<id> --channel discord --limit 50 -> renders 50 rows + "Reached --limit (50); raise it to fetch older history if any"
    • pnpm openclaw message read --target channel:<id> --channel discord --limit 50 --before <id> --after <id> -> renders 50 rows + heuristic hint
    • pnpm openclaw message read --target channel:<id> --channel discord -> renders 25 rows + "Showing 25 of 50; raise --limit to see more" (Discord returns 50 by default; we display 25)
    • pnpm openclaw message read --target channel:<id> --channel discord --limit 50 --after <recent-id> -> renders 3 rows, no hint (range exhausted)
  • Evidence after fix: terminal screenshot attached to this PR. Automated proof: 11/11 new formatter tests pass plus 5/5 existing message tests; oxfmt/oxlint/tsgo all clean.
  • Observed result after fix: rendered table honors --limit. Hint line varies based on signal: truncation when total > displayed, cursor when payload exposes pagination metadata, heuristic when the page filled exactly to --limit. No false positives when the range was exhausted (3 < limit).
  • What was not tested: live MS Teams Graph @odata.nextLink and Matrix nextBatch cursor branches against real provider sessions. Both branches are covered in src/commands/message-format.test.ts against representative payload shapes.
  • Before evidence (optional but encouraged): same command on main rendered 25 rows with no hint line; the cap was deterministic from the literal slice(0, 25).

Root Cause (if applicable)

  • Root cause: renderMessageList in src/commands/message-format.ts had a hardcoded messages.slice(0, 25) predating the --limit CLI option; the limit was never threaded into the renderer.
  • Missing detection / guardrail: no formatter-level test asserted that --limit > 25 actually produces more than 25 rendered rows. Existing tests asserted JSON payload shapes, not rendered text.
  • Contributing context (if known): provider plugins added their own --limit and cursor handling over time but the shared CLI renderer was never updated to match. Pagination metadata (hasMore, nextBatch, @odata.nextLink) was already populated by Slack/Matrix/MS Teams but consumed nowhere; Discord returned no metadata at all, so a full page silently looked the same as a complete page.

Regression Test Plan (if applicable)

  • Coverage level that should have caught this:
    • Unit test
    • Seam / integration test
    • End-to-end test
    • Existing coverage already sufficient
  • Target test or file: src/commands/message-format.test.ts.
  • Scenario the test should lock in: formatMessageCliText honors displayLimit for read, list-pins, and search; default behavior unchanged when no limit is passed; pagination hint appears for hasMore, nextBatch, @odata.nextLink, when payload exceeds the requested limit, and when the provider returned exactly --limit rows; no false hint when the page is partial.
  • Why this is the smallest reliable guardrail: the bug is at the formatter contract (one shared file across all three actions). A formatter-level unit test executes the same code path the CLI exercises and is fast and deterministic.
  • Existing test that already covers this (if any): none. The existing src/commands/message.test.ts covers config plumbing, not rendered output.
  • If no new test is added, why not: n/a (added).

User-visible / Behavior Changes

  • openclaw message read --limit N, openclaw message pins --limit N, and openclaw message search --limit N now render up to N rows instead of always capping at 25. Default remains 25 when --limit is omitted.
  • A muted hint line appears below the rendered table when:
    • the source payload had more rows than were displayed: Showing X of Y; raise --limit to see more
    • the provider returned hasMore: true, a Matrix nextBatch, or an MS Teams Graph @odata.nextLink: More results available beyond this page; use --json for the raw cursor
    • the provider returned exactly the requested --limit rows and no other signal exists: Reached --limit (N); raise it to fetch older history if any
  • JSON output (--json) is unchanged and still carries the raw cursor fields.

Diagram (if applicable)

n/a

Security Impact (required)

  • New permissions/capabilities? No
  • Secrets/tokens handling changed? No
  • New/changed network calls? No
  • Command/tool execution surface changed? No
  • Data access scope changed? No
  • If any Yes, explain risk + mitigation: n/a

Repro + Verification

Environment

  • OS: Linux WSL2 (Ubuntu)
  • Runtime/container: Node 22, pnpm 10
  • Model/provider: n/a (rendering only)
  • Integration/channel (if any): tested live against a Discord channel with thousands of historical messages
  • Relevant config (redacted): n/a

Steps

  1. pnpm build
  2. pnpm openclaw message read --target channel:<id> --channel discord --limit 50 (full-page heuristic)
  3. pnpm openclaw message read --target channel:<id> --channel discord (default-cap truncation)
  4. pnpm openclaw message read --target channel:<id> --channel discord --limit 50 --after <recent-id> (partial page; no hint)

Expected

  • Step 2 renders 50 rows + "Reached --limit (50); raise it to fetch older history if any".
  • Step 3 renders 25 rows + "Showing 25 of 50; raise --limit to see more" when Discord returned 50.
  • Step 4 renders fewer than 50 rows with no hint.
  • On main step 3 renders 25 rows with no hint regardless.

Actual

  • Matches expected. Screenshot attached to the PR.

Evidence

  • Failing test/log before + passing after
  • Trace/log snippets
  • Screenshot/recording
  • Perf numbers (if relevant)

The new formatter tests in src/commands/message-format.test.ts would have failed against the prior slice(0, 25) cap (e.g. expect(countRowsInOutput(out, 30)).toBe(30) for read with displayLimit: 30).

Screenshot 2026-05-09 042734

Human Verification (required)

  • Verified scenarios: ran message read --limit 50 against a live Discord channel with hundreds of messages and confirmed 50 rows + "Reached --limit (50)" hint; ran message read without --limit and confirmed "Showing 25 of 50; raise --limit to see more" hint; ran message read --limit 50 --after <recent-id> and confirmed partial page renders without a false-positive hint.
  • Edge cases checked: invalid --limit values (0, -3, non-numeric strings) fall through parsePositiveIntOrUndefined to undefined, which the formatter treats as "use default 25"; payload smaller than the requested limit renders all rows with no hint; provider cursor branches (hasMore, nextBatch, @odata.nextLink) covered by unit tests against representative payload shapes.
  • What I did not verify: live Matrix nextBatch and live MS Teams Graph @odata.nextLink cursor branches against real provider sessions.

Review Conversations

  • I replied to or resolved every bot review conversation I addressed in this PR.
  • I left unresolved only the conversations that still need reviewer or maintainer judgment.

Compatibility / Migration

  • Backward compatible? Yes
  • Config/env changes? No
  • Migration needed? No
  • If yes, exact upgrade steps: n/a

Risks and Mitigations

  • Risk: the full-page heuristic can produce a "Reached --limit" hint even when the user happened to request exactly the number of remaining messages.
    • Mitigation: the wording says "raise it to fetch older history if any", framing it as a possibility rather than a fact. Raising --limit quickly verifies.
  • Risk: rendering very large payloads when the caller passes a huge --limit.
    • Mitigation: caller-controlled, providers already enforce upstream caps in their fetch layer; the formatter only renders what the provider returned.
  • Risk: pagination hint phrasing diverges from existing CLI hint dialects elsewhere in the codebase (sessions.ts, status.command-sections.ts).
    • Mitigation: phrasing is muted-styled and additive; consolidating shared hint helpers is a separate follow-up.

Test plan

  • pnpm test src/commands/message-format.test.ts src/commands/message.test.ts
  • pnpm exec oxfmt --check --threads=1 src/commands/message-format.ts src/commands/message-format.test.ts src/commands/message.ts CHANGELOG.md docs/cli/message.md
  • node scripts/run-oxlint.mjs src/commands/message-format.ts src/commands/message-format.test.ts src/commands/message.ts
  • pnpm tsgo:all
  • pnpm build && pnpm openclaw message read --target channel: --channel discord --limit 50

Before / After

Before:

  • openclaw message read --limit 50 rendered 25 rows even when the provider returned 50.
  • Provider pagination signals (hasMore, nextBatch, @odata.nextLink) were silently discarded by the CLI renderer.
  • Discord's full-page reads gave no signal that older history existed.

After:

  • The CLI renders up to --limit rows for read, list-pins, and search. Default stays at 25 when --limit is omitted.
  • A muted "Showing X of Y; raise --limit to see more" hint appears under the table when the payload had more rows than were displayed.
  • A generic "More results available beyond this page; use --json for the raw cursor" hint appears when the provider returns hasMore, a Matrix nextBatch, or an MS Teams Graph @odata.nextLink.
  • A heuristic "Reached --limit (N); raise it to fetch older history if any" hint appears when the provider returned exactly the requested --limit rows and no other signal exists (covers Discord, which returns no hasMore).

Bugfix trigger

User invokes openclaw message read --target <channel> --limit 50 (or message pins --limit N, or Discord message search --limit N) against a channel/source that returns more than 25 messages. Before this fix, the rendered table always stops at 25 rows regardless of the requested limit, and any hasMore/nextBatch/@odata.nextLink returned by the provider is hidden. For Discord specifically, even after raising --limit the user has no way to tell whether the page is exhaustive or partial because Discord doesn't return a hasMore signal.

@openclaw-barnacle openclaw-barnacle Bot added docs Improvements or additions to documentation commands Command implementations size: M proof: supplied External PR includes structured after-fix real behavior proof. labels May 9, 2026
@clawsweeper

clawsweeper Bot commented May 9, 2026

Copy link
Copy Markdown
Contributor

Thanks for the context here. I swept through the related work, and this is now duplicate or superseded.

Close this PR as superseded: the same central message formatter limit fix is now covered by a newer clean, mergeable, proof-positive PR, while this branch is conflicting and still has unresolved review blockers.

Root-cause cluster
Relationship: superseded
Canonical: #99089
Summary: The current PR and the newer PR target the same shared message formatter display-limit and pagination-hint bug; the newer PR is the viable landing path.

Members:

Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

Canonical path: Close this conflicting branch and use #99089 as the landing path for the shared formatter limit fix; fold in the small pins --limit docs note separately if maintainers want it.

So I’m closing this here and keeping the remaining discussion on #99089.

Review details

Best possible solution:

Close this conflicting branch and use #99089 as the landing path for the shared formatter limit fix; fold in the small pins --limit docs note separately if maintainers want it.

Do we have a high-confidence way to reproduce the issue?

Yes. Source inspection on current main shows message --limit is not forwarded into the text formatter and rendered rows are still capped with messages.slice(0, 25).

Is this the best way to solve the issue?

No for this branch. The shared formatter and CLI bridge are the right fix boundary, but the cleaner current solution is the newer mergeable PR that drops the release-changelog edit and adds total-aware search hints.

Security review:

Security review cleared: The diff is limited to CLI formatter plumbing, formatter tests, docs, and changelog text with no dependency, workflow, credential, install, or supply-chain surface change.

AGENTS.md: found and applied where relevant.

What I checked:

Likely related people:

  • steipete: The unified message tool and CLI refactor added src/commands/message-format.ts with the 25-row cap, and later path history shows several related formatter/package refactors from this account. (role: introduced formatter behavior and frequent area contributor; confidence: high; commits: 3636a2bf51d9, 00d8d7ead059, de1dfab03ef0; files: src/commands/message-format.ts, src/commands/message.ts)
  • lzyyzznl: Recent history for src/commands/message-format.ts includes a merged dry-run output fix touching formatMessageCliText, making this account relevant for current formatter behavior routing. (role: recent formatter contributor; confidence: medium; commits: 508e3bf41353; files: src/commands/message-format.ts)
  • sudie-codes: The linked issue mentions Teams pagination, and MSTeams message-action history includes read, pin, unpin, and list-pins work in the provider paths adjacent to this formatter bug. (role: adjacent MSTeams message action contributor; confidence: medium; commits: 0f192710924f, f71ee71787c7; files: extensions/msteams/src/graph-messages.ts, extensions/msteams/src/channel.ts)

Codex review notes: model internal, reasoning high; reviewed against 9d68f877ac3e.

@clawsweeper clawsweeper Bot added the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 9, 2026
@openclaw-barnacle

Copy link
Copy Markdown

This pull request has been automatically marked as stale due to inactivity.
Please add updates or it will be closed.

@openclaw-barnacle openclaw-barnacle Bot added the stale Marked as stale due to inactivity label May 26, 2026
@clawsweeper clawsweeper Bot added proof: sufficient ClawSweeper judged the real behavior proof convincing. proof: 📸 screenshot Contributor real behavior proof includes screenshot evidence. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. P2 Normal backlog priority with limited blast radius. labels May 26, 2026
@clawsweeper

clawsweeper Bot commented May 26, 2026

Copy link
Copy Markdown
Contributor

ClawSweeper PR egg

🔥 Warming up: real-behavior proof passed; findings, security review, or rank-up moves are still in progress.

Hatch command

Comment @clawsweeper hatch when this PR is hatchable.

Hatchability rules:

  • Merged PRs are hatchable.
  • Open PRs are hatchable when they are status: 👀 ready for maintainer look, status: 🚀 automerge armed, or labeled clawsweeper:automerge.
  • Closed unmerged PRs are hatchable only when one of those hatchable labels is still present in the durable record.
What is this egg doing here?
  • Eggs appear after the PR passes real-behavior proof. It is here for vibes, not verdicts: it does not change labels, ratings, merge decisions, or automation.
  • The shell reacts to review momentum: open follow-up work warms it up, re-review makes it wobble, and a clean final review lets it hatch.
  • Hatchability usually comes from sufficient real-behavior proof, no blocking P0/P1/P2 findings, no security attention needed, and clean correctness. A merged PR is already final, so merge makes the egg hatchable independently.
  • The hatch is seeded from this repository and PR number, so the same PR keeps the same creature; the reviewed head SHA can only change safe visual details.
  • Rarity is just collectible sparkle: 🥚 common, 🌱 uncommon, 💎 rare, ✨ glimmer, and 🌈 legendary.

@clawsweeper

clawsweeper Bot commented Jul 4, 2026

Copy link
Copy Markdown
Contributor

🦞✅
ClawSweeper autoclose is complete.

Reason: structured ClawSweeper close marker: close-required (sha=3e984c7096a809293fd0f94aadbf0b64ac9a3de5)

Closed:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

commands Command implementations docs Improvements or additions to documentation P2 Normal backlog priority with limited blast radius. proof: 📸 screenshot Contributor real behavior proof includes screenshot evidence. proof: sufficient ClawSweeper judged the real behavior proof convincing. proof: supplied External PR includes structured after-fix real behavior proof. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. size: M status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(message): list chat / list messages should support pagination instead of hardcoded 25 limit

1 participant