feat: ask_user β structured questions from the agent with web card, channel buttons, and text answers#109922
Conversation
openclaw-barnacle
Bot
added
docs
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
π‘ Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: f86c661a13
βΉοΈ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with π.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| "question.requested", | ||
| "question.resolved", |
There was a problem hiding this comment.
When these Gateway events are added, SDK consumers still cannot observe them as stable question events: packages/sdk/src/normalize.ts:157-166 only maps approval/task/session broadcasts and otherwise returns raw, even though packages/sdk/src/types.ts:291-292 already exposes question event types. Any integration using normalizeGatewayEvent will see ask_user question broadcasts as raw events instead of question.requested/question.answered, so please add the corresponding normalization cases with tests.
Useful? React with πΒ / π.
| if ( | ||
| current && | ||
| missingResult?.status === "rejected" && | ||
| isQuestionNotFoundError(missingResult.reason) | ||
| ) { | ||
| markResolvedElsewhere(state, current); |
There was a problem hiding this comment.
If the UI has already marked a prompt locallyExpired, then reconnects after the Gateway's terminal-record grace window, question.get returns QUESTION_NOT_FOUND and this path rewrites the card to answeredElsewhere. That makes an expired ask_user prompt show the wrong terminal state; keep the existing expired state when current.locallyExpired is true instead of calling markResolvedElsewhere.
Useful? React with πΒ / π.
f86c661 to
8520893
Compare
clawsweeper
Bot
added
proof: πΈ screenshot
There was a problem hiding this comment.
π‘ Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 85208937b9
βΉοΈ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with π.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| ...(params.config?.tools?.deny ?? []), | ||
| ...(params.pluginToolDenylist ?? []), | ||
| ]); | ||
| return isPrimaryBootstrapRun(sessionKey) && isToolAllowedByPolicyName("ask_user", { deny }); |
There was a problem hiding this comment.
When a scheduled/background run has a cron session key such as agent:main:cron:job:run:abc, this gate still returns true because isPrimaryBootstrapRun only filters subagent/ACP keys. That exposes the blocking ask_user tool outside an interactive user turn, so a cron job that calls it can stall for the default 15-minute timeout and any prompt is tied to the run-scoped cron session instead of the user's active chat. Please exclude cron/background session keys or require an interactive inbound run before registering the tool.
Useful? React with πΒ / π.
clawsweeper
Bot
added
rating: π¦ gold shrimp
|
Codex review: needs real behavior proof before merge. Reviewed July 17, 2026, 5:17 PM ET / 21:17 UTC. Summary Reproducibility: yes. from source: use a limited-scope native client lacking Review metrics: 1 noteworthy metric.
Stored data model Merge readiness Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch. Rank-up moves:
Proof guidance:
Mantis proof suggestion Risk before merge
Maintainer options:
Next step before merge
Maintainer decision needed
Security Review findings
Review detailsBest possible solution: Land a single provider-neutral question contract only after native scope issuance is updated, prompt delivery is coupled to a real delivery outcome, and redacted final-head channel proof shows prompt delivery, answer resolution, and agent resume. Do we have a high-confidence way to reproduce the issue? Yes, from source: use a limited-scope native client lacking Is this the best way to solve the issue? No. The shared runtime is plausible, but it must preserve existing native operator access and couple the blocking tool to real prompt-delivery success before it is a safe permanent contract. Full review comments:
Overall correctness: patch is incorrect AGENTS.md: unclear because the file could not be read completely. Codex review notes: model internal, reasoning high; reviewed against 84f5d6d978c8. Label changesLabel justifications:
Evidence reviewedWhat I checked:
Likely related people:
What the crustacean ranks mean
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics. How this review workflow works
Review history (8 earlier review cycles)
|
d965b1a to
78cf16e
Compare
There was a problem hiding this comment.
π‘ Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 78cf16efb3
βΉοΈ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with π.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if (hasAskUserPayload(payload)) { | ||
| return payload; | ||
| } |
There was a problem hiding this comment.
When channel runs use the default verbose setting, this new hasAskUserPayload pass-through is still dropped by the later default-progress gate at dispatch-from-config.ts:2351-2358, which only allows media or exec approvals. In Slack/Discord/Telegram default sessions, an ask_user prompt has channelData.askUser but no media/exec approval and isForcedToolProgress is false, so the user never sees the question and the tool blocks until timeout; carry the ask_user exemption through that later gate too.
Useful? React with πΒ / π.
| } | ||
|
|
||
| public struct QuestionGetResult: Codable, Sendable { | ||
| public let question: QuestionRequestedEvent |
There was a problem hiding this comment.
The generated model now uses QuestionRequestedEvent for question.get/question.list, but this commit never emits that type (repo-wide search finds only these references). Building OpenClawKit after this change fails with an unresolved type before any client can consume the new question API; add a concrete question record/event model or use an existing generated shape for these results.
Useful? React with πΒ / π.
Mantis Telegram Desktop ProofSummary: Mantis captured native Telegram Desktop before/after GIFs for the deploy target question and inline answer buttons.
Motion-trimmed clips: |
β¦cs for ask_user
β¦ow control-ui e2e
β¦duplicate-label questions; UI retry and reconnect hardening
β¦user option labels to 64 chars
β¦vents, fix discord lint
β¦uestion surface
β¦y alias from schema registry
clawsweeper
Bot
added
status: π£ needs proof
38f03bd to
db09a1e
Compare
There was a problem hiding this comment.
π‘ Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: db09a1ef56
βΉοΈ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with π.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| { name: "question.request", scope: "operator.questions" }, | ||
| { name: "question.waitAnswer", scope: "operator.questions" }, | ||
| { name: "question.resolve", scope: "operator.questions" }, | ||
| { name: "question.get", scope: "operator.questions" }, | ||
| { name: "question.list", scope: "operator.questions" }, |
There was a problem hiding this comment.
These new RPCs now require operator.questions, but the native/mobile operator scope allowlists were not updated: I checked src/shared/device-bootstrap-profile.ts:22-35, apps/shared/OpenClawKit/Sources/OpenClawKit/GatewayChannelSupport.swift:50-55, and apps/android/app/src/main/java/ai/openclaw/app/gateway/GatewaySession.kt:1081-1088, and all still omit this scope. Admin-scoped clients bypass the check, but limited QR/setup-code or non-admin stored native tokens will be unable to call question.* or receive the new question broadcasts, so paired native clients that otherwise have read/write/approval access silently miss ask_user prompts until they are reissued with the new scope.
Useful? React with πΒ / π.
clawsweeper
Bot
removed
the
merge-risk: π¨ availability
There was a problem hiding this comment.
π‘ Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 4d092848ca
βΉοΈ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with π.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| }); | ||
| }) | ||
| .then( | ||
| () => settleAskUserPromptDelivery(questionId), |
There was a problem hiding this comment.
On ordinary same-channel deliveries, onToolResult only enqueues dispatcher.sendToolResult; the dispatcher returns synchronously and swallows later deliver/before-deliver failures in its send chain (src/auto-reply/reply/reply-dispatcher.ts:449-483, called from src/auto-reply/reply/dispatch-from-config.ts:2364-2369). This line therefore marks the ask_user prompt answerable before the Slack/Discord/Telegram message has actually been sent, so a send failure leaves the tool blocking until timeout instead of taking the prompt-delivery-failed cancellation path. Please settle from a real delivery outcome for ask_user payloads or otherwise await actual delivery before marking it answerable.
Useful? React with πΒ / π.
clawsweeper
Bot
added
the
merge-risk: π¨ availability
|
Merged via squash.
|
β¦hannel buttons, and text answers (openclaw#109922) * feat(gateway): add transient question runtime (question.* methods + broadcasts) * feat(agents): add blocking ask_user question tool with chat prompt delivery and text-reply claim * feat(ui): interactive in-thread question cards for ask_user * feat(channels): native tap-to-answer buttons for ask_user on Telegram, Discord, and Slack * feat(ui): unify codex and gateway question cards with interactive gateway answering * refactor(agents): collapse ask_user pending state to one registry; docs for ask_user * fix(agents): include ask_user in normal gateway runs; add question-flow control-ui e2e * test(ui): avoid credential-shaped fixture in question card test * refactor(ui): reorder stream-group context keys * fix(gateway,ui): validate question answers at resolve; reject secret/duplicate-label questions; UI retry and reconnect hardening * fix(gateway,agents): canonicalize accepted option answers; bound ask_user option labels to 64 chars * chore(ci): prune unused question exports, allowlist mobile question events, fix discord lint * chore(ci): regenerate protocol/i18n/docs/tool-display artifacts for question surface * fix(protocol): flatten QuestionRecord for native codegen; drop TS-only alias from schema registry * chore(android): regenerate ask-user localization resources * docs: regenerate docs map after rebase * fix(ci): avoid stale read-only dependency disks * test: remove stale reef lint suppression ratchet * fix(ci): keep source locale drift advisory in release gates * fix(ci): scope locale advisory handling to parity check
1 participant
Related: #109724 (Codex-parity slice 1: this PR is the provider-neutral question RPC that PR deliberately deferred)
What Problem This Solves
Agents had no first-class way to ask the human a structured question and wait for the answer. The embedded agent could only guess or write prose questions into chat; Codex harness questions got a text-only prompt on channels; and there was no provider-neutral pause/prompt/resume primitive that web UI and message channels could share (the pattern Claude Code ships as
AskUserQuestionand Codex ships asrequest_user_input).Why This Change Was Made
This adds the provider-neutral question runtime deferred by #109724, modeled on the existing exec-approval subsystem (the established blocking pauseβpromptβresume template):
question.request/question.waitAnswer/question.resolve/question.get/question.listRPCs plusquestion.requested/question.resolvedbroadcasts under a newoperator.questionsscope (additive; no protocol version bump). Answers are validated against the stored record (unknown ids, cardinality, option membership) and canonicalized before terminalizing;isSecretand duplicate option labels are rejected at the request boundary.ask_usertool for the embedded agent (main sessions only, not subagents): 1β3 questions, 2β4 options each (labels β€64 chars),multiSelect, free-text "Other" always available, recommended-option-first convention,timeoutSeconds30β3600 (default 900). The tool blocks like exec approvals; expiry returns ano_answerresult telling the model to proceed with best judgment.MessagePresentationwith native tap-to-answer buttons via a new runtime-authored{type:"question"}presentation action (Telegram inline keyboardstgq1:β€64 bytes, Discord custom idsocqβ€100 chars, Slack Block Kitslq1:), degrading to numbered text on button-less channels. A plain text reply (number, label, or free text) answers on any channel; the reply is consumed as the answer instead of steering into the run.ask_userquestions (broadcast-driven, click/multi-select/free-text, resolve over the gateway, terminal states incl. answered-elsewhere/expired) and the existing Codex stream questions from feat(codex): surface native questions and goalsΒ #109724 (composer submit path unchanged).Codex upstream contract checked directly at
openai/codex@9ff47868eb:codex-rs/core/src/tools/handlers/request_user_input_spec.rs,codex-rs/protocol/src/request_user_input.rs,codex-rs/app-server-protocol/src/protocol/v2/item.rs:1585-1637(schema conventions: header β€12 chars, recommended-first, forced free-text "Other", root-thread-only β mirrored here). Converging the Codex/Copilot harness bridges onto this runtime is a named follow-up, not part of this PR.User Impact
Agents can now pause and ask a real question with buttons. Users answer by clicking in the web Control UI, tapping a native button on Telegram/Discord/Slack, or just replying in text on any channel; the agent resumes with structured answers. Docs:
docs/tools/ask-user.md.Pending card (web):
Answered:
MultiSelect:
Terminal states (answered elsewhere / expired):
Evidence
--devstate dir,anthropic/claude-opus-4-8): agent calledask_userwith a single-select + multiSelect pair, blocked 1m03s, the web card rendered in-thread, options were clicked (Staging + Integration/E2E), the gateway resolved, and the agent resumed enumerating exactly those answers. This live run also caught and fixed a gating bug (isEmbeddedMode()is the TUI-host flag, not the embedded-runner flag βask_userwas invisible to normal gateway runs until commitce0251e6bb9's fix).ui/src/e2e/question-flow.e2e.test.ts(3 tests, mock gateway): render β resolve payload shape ({answers:{deploy_target:{answers:["Staging (Recommended)"]}}}) β resolved state; multiSelect arrays; answered-elsewhere + expired terminal states. Screenshots above are its artifacts.node scripts/run-vitest.mjs): gateway question manager + RPC handlers + protocol validators, ask_user tool (normalize/execute/claim/abort races), subscriber delivery, outbound presentation precedence, per-channel envelope round-trips incl. byte limits, UI prompt state machine + card + thread placement β all green (500+ tests across the touched files).tsgo:core,tsgo:core:test,tsgo:extensions,tsgo:extensions:test,tsgo:ui,tsgo:test:ui, targeted oxlint β green (Testbox + local fallback when Testbox returned 429).pnpm buildgreen, no[INEFFECTIVE_DYNAMIC_IMPORT].gpt-5.6-sol, xhigh): three iterations; 7 findings accepted and fixed (answer validation at resolve, isSecret fail-closed, duplicate-label rejection, UI retry latch, reconnectQUESTION_NOT_FOUNDterminalization, option-label bound, answer canonicalization); final run clean β "no accepted/actionable findings".Follow-ups (intentionally out of scope): converge Codex/Copilot harness user-input bridges onto the question runtime; reaction-based answering on WhatsApp/Signal/iMessage; unbind channel buttons after resolution; MCP elicitation routing (pending the 2026-07-28 MRTR spec).