Skip to content

Commit f981199

Browse files
sahrizviclaude
andcommitted
fix(question): return Unanswered in non-interactive mode
The earlier non-interactive policy in this branch scanned option label text for "safe" keywords (skip/cancel/no/abort/...) and fell back to the last option. That tried to recover semantics the LLM already knew at construction time, and false-positived on substrings — "no" matched inside "Snowflake", "Annotate", "Knowledge", "Honor". The fix: don't guess. Return Unanswered for every question when no TTY is present and let the agent decide. The agent has full context — it knows what action it was about to take and why it asked. It can pick a safe path from that context or report that user input is required. Pretending a decision was made that wasn't is the worse failure mode. Changes: - Drop SAFE_KEYWORDS and the label-text scan entirely. - Default non-interactive behavior returns [] (renders as "Unanswered" via the existing format()). - Cache isNonInteractive() once at execute() entry so the result prefix can't disagree with the path that produced the answer. - Non-interactive prefix tells the agent how to proceed AND names the escape hatch (ALTIMATE_AUTO_ANSWER=first|last|<label>) so it can surface it to the user when reporting back. - Keep =first / =last / =<exact label> as explicit user opt-ins for callers who genuinely want a default. Drop =skip — it's the new default. Addresses cubic-dev-ai review feedback on #937 by removing the heuristic that needed the word-boundary fix in the first place. Tests: 9 pass / 0 fail in packages/opencode/test/tool/question.test.ts. Refs #936 Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
1 parent a49442b commit f981199

2 files changed

Lines changed: 80 additions & 54 deletions

File tree

‎packages/opencode/src/tool/question.ts‎

Lines changed: 33 additions & 38 deletions
Original file line numberDiff line numberDiff line change
@@ -3,32 +3,30 @@ import { Tool } from "./tool"
33
import { Question } from "../question"
44
import DESCRIPTION from "./question.txt"
55

6-
// altimate_change start — non-interactive auto-answer support.
6+
// altimate_change start — non-interactive handling for the question tool.
7+
//
78
// When running under `claude --print`, CI, or any other context without a TTY,
8-
// there is nobody to click an option in the TUI. The default Question.ask()
9-
// behaviour is to await a Deferred indefinitely, which causes the parent
10-
// process to TaskStop the subprocess after a long wait — looking exactly like
11-
// a hang. See deliverable 02 (Run F first sub-session) for the trace.
9+
// there is nobody to click an option in the TUI. The upstream Question.ask()
10+
// awaits a Deferred indefinitely, which looks exactly like a hang to a parent
11+
// supervisor.
12+
//
13+
// Policy: when non-interactive, return Unanswered for every question and let
14+
// the calling agent decide what to do. The agent knows what it was about to
15+
// do and why it asked; it can pick a safe path from context or report that
16+
// user input is required. We deliberately do NOT guess based on label text
17+
// — every heuristic we tried (safe-keyword scan, last-option fallback)
18+
// either invented decisions the user didn't make or false-positive'd on
19+
// labels like "Snowflake" that happened to contain "no".
1220
//
13-
// Resolution policy: in non-interactive mode, pick the option whose label
14-
// contains a "safe" keyword (skip / cancel / profile only / no / abort).
15-
// If no such option exists, pick the LAST option (UX convention: safer/cancel
16-
// usually sits at the end). The agent then sees a concrete answer in the
17-
// tool result and can continue without blocking. Override via env var:
18-
// ALTIMATE_AUTO_ANSWER=first — always pick first option
19-
// ALTIMATE_AUTO_ANSWER=last — always pick last option (default)
20-
// ALTIMATE_AUTO_ANSWER=skip — return Unanswered for all questions
21-
const SAFE_KEYWORDS = [
22-
"skip",
23-
"cancel",
24-
"no",
25-
"abort",
26-
"profile only",
27-
"profile-only",
28-
"decline",
29-
"deny",
30-
"stop",
31-
]
21+
// Explicit overrides (for users who genuinely want a default and accept the
22+
// responsibility):
23+
// ALTIMATE_AUTO_ANSWER=first — always pick the first option
24+
// ALTIMATE_AUTO_ANSWER=last — always pick the last option
25+
// ALTIMATE_AUTO_ANSWER="<label>" — pick the option whose label matches
26+
//
27+
// TTY-detection overrides (for harnesses that lie about being a TTY):
28+
// ALTIMATE_FORCE_INTERACTIVE=1 — treat as interactive even without TTY
29+
// ALTIMATE_NON_INTERACTIVE=1 — treat as non-interactive even with TTY
3230

3331
function isNonInteractive(): boolean {
3432
if (process.env["ALTIMATE_FORCE_INTERACTIVE"] === "1") return false
@@ -37,20 +35,14 @@ function isNonInteractive(): boolean {
3735
}
3836

3937
function autoAnswer(questions: Question.Info[]): Question.Answer[] {
40-
const mode = (process.env["ALTIMATE_AUTO_ANSWER"] ?? "last").toLowerCase()
38+
const mode = process.env["ALTIMATE_AUTO_ANSWER"]?.toLowerCase()
4139
return questions.map((q) => {
42-
if (mode === "skip") return []
40+
if (!mode) return [] // default — Unanswered, agent decides
4341
if (mode === "first") return q.options[0] ? [q.options[0].label] : []
4442
if (mode === "last") {
45-
const safe = q.options.find((o) => {
46-
const text = `${o.label} ${o.description}`.toLowerCase()
47-
return SAFE_KEYWORDS.some((k) => text.includes(k))
48-
})
49-
if (safe) return [safe.label]
5043
const last = q.options[q.options.length - 1]
5144
return last ? [last.label] : []
5245
}
53-
// exact label match for explicit answers, e.g. ALTIMATE_AUTO_ANSWER="Profile only"
5446
const match = q.options.find((o) => o.label.toLowerCase() === mode)
5547
return match ? [match.label] : []
5648
})
@@ -64,8 +56,12 @@ export const QuestionTool = Tool.define("question", {
6456
}),
6557
async execute(params, ctx) {
6658
// altimate_change start — short-circuit when no human is listening.
59+
// Cache the mode once: env vars can change across the `await` below, and
60+
// we want the result prefix to describe the path the answer actually
61+
// came from, not whatever state we observe later.
62+
const nonInteractive = isNonInteractive()
6763
let answers: Question.Answer[]
68-
if (isNonInteractive()) {
64+
if (nonInteractive) {
6965
answers = autoAnswer(params.questions)
7066
} else {
7167
answers = await Question.ask({
@@ -83,11 +79,10 @@ export const QuestionTool = Tool.define("question", {
8379

8480
const formatted = params.questions.map((q, i) => `"${q.question}"="${format(answers[i])}"`).join(", ")
8581

86-
// altimate_change start — flag auto-answers explicitly so the agent
87-
// knows the user didn't actually answer and can decide whether to
88-
// proceed with that choice or fail back gracefully.
89-
const prefix = isNonInteractive()
90-
? `Running in non-interactive mode (no TTY). Auto-answered with safe defaults: `
82+
// altimate_change start — make the non-interactive case unambiguous to
83+
// the agent so it doesn't treat "Unanswered" as a real user choice.
84+
const prefix = nonInteractive
85+
? `Running in non-interactive mode (no TTY). No user was available to answer. Either pick a safe path from the context of the action you were about to take, or report that user input is required to proceed — the user can set ALTIMATE_AUTO_ANSWER=first|last|<exact option label> to pre-answer questions in this mode. Result: `
9186
: `User has answered your questions: `
9287
// altimate_change end
9388

‎packages/opencode/test/tool/question.test.ts‎

Lines changed: 47 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -127,7 +127,7 @@ describe("tool.question non-interactive auto-answer", () => {
127127
askSpy.mockRestore()
128128
})
129129

130-
test("picks safe-keyword option when present and does not invoke Question.ask", async () => {
130+
test("default returns Unanswered for every question and does not invoke Question.ask", async () => {
131131
const tool = await QuestionTool.init()
132132
const questions = [
133133
{
@@ -142,29 +142,39 @@ describe("tool.question non-interactive auto-answer", () => {
142142

143143
const result = await tool.execute({ questions }, ctx)
144144
expect(askSpy).not.toHaveBeenCalled()
145-
expect(result.output).toContain("Profile only")
145+
expect(result.output).toContain('"May I run row-level hashdiff comparisons?"="Unanswered"')
146+
// The non-interactive prefix tells the agent to pick a safe path from context
147+
// AND tells the agent about the escape hatch so it can surface it to the user
148+
// when reporting that input is required.
146149
expect(result.output).toContain("non-interactive mode")
150+
expect(result.output).toContain("safe path")
151+
expect(result.output).toContain("ALTIMATE_AUTO_ANSWER")
147152
})
148153

149-
test("falls back to last option when no safe keyword matches", async () => {
154+
test("does not invent answers based on label text (no label-text heuristic)", async () => {
155+
// Regression: a prior implementation scanned labels for substrings like
156+
// "skip" / "cancel" / "no". That false-positived on labels like
157+
// "Snowflake" (contains "no") and quietly picked the wrong option. The
158+
// default now returns Unanswered — the agent decides.
150159
const tool = await QuestionTool.init()
151160
const questions = [
152161
{
153-
question: "Pick a color",
154-
header: "Color",
162+
question: "Pick a warehouse",
163+
header: "Warehouse",
155164
options: [
156-
{ label: "Red", description: "The color of passion" },
157-
{ label: "Blue", description: "The color of sky" },
165+
{ label: "Snowflake", description: "Continue with Snowflake" },
166+
{ label: "Cancel", description: "Stop here" },
158167
],
159168
},
160169
]
161170

162171
const result = await tool.execute({ questions }, ctx)
163-
expect(askSpy).not.toHaveBeenCalled()
164-
expect(result.output).toContain("Blue")
172+
expect(result.output).toContain('"Pick a warehouse"="Unanswered"')
173+
expect(result.output).not.toContain('"Snowflake"')
174+
expect(result.output).not.toContain('"Cancel"')
165175
})
166176

167-
test("ALTIMATE_AUTO_ANSWER=first picks first option", async () => {
177+
test("ALTIMATE_AUTO_ANSWER=first picks first option (explicit opt-in)", async () => {
168178
process.env["ALTIMATE_AUTO_ANSWER"] = "first"
169179
const tool = await QuestionTool.init()
170180
const questions = [
@@ -179,22 +189,25 @@ describe("tool.question non-interactive auto-answer", () => {
179189
]
180190

181191
const result = await tool.execute({ questions }, ctx)
182-
expect(result.output).toContain("Red")
192+
expect(result.output).toContain('"Pick a color"="Red"')
183193
})
184194

185-
test("ALTIMATE_AUTO_ANSWER=skip returns Unanswered for each question", async () => {
186-
process.env["ALTIMATE_AUTO_ANSWER"] = "skip"
195+
test("ALTIMATE_AUTO_ANSWER=last picks last option (explicit opt-in)", async () => {
196+
process.env["ALTIMATE_AUTO_ANSWER"] = "last"
187197
const tool = await QuestionTool.init()
188198
const questions = [
189199
{
190200
question: "Pick a color",
191201
header: "Color",
192-
options: [{ label: "Red", description: "" }],
202+
options: [
203+
{ label: "Red", description: "" },
204+
{ label: "Blue", description: "" },
205+
],
193206
},
194207
]
195208

196209
const result = await tool.execute({ questions }, ctx)
197-
expect(result.output).toContain("Unanswered")
210+
expect(result.output).toContain('"Pick a color"="Blue"')
198211
})
199212

200213
test("ALTIMATE_AUTO_ANSWER=<exact label> picks matching option", async () => {
@@ -212,7 +225,25 @@ describe("tool.question non-interactive auto-answer", () => {
212225
]
213226

214227
const result = await tool.execute({ questions }, ctx)
215-
expect(result.output).toContain("Blue")
228+
expect(result.output).toContain('"Pick a color"="Blue"')
229+
})
230+
231+
test("ALTIMATE_AUTO_ANSWER=<unknown label> falls through to Unanswered", async () => {
232+
process.env["ALTIMATE_AUTO_ANSWER"] = "green"
233+
const tool = await QuestionTool.init()
234+
const questions = [
235+
{
236+
question: "Pick a color",
237+
header: "Color",
238+
options: [
239+
{ label: "Red", description: "" },
240+
{ label: "Blue", description: "" },
241+
],
242+
},
243+
]
244+
245+
const result = await tool.execute({ questions }, ctx)
246+
expect(result.output).toContain('"Pick a color"="Unanswered"')
216247
})
217248

218249
test("non-interactive prefix is set when Question.ask is bypassed", async () => {

0 commit comments

Comments
 (0)