Skip to content

Commit 965f97a

Browse files
anandgupta42claude
andcommitted
fix(analyst): keep full_text_columns output bounded and always recoverable
Letting named cells bypass the formatter cap did not deliver complete text: the session applies a per-result token cap before persisting any tool output, so an oversized cell was middle-truncated anyway, and because the clean path wrote no spill and reported `truncated: false`, the missing text could not be recovered. The mode is now "priority within the same bounded output, always recoverable": - Full-text mode always writes one complete lossless JSONL spill of the returned rows (including empty results and unknown names) and sets `result_path`; the banner says `read` with `char_offset`/`max_chars` recovers clipped text. - Named cells take the shared byte/line budget first, in row order; a cell that does not fit is clipped with a continued-in-spill marker and the metadata reports the clip. Once the budget is exhausted, remaining cells are omitted behind one note instead of a marker per cell. - Structured and binary cells are serialised with the same tagged serialiser as the spill, inline and on disk, instead of `String(value)`. - The schema line keeps its ordinary alias and column caps, with named columns eligible beyond the ordinary prefix; the missing-column, limit, lint and suggestion notes are counted inside the output budget. - The formatter enforces the effective row limit (20 in full-text mode), notes when a connector returned more, and spills all returned rows. Tests: 50 KiB named cell clipped within 16 KiB with its complete spill; twenty 1 MiB cells bounded with one spill; structured and binary cells lossless inline and in the spill; empty result with a missing name held to a 64-byte budget with the clip reported; 300 long aliases bounded beside one named column; 40 connector rows limited to 20 with all rows saved; notes counted in the budget. Co-Authored-By: Claude Fable 5.1 <[email protected]>
1 parent 425e6e2 commit 965f97a

4 files changed

Lines changed: 603 additions & 258 deletions

File tree

‎packages/opencode/src/altimate/tools/sql-execute.ts‎

Lines changed: 20 additions & 36 deletions
Original file line numberDiff line numberDiff line change
@@ -3,18 +3,13 @@ import { Effect } from "effect"
33
import { EffectBridge } from "../../effect/bridge"
44
import { Truncate } from "../../tool/truncate"
55
import { legacyToInit, type LegacyToolDef } from "../tool-zod-compat"
6-
import { formatSqlResult, SQL_MAX_PREVIEW_BYTES } from "./sql-result-format"
6+
import { formatSqlResult } from "./sql-result-format"
77
import { lintSqlResult } from "./sql-result-lint"
88
import { Tool } from "../../tool/tool"
99
import { Dispatcher } from "../native"
1010
import type { SqlExecuteResult } from "../native/types"
1111
// altimate_change start - SQL write access control + fingerprinting
12-
import {
13-
classifyAndCheck,
14-
classifyMongoCommand,
15-
computeSqlFingerprint,
16-
isScratchSafeWrite,
17-
} from "./sql-classify"
12+
import { classifyAndCheck, classifyMongoCommand, computeSqlFingerprint, isScratchSafeWrite } from "./sql-classify"
1813
import { statementsOf } from "./sql-tokenize"
1914

2015
const MONGO_WAREHOUSE_TYPES = new Set(["mongodb", "mongo"])
@@ -59,7 +54,7 @@ const parameters = z
5954
.array(z.string().min(1))
6055
.nullish()
6156
.describe(
62-
"Keep all selected columns and return the named columns as complete text without preview or spill cuts. [] or null, or omitting this field, turns the mode off. A non-empty list caps rows at 20; omit limit to use the default, or larger limits are clamped with a result note. Missing names produce a note listing available columns; rows are still returned.",
57+
"Keep all selected columns and prioritize named cell text within the ordinary output budget. Always save all returned rows to result_path; use read with char_offset/max_chars to recover clipped text. [] or null, or omitting this field, turns the mode off. A non-empty list caps preview rows at 20; omit limit to use the default, or larger limits are clamped with a result note. Missing names produce a bounded note listing available columns; rows are still returned.",
6358
),
6459
query_timeout_ms: z
6560
.number()
@@ -85,15 +80,15 @@ export const SqlExecuteTool = Tool.define(
8580
const bridge = yield* EffectBridge.make()
8681
return yield* legacyToInit({
8782
description:
88-
"Execute SQL against a connected data warehouse. Returns all selected columns as a formatted table. full_text_columns keeps named cells complete without preview or spill cuts; [] or null, or omitting the field, turns the mode off. A non-empty list caps rows at 20: omit limit to use the default; larger limits clamp to 20 with a result note. Missing column names produce a note listing available columns while still returning rows.",
83+
"Execute SQL against a connected data warehouse. Returns all selected columns as a formatted table. full_text_columns prioritizes named cells within the ordinary output budget and always saves returned rows to result_path; read with char_offset/max_chars recovers clipped text. [] or null, or omitting the field, turns the mode off. A non-empty list caps preview rows at 20: omit limit to use the default; larger limits clamp to 20 with a result note. Missing column names produce a bounded note listing available columns while still returning rows.",
8984
parameters,
9085
formatValidationError: (error) => error.issues?.map((issue) => issue.message).join("; ") || LIMIT_ERROR,
9186
async execute(args, ctx) {
9287
const fullText = Boolean(args.full_text_columns?.length)
9388
const limit = fullText ? Math.min(args.limit ?? 20, 20) : (args.limit ?? 100)
9489
const limitNote =
9590
fullText && args.limit !== undefined && args.limit > 20
96-
? `\n\nNote: limit ${args.limit} clamped to 20 for full_text_columns.`
91+
? `Note: limit ${args.limit} clamped to 20 for full_text_columns.`
9792
: ""
9893
// altimate_change start - SQL write access control
9994
// Permission checks OUTSIDE try/catch so denial errors propagate to the framework
@@ -122,7 +117,9 @@ export const SqlExecuteTool = Tool.define(
122117
// Connectors expose one result set; several prepare only one statement
123118
// or cannot normalize a batch response. Do not silently drop reads.
124119
if (count > 1) {
125-
throw new Error("Multi-statement reads are not supported by this execution path. Run each read separately.")
120+
throw new Error(
121+
"Multi-statement reads are not supported by this execution path. Run each read separately.",
122+
)
126123
}
127124
}
128125
// The ephemeral scratch database is the session's own workspace (what sql_materialize
@@ -224,48 +221,35 @@ export const SqlExecuteTool = Tool.define(
224221

225222
const limits = await bridge.promise(truncate.limits())
226223
const suggestion = PostConnectSuggestions.getProgressiveSuggestion("sql_execute")
227-
const extra =
228-
limitNote +
229-
(suggestion ? "\n\n" + suggestion : "") +
230-
(precedence.notice ? precedence.notice + "\n\n" : "")
224+
const lint = await lintSqlResult(args.query, args.warehouse, ctx.sessionID)
231225
const formatted = await formatSqlResult(result, {
232226
limit,
233227
fullTextColumns: args.full_text_columns,
234228
// The statement text lets the formatter add the empty/near-empty
235229
// result hint when a string-literal filter is present.
236230
sql: query,
237-
maxBytes: Math.max(
238-
0,
239-
Math.min(SQL_MAX_PREVIEW_BYTES, limits.maxBytes) - Buffer.byteLength(extra, "utf8"),
240-
),
241-
maxLines: Math.max(0, limits.maxLines - extra.split("\n").length + 1),
231+
notes: [precedence.notice ?? "", limitNote, lint?.output ?? "", suggestion ?? ""].filter(Boolean),
232+
maxBytes: limits.maxBytes,
233+
maxLines: limits.maxLines,
242234
write: (text) => bridge.promise(truncate.write(text)),
243235
})
244-
const lint = await lintSqlResult(args.query, args.warehouse, ctx.sessionID)
245-
let output = formatted.output + limitNote + (lint ? "\n\n" + lint.output : "")
246236
// altimate_change start — progressive disclosure suggestions
247-
if (suggestion) {
248-
output += "\n\n" + suggestion
237+
if (suggestion && formatted.output.includes(suggestion)) {
249238
PostConnectSuggestions.trackSuggestions({
250239
suggestionType: "progressive_disclosure",
251240
suggestionsShown: ["sql_analyze"],
252241
warehouseType: args.warehouse ?? "default",
253242
})
254243
}
255244
// altimate_change end
256-
// altimate_change — carries the fail-open notice when the target could not be
257-
// attributed to the workspace; a no-op otherwise.
258-
const response = Precedence.annotate(precedence, {
259-
title: `SQL: ${args.query.slice(0, 60)}${args.query.length > 60 ? "..." : ""}`,
260-
metadata: { ...formatted.metadata, ...(lint && { lint_rules: lint.rules }) },
261-
output,
262-
})
263-
if (precedence.notice && formatted.metadata.truncated) {
264-
// Keep the result contract first while retaining the routing notice.
265-
const newline = output.indexOf("\n")
266-
response.output = output.slice(0, newline + 1) + precedence.notice + "\n\n" + output.slice(newline + 1)
245+
// Preserve routing metadata while the formatter owns the bounded notice text.
246+
return {
247+
...Precedence.annotate(precedence, {
248+
title: `SQL: ${args.query.slice(0, 60)}${args.query.length > 60 ? "..." : ""}`,
249+
metadata: { ...formatted.metadata, ...(lint && { lint_rules: lint.rules }) },
250+
}),
251+
output: formatted.output,
267252
}
268-
return response
269253
} catch (e) {
270254
const msg = e instanceof Error ? e.message : String(e)
271255
// altimate_change: deliberately NOT fingerprinted, same reasoning as the

0 commit comments

Comments
 (0)