Skip to content

Commit 7254d08

Browse files
anandgupta42claude
andcommitted
fix: scope text_label to analyst and document the 30,000 row limit
- Register `text_label` only for `analyst` and `data-explorer` agents, not globally - Oversized-input error and `data-question` skill now describe splitting into disjoint id ranges and verifying combined coverage - Schema test rejects any enum containing an empty string - Model-resolution test persists the assistant message through the legacy session API so its assertions execute Co-Authored-By: Claude Opus 5.5 <[email protected]>
1 parent 91281f9 commit 7254d08

5 files changed

Lines changed: 22 additions & 13 deletions

File tree

‎.opencode/skills/data-question/SKILL.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -139,7 +139,7 @@ Dialect references: [DuckDB query_table](https://www.duckdb.org/docs/current/gui
139139

140140
Treat inferred labels as new data, not ground truth. Define a fixed rubric, allowed labels and uncertainty policy before classifying. For synthetic `messages(message_id, body)`, a rubric might classify a request to reverse a charge as `refund`, a question about shipping progress as `delivery`, and ambiguous text as `uncertain`. Specify tie-breaking for multi-topic messages.
141141

142-
Inspect text, then define the complete population of stable IDs. When `text_label` is available, materialize that population in `scratch` with `sql_materialize`, then call `text_label` with SQL over scratch, the stable ID and text column names, a fixed label set with a description for each label, and any rubric/tie-breaking instructions. It classifies every row through the session's LLM in bounded batches and persists the results in scratch; do not substitute regex or keyword guesses for semantic labels. Check `rows_in`, `labeled`, `failed` and `unparseable`, and retrieve failed IDs before claiming complete coverage. Optional spot-check agreement measures repeatability, not correctness. Review uncertain cases consistently; batching makes counting reproducible, not classification inherently correct.
142+
Inspect text, then define the complete population of stable IDs. When `text_label` is available, materialize that population in `scratch` with `sql_materialize`, then call `text_label` with SQL over scratch, the stable ID and text column names, a fixed label set with a description for each label, and any rubric/tie-breaking instructions. It classifies every row through the session's LLM in bounded batches and persists the results in scratch; do not substitute regex or keyword guesses for semantic labels. `text_label` accepts at most 30,000 rows per call. For a larger population, split it into disjoint, deterministic id ranges (one call per range, each into a distinct result table), combine the persisted result tables in SQL, and verify that combined coverage equals the population count before claiming results. Check `rows_in`, `labeled`, `failed` and `unparseable`, and retrieve failed IDs before claiming complete coverage. Optional spot-check agreement measures repeatability, not correctness. Review uncertain cases consistently; batching makes counting reproducible, not classification inherently correct.
143143

144144
Use the persisted `text_label` output for SQL counts. It retains the source ID column name, so use `message_id` as the ID column for this example. To reuse the checks below, select `message_id`, `label` and your declared rubric version as `rubric_version` in a SELECT/CTE named `labels`; `population` must cover the same intended IDs. If `text_label` is unavailable, small audited annotation batches can use an available permitted ingestion path to form `labels(message_id, label, rubric_version)`. When `scratch` and `sql_materialize` are available, these small batches can be expressed as a SELECT relation and materialized there. For example (DuckDB/SQLite SELECT syntax):
145145

‎packages/opencode/src/altimate/tools/text-label.ts‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -344,7 +344,7 @@ export async function labelScratch(input: TextLabelInput, context: LabelContext,
344344
})
345345
}, context.sessionID)
346346
if (data.rows.length > MAX_ROWS || data.truncated)
347-
return fail(`Input exceeds the ${MAX_ROWS} row limit; split the population into disjoint queries.`)
347+
return fail(`Input exceeds the ${MAX_ROWS} row limit; split the population into disjoint, deterministic id ranges (one call per range, each into a distinct target_table), then combine the persisted result tables in SQL and verify combined coverage equals the population count before claiming results.`)
348348
const columns = data.columns.map((column) => column.toLowerCase())
349349
const idIndex = columns.indexOf(args.id_column.toLowerCase())
350350
const textIndex = columns.indexOf(args.text_column.toLowerCase())

‎packages/opencode/src/tool/registry.ts‎

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -428,8 +428,6 @@ export namespace ToolRegistry {
428428
// altimate_change start - register custom data engineering tools
429429
SqlExecuteTool,
430430
SqlMaterializeTool,
431-
// text_label is on by default; ALTIMATE_TEXT_LABEL=0 disables registration.
432-
...(process.env.ALTIMATE_TEXT_LABEL !== "0" ? [TextLabelTool] : []),
433431
SqlCheckTool,
434432
SchemaInspectTool,
435433
SqlAnalyzeTool,
@@ -587,6 +585,12 @@ export namespace ToolRegistry {
587585
tools.push(await AppRuntime.runPromise(AnswerCommit.AnswerCommitTool))
588586
}
589587
// altimate_change end
588+
// altimate_change start — text_label is analyst-scoped (and its read-only data-explorer delegate), on by
589+
// default; ALTIMATE_TEXT_LABEL=0 disables registration. Other agents never receive it.
590+
if ((agent?.name === "analyst" || agent?.name === "data-explorer") && process.env.ALTIMATE_TEXT_LABEL !== "0") {
591+
tools.push(await AppRuntime.runPromise(Effect.scoped(Effect.provide(TextLabelTool, FetchHttpClient.layer))))
592+
}
593+
// altimate_change end
590594
const result = await Promise.all(
591595
tools
592596
.filter((t) => {

‎packages/opencode/test/altimate/text-label-model.test.ts‎

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -3,12 +3,11 @@ import { afterEach, expect, spyOn } from "bun:test"
33
import { APICallError } from "ai"
44
import { Effect, Layer } from "effect"
55
import { CrossSpawnSpawner } from "@opencode-ai/core/cross-spawn-spawner"
6-
import { ModelV2 } from "@opencode-ai/core/model"
7-
import { ProviderV2 } from "@opencode-ai/core/provider"
86
import { Provider } from "../../src/provider/provider"
97
import { ModelID, ProviderID } from "../../src/provider/schema"
108
import { LLM } from "../../src/session/llm"
119
import { Session } from "../../src/session/session"
10+
import { Session as LegacySession } from "../../src/session"
1211
import { MessageID } from "../../src/session/schema"
1312
import { closeScratch, getScratch } from "../../src/altimate/native/connections/scratch"
1413
import { TextLabelTool } from "../../src/altimate/tools/text-label"
@@ -54,20 +53,21 @@ it.instance("uses the turn's session model with small calls and preserves stream
5453
const sessions = yield* Session.Service
5554
const session = yield* sessions.create({})
5655
const messageID = MessageID.ascending()
57-
yield* sessions.updateMessage({
56+
// TextLabelTool reads the persisted legacy MessageV2 row, so persist through the legacy session API.
57+
yield* Effect.promise(() => LegacySession.updateMessage({
5858
id: messageID,
5959
sessionID: session.id,
6060
role: "assistant",
6161
parentID: MessageID.ascending(),
62-
providerID: ProviderV2.ID.make(model.providerID),
63-
modelID: ModelV2.ID.make(model.id),
62+
providerID: model.providerID,
63+
modelID: model.id,
6464
agent: "analyst",
6565
mode: "analyst",
6666
path: { cwd: "/", root: "/" },
6767
time: { created: Date.now() },
6868
cost: 0,
6969
tokens: { input: 0, output: 0, reasoning: 0, cache: { read: 0, write: 0 } },
70-
})
70+
}))
7171

7272
const error = new APICallError({
7373
message: "Provider rejected unsupported thinkingBudget",

‎packages/opencode/test/altimate/tools/tool-schema-anthropic.test.ts‎

Lines changed: 8 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -73,8 +73,8 @@ function schemaErrors(schema: object | boolean, rejectEmptySentinels = false): s
7373
// altimate_change start — reject empty sentinel schemas in model-facing tools.
7474
if (rejectEmptySentinels) {
7575
if (value.const === "") errors.push(`${location}/const must not be an empty string`)
76-
if (Array.isArray(value.enum) && value.enum.length === 1 && value.enum[0] === "") {
77-
errors.push(`${location}/enum must not contain only an empty string`)
76+
if (Array.isArray(value.enum) && value.enum.includes("")) {
77+
errors.push(`${location}/enum must not contain an empty string`)
7878
}
7979
}
8080
// altimate_change end
@@ -156,7 +156,9 @@ describe("Anthropic tool JSON schemas", () => {
156156
const tools = yield* registry.tools({ providerID, modelID: target.id, agent })
157157
expect(tools.map((tool) => tool.id)).toEqual(expect.arrayContaining(["sql_check", "sql_execute", "read"]))
158158
// altimate_change start — validate the labeling schema with each Anthropic provider.
159-
expect(tools.some((tool) => tool.id === "text_label")).toBe(process.env.ALTIMATE_TEXT_LABEL !== "0")
159+
expect(tools.some((tool) => tool.id === "text_label")).toBe(
160+
name === "analyst" && process.env.ALTIMATE_TEXT_LABEL !== "0",
161+
)
160162
// altimate_change end
161163
expect(tools.some((tool) => tool.id === "answer_commit")).toBe(name === "analyst")
162164

@@ -209,6 +211,9 @@ describe("Anthropic tool JSON schemas", () => {
209211
const enabled = yield* registry.tools({ providerID: target.providerID, modelID: target.id, agent: analyst })
210212
expect(enabled.map((tool) => tool.id)).toContain("text_label")
211213
expect(PermissionNext.disabled(["text_label"], analyst.permission).has("text_label")).toBe(false)
214+
const builder = yield* agents.get("builder")
215+
const builderTools = yield* registry.tools({ providerID: target.providerID, modelID: target.id, agent: builder })
216+
expect(builderTools.map((tool) => tool.id)).not.toContain("text_label")
212217

213218
process.env.ALTIMATE_TEXT_LABEL = "0"
214219
expect(yield* registry.ids()).not.toContain("text_label")

0 commit comments

Comments
 (0)