Skip to content

Commit 2aac8cc

Browse files
anandgupta42claude
andcommitted
fix: give the data-explorer subagent the labeling cap and size the job above it
- `data-explorer.txt`: label at most 500 rows by any path; it cannot ask the user, so above that it samples and reports an estimate. - `analyst.txt` and `data-question` skill: above the cap, tell the user the job size (about one model call per 50 rows, or a per-row warehouse charge) along with the row count before asking to confirm. Co-Authored-By: Claude Opus 5.5 <[email protected]>
1 parent 6cda4f5 commit 2aac8cc

3 files changed

Lines changed: 3 additions & 2 deletions

File tree

‎.opencode/skills/data-question/SKILL.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -139,7 +139,7 @@ Dialect references: [DuckDB query_table](https://www.duckdb.org/docs/current/gui
139139

140140
Treat inferred labels as new data, not ground truth. Define a fixed rubric, allowed labels and uncertainty policy before classifying. For synthetic `messages(message_id, body)`, a rubric might classify a request to reverse a charge as `refund`, a question about shipping progress as `delivery`, and ambiguous text as `uncertain`. Specify tie-breaking for multi-topic messages.
141141

142-
Inspect text, then define the complete population of stable IDs and count it before classifying. Labeling with a model costs one call per batch, so it grows with the table and can cost far more than the analysis itself. Without the user's confirmation, label at most 500 rows in total by any path: your own reading, subagents, or warehouse AI functions (for example classification functions in Snowflake, Databricks or BigQuery), which are billed per row too. To label more, first tell the user the row count and how the labels would be produced, and wait for their confirmation. When the population is larger than 500 rows, by default label a documented random sample of up to 500 rows (roughly ±4.5 percentage points at 95% confidence for a proportion) and report the result as an estimate with its sample size, not as an exact count. Review uncertain cases consistently.
142+
Inspect text, then define the complete population of stable IDs and count it before classifying. Labeling with a model costs one call per batch, so it grows with the table and can cost far more than the analysis itself. Without the user's confirmation, label at most 500 rows in total by any path: your own reading, subagents, or warehouse AI functions (for example classification functions in Snowflake, Databricks or BigQuery), which are billed per row too. To label more, first tell the user the row count, how the labels would be produced and the size of the job (about one model call per 50 rows, or a per-row charge on their warehouse bill for warehouse AI functions), and wait for their confirmation. When the population is larger than 500 rows, by default label a documented random sample of up to 500 rows (roughly ±4.5 percentage points at 95% confidence for a proportion) and report the result as an estimate with its sample size, not as an exact count. Review uncertain cases consistently.
143143

144144
To reuse the checks below, form `labels(message_id, label, rubric_version)` for the labeled IDs; `population` must cover the same intended IDs (for a sample, the sampled IDs). Small audited annotation batches can use an available permitted ingestion path. When `scratch` and `sql_materialize` are available, these small batches can be expressed as a SELECT relation and materialized there. For example (DuckDB/SQLite SELECT syntax):
145145

‎packages/opencode/src/altimate/prompts/analyst.txt‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ Inspect connections, schemas and actual rows before joining or parsing; key form
44

55
Derive large intermediate sets in-engine. For a cross-engine join, move the smaller side: a few hundred keys go into an IN list or VALUES against the big table; materialize into `scratch` only the side that is small after filtering, never a multi-million-row table when a filter or aggregate could shrink it first. Preserve requested weighting: group means weighted by contributing counts can be exact; unweighted averages change the weighting. For many similar tables, prefer catalog-driven SQL (information_schema, DuckDB query_table/UNION ALL BY NAME) after confirming dialect support.
66

7-
Treat inferred text labels as data: fixed rubric, stable IDs, coverage and duplicate checks, then deterministic counting, never by eye. Count the population first. Labeling grows in cost with the row count: without the user's confirmation, label at most 500 rows in total by any path (your own reading, `data-explorer` subagents or warehouse AI functions). For a larger population, label a documented random sample of up to 500 rows and report an estimate with its sample size; label more only after telling the user the row count and getting their confirmation. When available, `task` with read-only `data-explorer` can verify reasoning.
7+
Treat inferred text labels as data: fixed rubric, stable IDs, coverage and duplicate checks, then deterministic counting, never by eye. Count the population first. Labeling grows in cost with the row count: without the user's confirmation, label at most 500 rows in total by any path (your own reading, `data-explorer` subagents or warehouse AI functions). For a larger population, label a documented random sample of up to 500 rows and report an estimate with its sample size; label more only after telling the user the row count, the size of the job (about one model call per 50 rows, or a per-row charge on their warehouse bill for warehouse AI functions) and getting their confirmation. When available, `task` with read-only `data-explorer` can verify reasoning.
88

99
## Cost-conscious exploration
1010
Use selected columns, partition filters and LIMIT/sampling to inspect cheaply; estimates are not exact final computations. Read JSONL spill paths with `read`; spills contain returned rows, not necessarily all matches. Aggregate in SQL for exact answers. On timeout, revise the query. Report costs only from evidence.

‎packages/opencode/src/altimate/prompts/data-explorer.txt‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,4 +6,5 @@ Rules:
66
- Use `sql_check` when keys, join fan-out, grain, nulls or extraction coverage could affect the delegated answer.
77
- Report exact values as returned (do not round or reformat), together with the exact SQL you ran and the table(s) it ran against.
88
- State limits plainly: row limits or truncation, empty results, ambiguous columns, queries that failed, and anything you could not verify. Never fill a gap with a guessed value.
9+
- Label or classify at most 500 rows in total, by reading them or with warehouse AI functions (these bill per row). You cannot ask the user for more, so if the sub-question needs more, label a documented random sample of up to 500 rows and report it as an estimate with its sample size and the full row count.
910
- Stay on the delegated sub-question. Keep the final reply short: the answer, the SQL used, and any caveats.

0 commit comments

Comments
 (0)