You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 7254d08
Browse filesBrowse the repository at this point in the historyBrowse files
fix: scope text_label to analyst and document the 30,000 row limit
- Register `text_label` only for `analyst` and `data-explorer` agents, not globally
- Oversized-input error and `data-question` skill now describe splitting into disjoint id ranges and verifying combined coverage
- Schema test rejects any enum containing an empty string
- Model-resolution test persists the assistant message through the legacy session API so its assertions execute
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Treat inferred labels as new data, not ground truth. Define a fixed rubric, allowed labels and uncertainty policy before classifying. For synthetic `messages(message_id, body)`, a rubric might classify a request to reverse a charge as `refund`, a question about shipping progress as `delivery`, and ambiguous text as `uncertain`. Specify tie-breaking for multi-topic messages.
141
141
142
-
Inspect text, then define the complete population of stable IDs. When `text_label` is available, materialize that population in `scratch` with `sql_materialize`, then call `text_label` with SQL over scratch, the stable ID and text column names, a fixed label set with a description for each label, and any rubric/tie-breaking instructions. It classifies every row through the session's LLM in bounded batches and persists the results in scratch; do not substitute regex or keyword guesses for semantic labels. Check `rows_in`, `labeled`, `failed` and `unparseable`, and retrieve failed IDs before claiming complete coverage. Optional spot-check agreement measures repeatability, not correctness. Review uncertain cases consistently; batching makes counting reproducible, not classification inherently correct.
142
+
Inspect text, then define the complete population of stable IDs. When `text_label` is available, materialize that population in `scratch` with `sql_materialize`, then call `text_label` with SQL over scratch, the stable ID and text column names, a fixed label set with a description for each label, and any rubric/tie-breaking instructions. It classifies every row through the session's LLM in bounded batches and persists the results in scratch; do not substitute regex or keyword guesses for semantic labels. `text_label` accepts at most 30,000 rows per call. For a larger population, split it into disjoint, deterministic id ranges (one call per range, each into a distinct result table), combine the persisted result tables in SQL, and verify that combined coverage equals the population count before claiming results. Check `rows_in`, `labeled`, `failed` and `unparseable`, and retrieve failed IDs before claiming complete coverage. Optional spot-check agreement measures repeatability, not correctness. Review uncertain cases consistently; batching makes counting reproducible, not classification inherently correct.
143
143
144
144
Use the persisted `text_label` output for SQL counts. It retains the source ID column name, so use `message_id` as the ID column for this example. To reuse the checks below, select `message_id`, `label` and your declared rubric version as `rubric_version` in a SELECT/CTE named `labels`; `population` must cover the same intended IDs. If `text_label` is unavailable, small audited annotation batches can use an available permitted ingestion path to form `labels(message_id, label, rubric_version)`. When `scratch` and `sql_materialize` are available, these small batches can be expressed as a SELECT relation and materialized there. For example (DuckDB/SQLite SELECT syntax):
returnfail(`Input exceeds the ${MAX_ROWS} row limit; split the population into disjoint queries.`)
347
+
returnfail(`Input exceeds the ${MAX_ROWS} row limit; split the population into disjoint, deterministic id ranges (one call per range, each into a distinct target_table), then combine the persisted result tables in SQL and verify combined coverage equals the population count before claiming results.`)
0 commit comments