Skip to content

Commit de3d05a

Browse files
anandgupta42claude
andauthored
feat(telemetry): add masked error message and source stack frames to error_fingerprint (#1544)
* feat(telemetry): add a masked error message template to `error_fingerprint` `error_fingerprint` carried only a hash of the error. It now also carries `error_message`: an allowlist template of the error's first line in which identifiers, numbers, quoted text, names and credential values become `<x>`. `maskString` and the fingerprint hash input are unchanged. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * fix(telemetry): mask the rest of the line after any credential label A token with a part starting with a credential stem (pass, secret, token, key, auth, credential, bearer, pwd, cookie, session) now masks everything after it on the line. `client_secret: a b c`, `access_token=x y` and `passphrase a b c` sent their values before. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * fix(telemetry): cut the error line at the first credential word, path or escape Quoted labels (`"password": x`), backslash-escaped quotes and paths containing spaces could carry values past the token-by-token template. The line is now cut at the token holding the first credential stem or path separator and the rest masked. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * docs: qualify the SQL privacy claim for the error_fingerprint template Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * fix(telemetry): cut the error line at a backtick-quoted span Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * fix(telemetry): cut the error line at the first non-ASCII character Typographic quotes and other non-ASCII delimiters wrapped user content the ASCII quote handling did not see. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * refactor: build `error_message` from a fixed vocabulary of error words Split the first line into letter runs and keep only words in `ERROR_VOCABULARY`; every other run becomes `<x>`. Drops the punctuation/quote/credential-stem heuristics, which could not be made complete. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> --------- Co-authored-by: Claude Opus 5.5 (1M context) <[email protected]>
1 parent 3ed4561 commit de3d05a

5 files changed

Lines changed: 550 additions & 15 deletions

File tree

‎docs/docs/reference/telemetry.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -49,7 +49,7 @@ We collect the following categories of events:
4949
| `task_outcome_signal` | Behavioral quality signal at session end — accepted, error, abandoned, or cancelled. Includes tool count, step count, duration, and last tool category. No user content. |
5050
| `task_classified` | Intent classification of the first user message using keyword matching — category (e.g. `debug_dbt`, `write_sql`, `optimize_query`), confidence score, and detected warehouse type. No user text is sent — only the classified category. |
5151
| `tool_chain_outcome` | Aggregated tool execution sequence at session end — ordered tool names (capped at 50), error count, recovery count, final outcome, duration, and cost. No tool arguments or outputs. |
52-
| `error_fingerprint` | Hashed error pattern for anonymous grouping — SHA-256 hash of masked error message, error class, tool name, and whether recovery succeeded. Raw error content is never sent. |
52+
| `error_fingerprint` | Error pattern for grouping — SHA-256 hash of the masked error message, error class, tool name, whether recovery succeeded, plus `error_message`. `error_message` contains only words from a fixed list of common error terms taken from the first line of the error; every other word is replaced with `<x>`, so no user text, identifiers, paths, values or SQL can appear (max 200 characters). |
5353
| `sql_fingerprint` | SQL structural shape via AST parsing — statement types, table count, function count, subquery/aggregation/window function presence, and AST node count. No table names, column names, or SQL content. |
5454
| `schema_complexity` | Warehouse schema structural metrics from introspection — bucketed table, column, and schema counts plus average columns per table. No schema names or content. |
5555
| `validator_check` | A completion-gate validator ran on session end — validator name, `ok` boolean, step, retry count, `enforced` flag (false in shadow mode), and structured `details` (model counts, elapsed time, concurrency limit — no SQL or model content). Only emitted when `ALTIMATE_VALIDATORS_ENABLED=1` or `ALTIMATE_VALIDATORS_SHADOW=1`. See [Validators](../data-engineering/validators.md). |
@@ -210,7 +210,7 @@ Event type names use **snake_case** with a `domain_action` pattern:
210210
- `error_recovered` for error recovery events
211211
- `task_outcome_signal`, `task_classified` for session quality signals
212212
- `tool_chain_outcome` for tool execution chain aggregation
213-
- `error_fingerprint` for anonymous error pattern grouping
213+
- `error_fingerprint` for error pattern grouping, with a one-line error message built from a fixed vocabulary of common error words
214214
- `sql_fingerprint` for SQL structural analysis
215215
- `schema_complexity` for warehouse schema metrics
216216

0 commit comments

Comments
 (0)