Tool invoke rates: tool, MCP server, skill and subagent names in Loki and Grafana (stacked on #364) by seathatflowsinourveins · Pull Request #366 · native-agent-stack/native-agent-stack

seathatflowsinourveins · 2026-09-26T20:34:04Z

Summary

Tool, MCP server, skill and subagent invoke rates become visible in Loki and Grafana, and no command,
prompt or tool content is stored.

Collector: a new processor, transform/tool_names, runs in both logs pipelines just before
transform/privacy. It copies names out of the events. The widened log allowlist then keeps only
those names and a few derived fields: tool_family, actor, client, shell_rtk and
tool_details.
Dashboard: row 28, panels 29-38, counts calls per minute by family, MCP server, skill, client
and actor. It also shows subagent and workflow launches, MCP share, ctx and rtk adoption, and an
integrity panel whose values should all be 0.
Claude Code: OTEL_LOG_TOOL_DETAILS goes from "false" to "1" in both Claude settings
examples. That is the only way Claude Code exports MCP server, tool and subagent names. Codex
0.157.1 needs no key change.

User decision. On 2026-09-26, in the workstation coordinator session, the user answered the
question about enabling OTEL_LOG_TOOL_DETAILS=1 with Collector-side name-only filtering: "1 and
full sota convergence practice we proceed". This PR records it as a dated exception in
docs/secret-storage.md ("Telemetry and pasted values"). The exception names the control:

transform/tool_names keeps only MCP server and tool names, activated-skill names and built-in
agent types;
tool_parameters, tool_input (and with them full_command) and user.email are never
exported;
the canary proof described below.

Stacked on #364 (claude/monitor-metrics-integrity-20260926, head dfd5460f). This PR's
base is that branch, so its diff shows only this change. It retargets to main after #364
merges: rebase with git rebase --onto origin/main dfd5460f and push with --force-with-lease.
While the base is not main, the required verdict-review-gate fails closed by design ("the gate
judges only pull requests into main"). The rebase push after the retarget rebuilds its run on main.

Scope

The reviewed change:
- observability/collector/collector.yaml: transform/tool_names, the log allowlist and the
  logs pipelines;
- observability/backends/templates/ecosystem-dashboard.json.example: row 28;
- OTEL_LOG_TOOL_DETAILS in adoption/templates/claude.settings.template.json and
  observability/collector/claude-settings.json.example;
- the Collector and backends READMEs;
- a new decision record, docs/decisions/2026-09-26-tool-invoke-rates.md;
- a new test module, tests/test_observability_tool_names.py.
Added in this commit after the reviews:
- The dated exception in docs/secret-storage.md, with the quote, the control and the canary
  proof. The marker "changed after v2026.09.26.2" sits on one line, and that paragraph quotes
  no other tag.
- The decision record's status line.
- One clause each in adoption/bootstrap.md and adoption/platforms/macos-arm64.md. Those pages
  already list the template's changes after v2026.09.26.2 under that marker; the clause adds
  the tool-details flag.
- The test fixture's two thread ids are now built at runtime with uuid.UUID(int=…). Values are
  unchanged. scripts/validate.py refused the literal UUIDs as possible local session ids, and
  the repository's other tests already use this idiom.
Base commit: dfd5460f, the head of Telemetry writer identity (G1): one writer per token series, integrity alerts and a committed host recipe #364. The patch applied with git apply --check exit 0,
and every applied file equals the reviewed stage byte for byte.
Lane: lane:foundation. manifests/evidence.json is only re-registered, and the two generated
reports are unchanged.

What was verified

Claim	Evidence class	Command / receipt
The staged config is valid	native	`otelcol-contrib 0.161.0 validate` exits 0. Copies with an undefined function or a broken `replace_pattern` exit 1, so the new statements are parsed.
Replay of real captures	local integration	`replay-test.sh`: 109 passed, 0 failed. 567 records (536 captured from Claude Code 2.1.283 and codex-cli 0.157.1, 31 synthetic) went through the staged pipeline on the pinned otelcol-contrib and a scratch Loki 3.7.8 built from the repository template. Every stream's labels are exactly `{service_name}`. All 62 metadata keys are allowlisted. All 19 dashboard targets equal values computed from the exported records, and prove.sh's 21 Loki checks pass.
Canary proof	local integration	72 known strings (31 from captures, 41 synthetic): Bash commands, prompts, a workflow script, search queries, Codex `arguments`, `cmd` and `output`, and agent messages. 0 were found in the file exporter or Loki. No `tool_parameters`, `tool_input` or `user.*` key was exported.
Unit tests	synthetic	`tests/test_observability_tool_names.py`: 11 of 11, including the pinned-binary class. It fails on `dfd5460f` alone with 1 failure and 10 errors.
Repository checks on this head	local integration	`scripts/validate.py` (6,936 hashed files, 159 receipts), `evidence_manifest.py --check`, `component_matrix.py --check`, `new_host_grand_list.py --check` and `build_ecosystem.py --check` all exit 0. 16 related modules (observability, adoption pages, secret guard, dashboards, validate): 614 OK (skipped=20). Full `unittest discover -s tests -q` on `a9c3a4a5`: 6,292 tests OK (skipped=635), first run, no gitleaks-lock rerun needed.
Host helpers	local integration	on synthetic host copies and settings files: merge/install/rollback 38 of 38, flag record/restore 33 of 33
Production Loki, read-only, before any apply	host observation	the 19 staged targets and the 15 proof queries parse and return. Over the preceding hour: about 83 Claude and 43 Codex tool calls per minute.

Reviews (two GPT-6 cross-family rounds, codex exec, gpt-6-astra, read-only; the notes are in
the session scratchpad):

Round 1 (u6/design/codex-review.txt) raised 3 high, 5 medium and 1 low findings. Eight were
fixed. The ninth, that shape checks cannot bound meaning, was recorded as a limit.
Round 2 (u6/review-gpt6.md) demonstrated that limit and raised 2 high, 4 medium and 1 low
findings. All are fixed; u6/REVIEW-RESOLUTION.md has each with its evidence:
1. Model-typed values are now dropped or mapped to a closed list: the Skill tool's skill_name,
  workflow names and Codex agent paths are never exported, and agent types outside the
  documented built-ins become custom.
2. The host merge helper keeps every host processor and statement and refuses unexpected
  configurations.
3. A normalized client separates app-server front-ends.
4. Claude launches are counted at the accepted tool_decision, which is logged before the tool
  runs.
5. Install and rollback check hashes and refuse drift.
6. The flag helper records and restores an originally absent key.
7. Panel 38 is retitled.
No Claude adversarial round is recorded for this unit, and no third round was run.

Host-apply plan (not run yet)

$REPO is this branch's checkout, which holds both this change and #364. The host helpers
(merge_collector_logs.py, claude_tool_details.py and prove.sh) are in the session scratchpad.
The full plan is in the scratchpad's u6/DESIGN.md, "Apply and rollback".

Dry run: merge_collector_logs.py merge --diff on the host collector.yaml, then
otelcol-contrib validate. It prints host_sha256 and candidate_sha256 and makes exactly two
edits: transform/tool_names before transform/privacy in each logs pipeline, and the missing
keys on the one log keep_keys.
Quiet window: no Workflow or Dagu run in flight, near-zero tool events in Loki, and an
exporter queue of 0.
Backup and flag record: create ${XDG_STATE_HOME:-~/.local/state}/native-agent-stack/u6-tool-names/backup-<UTC>/
with mode 0700, then run claude_tool_details.py --record.
Install: merge_collector_logs.py install with both expected hashes. Do not restart.
Collector and restart: the Telemetry writer identity (G1): one writer per token series, integrity alerts and a committed host recipe #364 recipe runs
apply.sh --apply --repo "$REPO" --only prometheus,dashboards,collector,claude-setting with the
Codex launcher excluded. It merges its metrics side into the installed file and restarts the
Collector once.
Client flag: claude_tool_details.py --set 1, the recorded user decision. It changes one key,
and only new sessions pick it up.
Proof: prove.sh runs one claude -p and one codex exec tagged with ecosystem.task.id.
It checks the name counts in Loki within 120 s and runs the canary check on production: no probe
prompt sentence or command may appear in Loki or events*.jsonl.

Rollback:

Run u1's rollback.sh --apply first.
Run merge_collector_logs.py rollback --backup-dir "$B/collector" and restart the Collector.
Run claude_tool_details.py --restore … --expect-current 1.

Each step refuses a file or key that changed after the apply.

Residuals

Host apply and host acceptance (prove.sh) have not run. The host helpers and the sanitized
replay receipt are not committed; the decision record commits the receipt together with the host
proof.
MCP server, MCP tool and skill names are bounded by shape only, because they come from the host's
configuration. A server or skill whose configured name carries private text would export it.
Agent-subagent tool calls count as main_or_subagent. Per-call attribution needs traces, which
are off.
api_request attribution counts requests, so several MCP results consumed by one request count
once.
Custom agents show as custom. An accepted launch that is interrupted before it runs still counts.
client separates app-server processes, not connections. Codex spawn messages carry no
originator, so spawns stay per service.
Codex functions/wait counts as code_mode, although multi-agent v1 also names a tool wait.
shell_rtk reads only the first word of a command.
Sessions started before the flag change carry no Claude names.
The installer has a short window between its last hash check and the rename, and the restart
stays a manual step.
Codex identity launcher: out of scope (Telemetry writer identity (G1): one writer per token series, integrity alerts and a committed host recipe #364 residual). This change's Codex actor, client
and spawn series do not depend on it.
Side findings for Telemetry writer identity (G1): one writer per token series, integrity alerts and a committed host recipe #364, not changed here:
- otelcol-contrib 0.161.0 logs that the otlphttp alias is deprecated;
- merge_collector.py replaces the whole metrics processor list and metric_statements.

SOTA sources

Claude Code docs, fetched 2026-09-26:
- https://code.claude.com/docs/en/monitoring-usage: OTEL_LOG_TOOL_DETAILS, tool_parameters
  (mcp_server_name, mcp_tool_name, subagent_type, bash_command, full_command),
  tool_decision, tool_result, skill_activated, subagent_completed and api_request
  (query_source);
- https://code.claude.com/docs/en/sub-agents, "Built-in subagents".
open-telemetry/opentelemetry-collector-contrib v0.161.0:
- processor/transformprocessor/README.md (error_mode, group conditions, cache);
- pkg/ottl/ottlfuncs/README.md (ParseJSON, replace_pattern, ContainsValue,
  delete_matching_keys, keep_keys, IsMatch);
- pkg/ottl/LANGUAGE.md and pkg/ottl/contexts/ottllog/README.md;
- exporter/fileexporter/README.md.
grafana/loki v3.7.8:
- docs/sources/send-data/otel/_index.md (OTLP attributes become structured metadata);
- docs/sources/get-started/labels/structured-metadata.md and bp-labels.md;
- docs/sources/query/log_queries/_index.md (keep).
grafana/grafana v13.2.2: docs/sources/datasources/loki/template-variables/index.md
($__auto).
prometheus/prometheus v3.15.0: docs/feature_flags.md. The Prometheus features come from
Telemetry writer identity (G1): one writer per token series, integrity alerts and a committed host recipe #364, and this change adds no metric dimension.
Codex [otel] configuration: https://developers.openai.com/codex/config-advanced, "Observability
and telemetry" (exporter, log_user_prompt, [otel.tool_result] max_bytes), fetched
2026-09-26.
openai/codex rust-v0.157.1, the log fields and originators:
- codex-rs/otel/src/events/shared.rs;
- codex-rs/protocol/src/agent_path.rs;
- codex-rs/login/src/auth/default_client.rs;
- codex-rs/app-server/src/lib.rs;
- codex-rs/core/src/tools/handlers/multi_agents_v2/spawn.rs.

Decision record

docs/decisions/2026-09-26-tool-invoke-rates.md covers the alternatives, the overturn conditions
and the limits.

🤖 Generated with Claude Code

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a9c3a4a572

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

seathatflowsinourveins · 2026-09-26T21:04:08Z

CI status before the retarget, for the record:

verdict-review-gate fails closed by design while the base is not main: "the pull request's base branch is 'claude/monitor-metrics-integrity-20260926', not main".
validate hit the job's timeout-minutes: 10 twice on run 36270315667; neither was a test failure.
- Attempt 1: the full suite finished with Ran 6292 tests in 541.241s, OK (skipped=679), and the job was cancelled in the same second.
- Attempt 2, one re-run: the job was cancelled mid-suite after about 545 s.
- For comparison, Telemetry writer identity (G1): one writer per token series, integrity alerts and a committed host recipe #364's validate passed in 8 m 30 s (Ran 6307 tests in 447.251s). validate.yml is identical on both bases, so this is runner speed against the 10-minute limit.
Locally on the same head a9c3a4a5, the full suite passes: Ran 6292 tests, OK (skipped=635).

The rebase onto main after #364 merges re-runs every check, with verdict-review-gate judging this PR against main.

…cy checker, correct client grouping Resolves the review threads on #366 and the window-2 GPT-6 findings 2 and 5. - evidence/artifacts/tool-invoke-rates-20260926/: - the replay harness, its reference implementation and the sanitized 109/109 result (otelcol-contrib 0.161.0, Loki 3.7.8); - the live host proof (2026-09-26T23:47:42Z-23:49:21Z, 33 passed), with counts and hashes only for real captures; - the fixed checker. - prove_check.py and prove.sh: no length floor on forbidden strings, and every tagged record's body must equal the Collector placeholder exactly. - Synthetic A/B: the old checker passed a leaked "pwd" body; the fixed one fails it. - Offline re-check of the retained events-file sink of the 23:47Z proof: 29 tagged lines, all "[content omitted]", 0 forbidden hits. Its Loki sink was not re-checked by the fixed checker, and the decision record says so. - client is documented as a grouping by normalized originator name: separate app-server processes with the same originator share one label. - Panel 34: one description sentence. LogQL rate over unwrap is sum/range (Loki v3.7.8 pkg/logql/range_vector.go rateLogs), so the query is unchanged. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

seathatflowsinourveins · 2026-09-27T03:48:47Z

GPT-6 cross-family review (gpt-6-astra, max effort, read-only) at 5ddfd93: DEFECTS FOUND. 1 high, 2 medium, 1 low. The repair round is in progress.

Severity	Where	Finding
high	`checker-fix/prove_check.py:149`	Only the last body in each OTLP batch is checked, so a `"pwd"` body followed by a placeholder body passes, and missing bodies pass too. The offline re-check inherits this.
medium	`checker-fix/prove_check.py:150`	Resource, scope and nested attribute values are not scanned, and the banned-key check needs minified JSON.
medium	`scratch-replay/replay.py:266`	The documented synthetic-only replay fails on an empty capture (`max()`), and its historical assertions need the omitted private captures.
low	`docs/secret-storage.md:740`	"It has not run yet" contradicts the retained live proof.

Until the repair lands, the receipt's privacy claims (the offline re-check and the checker fix) are not accepted evidence.

To run the CI of #374, #377 and #366 in parallel, the three are now stacked: #377 sits on #374, and #366 will sit on #377. Each squash merge then lands exactly its CI-tested tree. The repaired, stacked #366 head will be pushed after verification.

🤖 Generated with Claude Code

…cy checker, correct client grouping Resolves the review threads on #366 and the window-2 GPT-6 findings 2 and 5. - evidence/artifacts/tool-invoke-rates-20260926/: - the replay harness, its reference implementation and the sanitized 109/109 result (otelcol-contrib 0.161.0, Loki 3.7.8); - the live host proof (2026-09-26T23:47:42Z-23:49:21Z, 33 passed), with counts and hashes only for real captures; - the fixed checker. - prove_check.py and prove.sh: no length floor on forbidden strings, and every tagged record's body must equal the Collector placeholder exactly. - Synthetic A/B: the old checker passed a leaked "pwd" body; the fixed one fails it. - Offline re-check of the retained events-file sink of the 23:47Z proof: 29 tagged lines, all "[content omitted]", 0 forbidden hits. Its Loki sink was not re-checked by the fixed checker, and the decision record says so. - client is documented as a grouping by normalized originator name: separate app-server processes with the same originator share one label. - Panel 34: one description sentence. LogQL rate over unwrap is sum/range (Loki v3.7.8 pkg/logql/range_vector.go rateLogs), so the query is unchanged. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

…ucturally; synthetic-only replay runs natively Findings from the cross-family review of 5ddfd93, each fixed with failing-first controls: - high: prove_check.py kept only the last body of each OTLP batch, and missing or non-string bodies passed. scan_otlp now keeps and validates every record's body. - medium: resource, scope and nested (ArrayValue/KeyValueList) values were not scanned, and banned keys matched only minified JSON. The scan now traverses decoded keys and values at every OTLP level (opentelemetry-proto v1.9.0 logs.proto and common.proto). The live and offline paths share one events_privacy_checks(). Structural controls: 2 passed, 10 failed before; 12 passed after. - medium: the synthetic-only replay failed on max() of an empty capture and required the omitted private captures. It now has a default timestamp and runs the historical assertions only when the full capture set is present. - Stubbed control: 25/20 before, 25/0 after. - Native run by the coordinator on the host, on the pinned Collector 0.161.0 and Loki 3.7.8 with scratch ports 45700-45703: 67 passed, 0 failed. - low: docs/secret-storage.md still said the live proof had not run. The corrected offline re-check of the 23:47Z proof's retained events file ran at 03:55:26Z: 270 records in 29 batches, 3 passed. The exporter (10 MB x 3 backups) rotated that file out at 04:05:11Z, and a re-run at 04:14Z found 0 tagged lines. The receipt records that the result cannot be repeated. The three observability test modules pass on the host (78 OK), where the sandbox had blocked 21 socket tests. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

seathatflowsinourveins · 2026-09-27T04:19:40Z

Resolutions of the GPT-6 review at 5ddfd93. Repaired in 7a15a2e, now stacked on #377 (b6cab3e), which sits on #374.

Severity	Finding	Resolution	Evidence
high	Only the last body per OTLP batch was checked; missing or non-string bodies passed	`scan_otlp` keeps and validates every record's body	Structural controls: 2 passed / 10 failed before, 12 / 0 after (`failing_first_ab.red.out`, `.out`)
medium	Resource, scope and nested values were not scanned; banned keys matched only minified JSON	Structural traversal of decoded keys and values at every OTLP level (opentelemetry-proto v1.9.0), with one shared `events_privacy_checks()` for the live and offline paths	Same controls, cases (c)-(e)
medium	The synthetic-only replay failed on `max()` of an empty capture and needed private captures	Default timestamp; historical assertions run only when the full capture set is present	Stubbed control 25/20 before, 25/0 after. Native run on the host (Collector 0.161.0, Loki 3.7.8, scratch ports 45700-45703): 67 passed, 0 failed (`scratch-replay/synthetic-native.out`)
low	`docs/secret-storage.md` said the proof had not run	Now records the 23:47Z proof (33 passed, pre-fix checker) and the current limits	—

The corrected offline re-check cannot be repeated.

It ran at 03:55:26Z against the proof's retained events file: 270 records in 29 batches, 3 passed.
The exporter keeps 10 MB x 3 backups and rotated that file out at 04:05:11Z.
My re-run at 04:14Z found 0 tagged lines (offline_recheck.rerun-after-rotation.out).
The receipt states this. The Loki sink of that proof remains verified only by the pre-fix checker.

Host test results:

The three observability modules pass (78 OK). The sandbox had blocked 21 socket tests.
439 related tests OK; validate FAILS=0; the privacy scan is clean on all 17 changed files.

A scoped GPT-6 re-check of 7a15a2e is running.

🤖 Generated with Claude Code

seathatflowsinourveins · 2026-09-27T04:54:04Z

GPT-6 re-check of 7a15a2e found 2 more medium issues. Both are fixed in 58451c1 with failing-first controls.

Finding	Fix	Evidence
`prove_check.py`: a base64 ProtoJSON `bytesValue` was scanned only encoded, so `cHdk` (`pwd`) passed	`scan_otlp()` also scans the decoded text; an undecodable value fails closed	12 passed / 3 failed before, 15 / 0 after (`failing_first_ab.bytes-red.out`, `.out`)
`replay-test.sh`: Loki's fixed-body assertion ran only with the historical captures	`replay.py loki_body_checks()` runs on every replay	`loki_body_control.py` runs the real `loki()` against a stubbed Loki. With the prior `replay.py`, no body check ran and 31 `pwd` lines went unnoticed (0/2); after the fix, the clean records pass and the leak is caught (2/2). Native synthetic replay on the host: 68 passed, 0 failed

Residual: Loki's structured metadata stores attribute values as strings, so a bytes-typed attribute stays base64 in the Loki sink. The events-file sink keeps the type and decodes it.

This closes the review loop for this PR: one review, a repair, a re-check and a second repair. Any further findings will be recorded as residuals rather than starting a third round. 439 related tests OK, validate FAILS=0, and the privacy scan is clean on all changed files.

#377 is now merged, so the next step is to rebase this PR onto main for its final CI run.

🤖 Generated with Claude Code

… and Grafana, with the dated tool-details exception Stacked on the telemetry writer identity change (dfd5460). - Collector: transform/tool_names runs in both logs pipelines before transform/privacy. From Claude Code's tool_parameters it copies only the MCP server and tool names and the Agent tool's subagent_type, plus one boolean, shell_rtk. Skill names come from skill_activated. Registry names that are not short identifiers become "other", and agent types outside the documented built-ins become "custom". tool_parameters, tool_input, arguments, output, content and error are deleted, and the log allowlist gains 19 name, id and derived keys. - Dashboard row 28 (panels 29-38): calls by family, MCP server, skill, client and actor; subagent and workflow launches; MCP share; ctx and rtk adoption; and an integrity panel. - OTEL_LOG_TOOL_DETAILS is "1" in both Claude settings examples. docs/secret-storage.md records it as a dated exception. On 2026-09-26, in the workstation coordinator session, the user answered "1 and full sota convergence practice we proceed". The exception states the control and the canary proof, and marks the template changed after `v2026.09.26.2`. The decision record's status and the new-host template notes in bootstrap.md and macos-arm64.md say the same. - tests/test_observability_tool_names.py: its two fixture thread ids are now built at runtime with uuid.UUID(int=...), with the same values. scripts/validate.py rejected the UUID literals as possible local session ids. Evidence (local integration and synthetic; the host is not applied): otelcol-contrib 0.161.0 validate exits 0. The replay of real Claude Code and Codex captures plus synthetic records passed 109 of 109 checks: 72 canary strings, 0 found in the file exporter or Loki; 19 of 19 dashboard targets. The unit test passes 11 of 11 and fails on dfd5460 alone. validate.py, evidence_manifest --check and the related modules pass. Reviews: two GPT-6 cross-family rounds; every high and medium finding is fixed. Co-Authored-By: Claude Opus 5.5 <[email protected]>

…cy checker, correct client grouping Resolves the review threads on #366 and the window-2 GPT-6 findings 2 and 5. - evidence/artifacts/tool-invoke-rates-20260926/: - the replay harness, its reference implementation and the sanitized 109/109 result (otelcol-contrib 0.161.0, Loki 3.7.8); - the live host proof (2026-09-26T23:47:42Z-23:49:21Z, 33 passed), with counts and hashes only for real captures; - the fixed checker. - prove_check.py and prove.sh: no length floor on forbidden strings, and every tagged record's body must equal the Collector placeholder exactly. - Synthetic A/B: the old checker passed a leaked "pwd" body; the fixed one fails it. - Offline re-check of the retained events-file sink of the 23:47Z proof: 29 tagged lines, all "[content omitted]", 0 forbidden hits. Its Loki sink was not re-checked by the fixed checker, and the decision record says so. - client is documented as a grouping by normalized originator name: separate app-server processes with the same originator share one label. - Panel 34: one description sentence. LogQL rate over unwrap is sum/range (Loki v3.7.8 pkg/logql/range_vector.go rateLogs), so the query is unchanged. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

…re-register U6 on main Correcting the comment from #364 (8e66134). A bash 3.2.57 re-check shows that nested quoted expansions inside $(...) parse correctly. What breaks is a case pattern's closing parenthesis, which ends the substitution early. bash -n passes; at run time the shell prints a syntax error and continues with a wrong value. U6 is rebased onto main 881164e: its two commits, with manifests/evidence.json re-registered and the matrix and grand list regenerated. validate FAILS=0. 438 observability and manifest tests OK; the writer-identity modules also pass under bash 3.2.57 as bash and sh (67 OK). Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

…ucturally; synthetic-only replay runs natively Findings from the cross-family review of 5ddfd93, each fixed with failing-first controls: - high: prove_check.py kept only the last body of each OTLP batch, and missing or non-string bodies passed. scan_otlp now keeps and validates every record's body. - medium: resource, scope and nested (ArrayValue/KeyValueList) values were not scanned, and banned keys matched only minified JSON. The scan now traverses decoded keys and values at every OTLP level (opentelemetry-proto v1.9.0 logs.proto and common.proto). The live and offline paths share one events_privacy_checks(). Structural controls: 2 passed, 10 failed before; 12 passed after. - medium: the synthetic-only replay failed on max() of an empty capture and required the omitted private captures. It now has a default timestamp and runs the historical assertions only when the full capture set is present. - Stubbed control: 25/20 before, 25/0 after. - Native run by the coordinator on the host, on the pinned Collector 0.161.0 and Loki 3.7.8 with scratch ports 45700-45703: 67 passed, 0 failed. - low: docs/secret-storage.md still said the live proof had not run. The corrected offline re-check of the 23:47Z proof's retained events file ran at 03:55:26Z: 270 records in 29 batches, 3 passed. The exporter (10 MB x 3 backups) rotated that file out at 04:05:11Z, and a re-run at 04:14Z found 0 tagged lines. The receipt records that the result cannot be repeated. The three observability test modules pass on the host (78 OK), where the sandbox had blocked 21 socket tests. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

…lue; always check Loki bodies - A ProtoJSON bytesValue (base64, standard or URL-safe, with or without padding) was scanned only as encoded text, so bytesValue "cHdk" (pwd) passed. scan_otlp now scans the decoded text too. An undecodable value adds a marker that the checks treat as a banned key, so they fail closed. Controls: 12 passed, 3 failed before (bytes attribute, nested bytes, undecodable); 15 passed, 0 failed after. - Loki's fixed-body privacy assertion ran only with the historical capture set. replay.py loki_body_checks() now runs on every replay: every stored Loki line must equal the Collector placeholder. loki_body_control.py runs the real loki() against a stubbed Loki: on the prior replay.py no body check ran and 31 "pwd" lines went unnoticed (0/2); after the fix, clean records pass and the leak is caught (2/2). - The native synthetic replay on the host (Collector 0.161.0, Loki 3.7.8, scratch ports) now reports 68 passed, 0 failed. - Residual: Loki structured metadata keeps attribute values as strings, so a bytes-typed attribute stays base64 there. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

Replayed onto main: the bootstrap.md note merges after #366's OTEL tool-details sentence, manifests/evidence.json is re-registered, and the matrix and grand list are regenerated. validate FAILS=0; 438 related tests OK. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

…be, measured RTK rule) (#378) * SubagentStart token-lanes carrier for non-blind subagents Adds a stdlib, fail-open SubagentStart hook that returns adoption/hooks/claude/token-lanes-block.md as additionalContext to every subagent except agent types starting "blind-", registers it in the Claude settings template beside ai-memory, and installs both files through the existing sha256-checked guard step. docs/token-session-handbook.md "Token lanes carried into subagents" is the source of truth the block copies line for line. Its RTK line is the measured RTK 0.50.0 rule (rtk hook check and the rtk hook claude PreToolUse hook; rewrite_multiline_block, rtk-ai/rtk#3319): && chains and multi-line blocks are rewritten segment by segment; $(...), backticks, <(...), file redirects and heredocs are never rewritten; gh pipelines are not rewritten. Native probe (Claude Code 2.1.283, Agent-tool Haiku children): with the hook the child found and quoted the block's first 60 characters exactly; without it, and for a blind-judge child with it, the child found nothing. Decision record: docs/decisions/2026-09-27-token-lanes-subagent-start.md. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * Native probe and RTK hook-check receipt for the token-lanes carrier evidence/artifacts/token-lanes-subagent-start-20260927/ retains, sanitized, the five native probe runs (Claude Code 2.1.283, Agent-tool Haiku children: without the hook, with it, and a blind-judge child with it, plus the failed v1 question kept as a failed design) and 17 RTK 0.50.0 inputs through both `rtk hook check` and the `rtk hook claude` PreToolUse processor. Raw CLI output and child transcripts stay private and are identified by SHA-256 and counts. Evidence class local_integration; not native_proven for Workflow children, which have not run with the hook. The handbook's RTK sources paragraph and the decision record now cite the receipt instead of unretained observations. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * Count the carrier block's budget in UTF-8 bytes (cross-family review) The GPT-6 review of #378 at 85dd27c found that the text test checked len(block), a character count, although the decision record states a 3,500-byte budget: a 3,280-character, 3,520-byte block would pass. The test now measures the file's bytes through fits_budget() and adds that case as a regression test. Controls: a character-counting fits_budget fails the regression test, and a block padded to 3,560 bytes (3,360 characters) fails the byte check but passed the old character check. The committed block is ASCII, 3,159 bytes. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * Register the carrier's files on main 1aa9776 (after #366 and #380) Replayed onto main: the bootstrap.md note merges after #366's OTEL tool-details sentence, manifests/evidence.json is re-registered, and the matrix and grand list are regenerated. validate FAILS=0; 438 related tests OK. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> --------- Co-authored-by: Scout <scout@local> Co-authored-by: Claude Opus 5.5 (1M context) <[email protected]>

Correct the Claude custom-attribute citation and resource/event/datapoint description, whole-run prefix queries, deployment path and dated E2E gates. Publish original and repair command/result excerpts directly in the PR body. Explicitly delete Codex arguments with native OTTL and preserve tool namespace, guarded MCP server, timing, success, correlation and supplied size metadata. Test duplicate Claude run attributes and Codex records with and without server metadata through installed Collector 0.161.0 and Loki 3.7.8. Sources: open-telemetry/opentelemetry-collector-contrib v0.161.0, processor/transformprocessor/README.md and pkg/ottl/ottlfuncs/README.md; openai/codex rust-v0.157.1, codex-rs/otel/src/tool_result.rs:55-79,94, codex-rs/otel/src/events/shared.rs:59-60 and codex-rs/core/src/tools/call_trace.rs:38-54; https://code.claude.com/docs/en/monitoring-usage#multi-team-organization-support; https://grafana.com/docs/loki/latest/query/log_queries/ and send-data/otel/; open-telemetry/opentelemetry-proto v1.9.0 docs/specification.md; base ec27a30, #364 c71d66d and #366 a464d28 historical receipts. Verification: failing-first C1 checks, then 83 covering unittests pass with no skips; complete native Collector validate and git diff --check pass. Evidence is local integration with synthetic fixtures, not live-provider/host acceptance. The coordinator owns receipt hash registration and commit creation. Content written by GPT-6 (gpt-6-astra, effort max) through the Codex worker lane and the OmniRoute pool; committed by the coordinator harness. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

…drop an unsupported #366 citation Coordinator correction after the independent repair verification: the repair's receipt edits were withdrawn (RUNBOOK.md is hash-sealed; the E2E README changes only by amendment), so the README must not say they carry dated updates; the #366 receipt does not record a byte-identical host deployment. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

…dex tool arguments, whole-run query guidance (#408) * Preserve Claude tool joins and estimated cost through Loki Retain tool_use_id and cost_usd alongside the existing workflow.run_id, session.id and ecosystem.task.id resource route. Keep the current structured metadata policy and metric writer identity. Add red-before-green unittest coverage through native Collector 0.161.0 and Loki 3.7.8, including per-arm usage, resource-only runs, private tool joins and indexed-series checks. Sources: Anthropic monitoring documentation, checked 2026-09-27: https://code.claude.com/docs/en/monitoring-usage anthropics/claude-code v2.1.283 release notes and installed client. open-telemetry/opentelemetry-collector-contrib v0.161.0: processor/transformprocessor/README.md and pkg/ottl/ottlfuncs/func_keep_keys.go. grafana/loki v3.7.8 with https://grafana.com/docs/loki/latest/send-data/otel/ and https://grafana.com/docs/loki/latest/reference/loki-http-api/. open-telemetry/opentelemetry-proto v1.9.0, docs/specification.md. Validation: 81 covering unittest checks passed without skips; the complete Collector profile passed native validation after creating documented scratch directories. Local integration uses synthetic inputs, not a new model run. Content written by GPT-6 (gpt-6-astra, effort max) through the Codex worker lane and the OmniRoute pool; committed by the coordinator harness. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * Repair collector evidence and Codex tool metadata privacy Correct the Claude custom-attribute citation and resource/event/datapoint description, whole-run prefix queries, deployment path and dated E2E gates. Publish original and repair command/result excerpts directly in the PR body. Explicitly delete Codex arguments with native OTTL and preserve tool namespace, guarded MCP server, timing, success, correlation and supplied size metadata. Test duplicate Claude run attributes and Codex records with and without server metadata through installed Collector 0.161.0 and Loki 3.7.8. Sources: open-telemetry/opentelemetry-collector-contrib v0.161.0, processor/transformprocessor/README.md and pkg/ottl/ottlfuncs/README.md; openai/codex rust-v0.157.1, codex-rs/otel/src/tool_result.rs:55-79,94, codex-rs/otel/src/events/shared.rs:59-60 and codex-rs/core/src/tools/call_trace.rs:38-54; https://code.claude.com/docs/en/monitoring-usage#multi-team-organization-support; https://grafana.com/docs/loki/latest/query/log_queries/ and send-data/otel/; open-telemetry/opentelemetry-proto v1.9.0 docs/specification.md; base ec27a30, #364 c71d66d and #366 a464d28 historical receipts. Verification: failing-first C1 checks, then 83 covering unittests pass with no skips; complete native Collector validate and git diff --check pass. Evidence is local integration with synthetic fixtures, not live-provider/host acceptance. The coordinator owns receipt hash registration and commit creation. Content written by GPT-6 (gpt-6-astra, effort max) through the Codex worker lane and the OmniRoute pool; committed by the coordinator harness. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * Collector README: state that the sealed #381 receipts are unchanged; drop an unsupported #366 citation Coordinator correction after the independent repair verification: the repair's receipt edits were withdrawn (RUNBOOK.md is hash-sealed; the E2E README changes only by amendment), so the README must not say they carry dated updates; the #366 receipt does not record a byte-identical host deployment. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * Hot files: evidence registration and generated reports Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> --------- Co-authored-by: Scout <scout@local> Co-authored-by: Claude Opus 5.5 (1M context) <[email protected]>

seathatflowsinourveins added the lane:foundation Foundation lane: Claude/Codex setup, hosts, memory, RAG, research, workers label Sep 26, 2026

seathatflowsinourveins mentioned this pull request Sep 26, 2026

Telemetry writer identity (G1): one writer per token series, integrity alerts and a committed host recipe #364

Merged

chatgpt-codex-connector Bot reviewed Sep 26, 2026

View reviewed changes

Comment thread docs/decisions/2026-09-26-tool-invoke-rates.md Resolved

Comment thread observability/backends/templates/ecosystem-dashboard.json.example Resolved

seathatflowsinourveins force-pushed the claude/monitor-metrics-integrity-20260926 branch from dfd5460 to 8172c17 Compare September 26, 2026 21:45

seathatflowsinourveins force-pushed the claude/tool-invoke-rates-20260926 branch from a9c3a4a to a6d00d0 Compare September 26, 2026 21:46

seathatflowsinourveins force-pushed the claude/monitor-metrics-integrity-20260926 branch from 8172c17 to f4f772c Compare September 27, 2026 01:24

Base automatically changed from claude/monitor-metrics-integrity-20260926 to main September 27, 2026 02:45

seathatflowsinourveins force-pushed the claude/tool-invoke-rates-20260926 branch from a6d00d0 to 5ddfd93 Compare September 27, 2026 03:35

seathatflowsinourveins mentioned this pull request Sep 27, 2026

SubagentStart token-lanes carrier for non-blind subagents (native probe, measured RTK rule) #378

Merged

seathatflowsinourveins force-pushed the claude/tool-invoke-rates-20260926 branch from 5ddfd93 to 7a15a2e Compare September 27, 2026 04:19

Scout and others added 5 commits September 27, 2026 00:55

seathatflowsinourveins force-pushed the claude/tool-invoke-rates-20260926 branch from 58451c1 to e047376 Compare September 27, 2026 04:57

seathatflowsinourveins merged commit a464d28 into main Sep 27, 2026
24 checks passed

seathatflowsinourveins deleted the claude/tool-invoke-rates-20260926 branch September 27, 2026 05:27

seathatflowsinourveins mentioned this pull request Sep 27, 2026

Collector: keep Claude tool_use_id and cost_usd through Loki, drop Codex tool arguments, whole-run query guidance #408

Merged

Navigation Menu

Repository navigation

Tool invoke rates: tool, MCP server, skill and subagent names in Loki and Grafana (stacked on #364) - #366

Tool invoke rates: tool, MCP server, skill and subagent names in Loki and Grafana (stacked on #364)#366
seathatflowsinourveins merged 5 commits into
mainfrom
claude/tool-invoke-rates-20260926

seathatflowsinourveins commented Sep 26, 2026 •

edited

Loading

Uh oh!

chatgpt-codex-connector Bot left a comment

Uh oh!

Uh oh!

Uh oh!

seathatflowsinourveins commented Sep 26, 2026

Uh oh!

seathatflowsinourveins commented Sep 27, 2026

Uh oh!

seathatflowsinourveins commented Sep 27, 2026

Uh oh!

seathatflowsinourveins commented Sep 27, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

seathatflowsinourveins commented Sep 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Scope

What was verified

Host-apply plan (not run yet)

Residuals

SOTA sources

Decision record

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

seathatflowsinourveins commented Sep 26, 2026

Uh oh!

seathatflowsinourveins commented Sep 27, 2026

Uh oh!

seathatflowsinourveins commented Sep 27, 2026

Uh oh!

seathatflowsinourveins commented Sep 27, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

seathatflowsinourveins commented Sep 26, 2026 •

edited

Loading