Skip to content

Add fenced Claude usage sampling and Grafana panels - #954

Merged
seathatflowsinourveins merged 6 commits into
mainfrom
codex/ns2604-claude-usage-panel-20261010
Oct 12, 2026
Merged

seathatflowsinourveins merged 6 commits into
mainfrom
codex/ns2604-claude-usage-panel-20261010

Conversation

@seathatflowsinourveins

@seathatflowsinourveins seathatflowsinourveins commented Oct 10, 2026 •

Copy link
Copy Markdown
Collaborator

Scope

Add a Claude usage row after the token panels in /d/cc-lanes, plus a CC-installable fenced sampler. The row shows opaque-account five-hour/seven-day Max fractions, UTC reset times, persistent account status/age and per-key API ledger charges against the $200 edge. Every panel links to OmniRoute's GPT-pool dashboard.

Rejected/exhausted observations render 1.0. Missing, failed or reset-expired observations remain UNKNOWN; exhausted stale accounts remain visible as STALE. Assumed 100% is explicitly marked and cannot overwrite a confirmed allowed window. Signed reconciliation credits, reservations, uncertain charge subsets and provider snapshots have separate ledger semantics.

The sampler uses a one-way identifier hash to assign stable acct-N indexes, records attempts before launching, enforces a 900-second guard and 180-second timeout, refuses local settings, and publishes through the native Prometheus textfile writer. Shipped user units/timer, loopback node_exporter flags, scrape configuration and a hash-enforced runtime installer are for the CC to install after landing. The lane tested only stub clients and fixture identities; it ran no real probe, read no real account identifier and installed no unit.

  • Landing base: 569bbc59db2b5f7c8a932357726efb7acca90b91
  • Lane: lane:foundation
  • Owned paths: observability/claude_usage_{metrics,sampler}.py, observability/lanes_dashboard.py, observability/claude-usage/, tests/docs, generated grafana-lanes.json, shared dependency/inventory hashes and catalog-freshness's full-suite install.
  • Current publication/locator correction: 965a3c739a817bec01bc22d5efd98cc89d1ad090, parent ee16ae8bc9aa76e35d0d8237ebae0d27e2b474d7.

Landing rebase and stable symbol citations

The three reviewed commits were rebased onto main 569bbc59db2b5f7c8a932357726efb7acca90b91 on the CC's cue. The native-lane decision’s token query references at line 390 now identify observability/lanes_dashboard.py, dashboard() (the Codex and Claude request-usage panels), and usage(). The MCP references at line 399 identify the same file’s dashboard() per-session panel and inventory observation expression, plus records(). These symbol references survive source insertions and a squash merge, with no PR-branch SHA in a query link. This correction changes documentation and its native evidence registration only. Native symbol/content checks verify all four query blocks without branch-SHA links. python3 scripts/validate.py passes70 components /11,687 hashes /four profiles /239 receipts; native Gitleaks8.30.1 passes the five-commit PR range with zero leaks.

Conflict resolution preserves main's jsonschema dependency additions and the collector's existing locked client. The daily catalog workflow's full-CI-dependency install includes both through the existing supported command. The native manifest tools remeasure merged hashes, normalize registrations and preserve every current-main entry; obsolete registrations for main's retired Claude-review workflows are removed. No grant-count conflict occurred, so the count-conflict recomputation helper's assertion-driven route was inapplicable.

Current verification:

  • python3 scripts/validate.py: exit 0; 70 components / 11,687 hashed files / four profiles / 239 receipts.
  • Usage metrics, sampler and dashboard modules: 57 tests passed, zero skips, exit 0, with native Prometheus 3.15.0 promtool enabled.
  • Full dependency-inventory and workflow-coverage modules: 71 tests passed, exit 0.
  • python3 observability/ns2604_dashboards.py --check, native manifest check and diff check: exit 0.
  • Native Gitleaks 8.30.1: four-commit PR range, zero leaks, exit 0. Normal commit and pre-push hooks retained.

Earlier fixture-test attempts exposed a missing local prometheus-client prerequisite; the current hash-locked CI requirements installed it before the passing run. An intermediate conflict-resolution registry retained two paths main had retired; integrity validation rejected those entries, which were removed before the passing check. These intermediate failures are recorded in the packet.

Retained packet: pr954-citations-20261011. The range-diff retains the three reviewed commits with dependency/workflow/registry integration changes and adds one citation/registry correction. One delta micro per family and the CC carry check follow the docs-only symbol correction.

Publication marker and locator regression correction

The unrecoverable private guard reference now publishes exactly private, not published (host state). No private preimage was recovered and no unpublished guard digest is repeated here. The independently justified harness source reference is retained under the CC's #161 ruling. Native evidence registrations are refreshed.

Committed tests/test_claude_usage_citations.py verifies the path-plus-symbol query references against the renderer source, includes a missing-symbol negative control, and requires the private guard marker without a digest. Its locator method fails at db029ae and passes at this correction. The decision now cites installed Git 2.53.0's upstream squash semantics. Hostname hygiene and packet-mode preservation stay in Follow-ups.

Current correction head: 965a3c739a817bec01bc22d5efd98cc89d1ad090. Owning citation module: three tests passed, zero skips. Native integrity validation passes 70 components /11,688 hashes /four profiles /239 receipts; Gitleaks8.30.1 passes the six-commit PR range with zero leaks. The earlier feature and four-module counts remain historical at their explicitly named heads.

SOTA sources

  • prometheus/node_exporter v1.12.1, 6044da783597cc3b57aef7580ddcdcff58a4ee99: final machine-local *.prom collection, no sample timestamps.
  • prometheus/client_python v0.26.0, 9b6b971f903e7a08f88da5a95048b7a2a28f55a5: isolated registry, native serialization and atomic writer; official wheel SHA-256 fa93d06737aa02bacd05794768508bb97d2fbee28cb3bca04eaae92f0ca953d6 shared by runtime and CI.
  • anthropics/claude-agent-sdk-python v0.2.165, f7529348747235da34af3ed202251228ff31db08: native rate-limit event fields. Claude Code v2.1.296 and headless docs: print/stream-json fences. Source/version/help only; no real probe or SDK execution.
  • grafana/grafana v13.2.3, 6193dc03311b631b9727b560d24369e683dc396e: native fraction bars, value mappings, datetime units and UTC dashboard timezone. JSON comes from the existing builder.
  • prometheus/prometheus v3.15.0, 5241a27fe3c6983549fccc32f6e65917408c63cd: native query tests against generated expressions; official release checksum verified before extracting promtool.
  • systemd/systemd v260.1, c0a5a2516d28601fb3afc1a77d7b42fcfe38fced: UTC calendar/no catch-up, private mask and bounded oneshot. Native unit verification uses fixture executables and starts no unit.
  • python/cpython v3.13.16, cbc944f4bc59639a444dd971c737788ba2283a91: standard one-way hash and subprocess/locking APIs. pypa/pip 26.1.2, 31d7d168953668aad85154d6121879d07fbeac27: hash-enforced binary-wheel installs.
  • Local source references: CC w5_guard.py and api-actions harness/common.py; source hashes, alternatives and boundaries are in the decision record.

Evidence-class table

Claim Evidence class Reproduction
Maintained event, writer, panel, timer and runtime contracts source_review Pinned primary source reads; native client version/help only
Parser/rejection/reset/recovery, opaque metrics/state, ledger attribution, CLI and fenced sampler synthetic 45 feature contracts (32 metrics + 13 sampler, including native query execution); stub client only, invented private identity files
Actual queries retain failed/stale/never-observed rows, mark assumptions, expire resets and recover to 0.1/MEASURED synthetic Native promtool executes the builder expressions
Generated dashboard and existing contracts local_integration 17 dashboard contracts and renderer --check
UTC timer/user-unit and scrape syntax local_integration Native calendar, unit verification with fixture executables and promtool config validation; no installation/start
Runtime install enforces shared hashes synthetic Installer uses fake Python/install executables in tests
Existing authorized ledger projection matches producer accounting local_integration 621 amount/schema-only rows and three columns; exact match without importing producer runtime
Publication integrity local_integration FULL validate: 70 components, 11,687 hashed files, 4 profiles, 239 receipts

Historical round 2 at 40c7209c, kept for the record: the initial implementation had 21 fail-before tests; the review fix’s initial 37-test run failed on 12 contracts, workflow/feature checks passed 233 tests with one optional skip, and the OSV inventory/policy run passed 61 tests. The current four-module run at db029ae6 passes 62 tests, including native query execution. No live-account or Grafana UI claim.

Local commands run — refreshed at db029ae

The verified test interpreter and native Prometheus 3.15.0 promtool were supplied for the query checks. All results below were run at the unchanged head db029ae6a88b0cd67bc113a750adeb9b46dd27ab.

/tmp/n970-ci313/bin/python -m unittest tests.test_claude_usage_metrics tests.test_claude_usage_sampler tests.test_ns2604_dashboards tests.test_j775_panel_contracts -q
# 62 tests, native promtool enabled, zero skips, exit 0
/tmp/n970-ci313/bin/python -m unittest tests.test_osv_lockfile_coverage tests.test_workflow_security_coverage
# 71 tests, zero skips, exit 0
python3 observability/ns2604_dashboards.py --check
# exit 0
python3 scripts/validate.py
# 70 components, 11,687 hashed files, 4 profiles, 239 receipts, exit 0
gitleaks git . --config .gitleaks.toml --gitleaks-ignore-path .gitleaksignore --log-opts='569bbc59db2b5f7c8a932357726efb7acca90b91..db029ae6a88b0cd67bc113a750adeb9b46dd27ab' --redact --no-banner --report-format json --report-path /tmp/ns954-gitleaks.json
# PR range only, 4 commits, no leaks, exit 0

Decision record

Decision and review corrections and CC installation/operation handoff.

The CC supplies the non-secret identity files and authenticated runtime paths, confirms the legacy ledger alias if needed, installs the supplied runtime/units/scrape configuration, and qualifies live data after landing. Without that alias, legacy spend remains visibly unattributed. CC micro-review is requested at the fix head; draft state and landing remain CC/script-owned.

Host evidence

No new/changed evidence/hosts/ receipt or platform-status claim. The changed dashboard is generated install-plan configuration.

Checklist

  • Existing workflow pins/permissions retained; scheduled full suite installs the hash-locked client before tests.
  • No credentials, real account identifiers or user configuration read; no real probe or SDK run.
  • Unit/configuration files only; no service, timer, account or runtime installed on the shared host.
  • No new paid hosting/subscription/billing surface.
  • Peer-owned files and worktrees preserved.
  • Three reviewed commits rebased on the landing cue, one citation/registry correction, exact lease and normal pre-push hooks retained.
  • CC/script owns ready state and landing.

Follow-ups

  • Optional after landing: add immutable query links to the Git-read squash commit after checking main ancestry. The current P1 is fixed by stable file plus symbol references, so no post-landing re-pin is required for correctness.

  • Locator regression resolved in this correction: the new committed source contract rejects the old branch-SHA references at db029ae, validates the renderer symbols and rejects a missing usage() reference.

  • Review the pre-existing machine-name labels confirmed by Astra fix4: 26 occurrences on 24 lines, all in unchanged content of changed files, with zero in the full patch. Any normalization belongs in a separate evidence-preserving cleanup with native re-registration; the host value is omitted (P3, counted only).

  • Guard recovery clears against observation timestamp ordering so an older replay cannot erase a newer rejection. (Codex 4243783423: follow-up, not a blocker under the terminal rule; CC ruling 2026-10-12T02:47:45Z).

  • Resolve identity and runtime paths before containment comparisons; research CPython Path.resolve() before is_relative_to() through the co-op’s research queue. (Codex 4243783426: follow-up, not a blocker under the terminal rule; CC ruling 2026-10-12T02:47:45Z).

  • Reject duplicate authenticated config_dir values, including aliases that resolve to the same directory, before launching per-account probes. (Codex 4243783429: follow-up, not a blocker under the terminal rule; CC ruling 2026-10-12T02:47:45Z).

  • Bound probe stdout during capture or parsing so the service never accumulates an unbounded stream before the existing size check. (Codex 4243783432: follow-up, not a blocker under the terminal rule; CC ruling 2026-10-12T02:47:45Z).

  • Preserve executable modes from the native merged Git tree when assembling review packets; Astra fix4 confirms the overlay still supplies 0644 for files recorded as 100755, including the pre-push hook and landing scripts. Restore native tree modes during packet construction; this is a packet construction P3, with no PR source defect.

@seathatflowsinourveins seathatflowsinourveins added the lane:foundation Foundation lane: Claude/Codex setup, hosts, memory, RAG, research, workers label Oct 10, 2026
@socket-security

socket-security Bot commented Oct 10, 2026 •

Copy link
Copy Markdown

Dependency limit exceeded — report not shown.

This pull request scan exceeded the 10,000-dependency limit applied to this scan, so the results are incomplete and may be inaccurate. To avoid reporting false positives, Socket has not posted a report.

Upgrade your plan to raise the dependency limit and get complete reports, or view the partial scan in the dashboard.

Socket is always free for open source. If this is a non-commercial open source project, contact us to request a free Team account.

@seathatflowsinourveins seathatflowsinourveins changed the title Add Claude usage limits and API spend to Lanes Add fenced Claude usage sampling and Grafana panels Oct 10, 2026
@seathatflowsinourveins
seathatflowsinourveins force-pushed the codex/ns2604-claude-usage-panel-20261010 branch from 513788a to db029ae Compare October 11, 2026 21:46
@seathatflowsinourveins
seathatflowsinourveins marked this pull request as ready for review October 11, 2026 23:48
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 11, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-12T07:12:33.898330Z 965a3c7 Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: db029ae6a8

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread observability/claude_usage_metrics.py Resolved
Comment thread observability/claude_usage_sampler.py Resolved
Comment thread observability/claude_usage_sampler.py Resolved
Comment thread observability/claude_usage_sampler.py Resolved
@seathatflowsinourveins

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Something went wrong. Try again later by commenting “@codex review”.

An unknown error occurred
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@seathatflowsinourveins

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. 🚀

Reviewed commit: 965a3c739a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@seathatflowsinourveins
seathatflowsinourveins merged commit 83cec12 into main Oct 12, 2026
37 checks passed
@seathatflowsinourveins
seathatflowsinourveins deleted the codex/ns2604-claude-usage-panel-20261010 branch October 12, 2026 07:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

lane:foundation Foundation lane: Claude/Codex setup, hosts, memory, RAG, research, workers

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant