Skip to content

Classify tool authority categories#205

Merged
luoyuctl merged 1 commit into
masterfrom
quality/201-tool-authority-categories
May 17, 2026
Merged

Classify tool authority categories#205
luoyuctl merged 1 commit into
masterfrom
quality/201-tool-authority-categories

Conversation

@luoyuctl

Copy link
Copy Markdown
Owner

Summary

  • add a shared conservative tool authority classifier for reportable tool calls
  • expose authority category counts and highest authority in session and overview JSON reports
  • include authority categories in baseline comparison surfaces and TUI detail evidence
  • extend CI report checks for deterministic authority fields

Closes #201

Test plan

  • go test ./...
  • go build -o /tmp/agenttrace-quality-201 ./cmd/agenttrace
  • git diff --check
  • AGENTTRACE_BIN=/tmp/agenttrace-quality-201 AGENTTRACE_CI_OUT=/tmp/agenttrace-quality-201-ci scripts/ci/check-output-contract.sh
  • AGENTTRACE_BIN=/tmp/agenttrace-quality-201 AGENTTRACE_CI_OUT=/tmp/agenttrace-quality-201-ci scripts/ci/check-deterministic-output.sh
  • AGENTTRACE_BIN=/tmp/agenttrace-quality-201 AGENTTRACE_CI_OUT=/tmp/agenttrace-quality-201-ci scripts/ci/check-report-semantics.sh
  • AGENTTRACE_BIN=/tmp/agenttrace-quality-201 AGENTTRACE_CI_OUT=/tmp/agenttrace-quality-201-ci scripts/ci/check-docs-commands.sh
  • AGENTTRACE_BIN=/tmp/agenttrace-quality-201 AGENTTRACE_CI_OUT=/tmp/agenttrace-quality-201-ci scripts/ci/check-release-surfaces.sh
  • AGENTTRACE_BIN=/tmp/agenttrace-quality-201 AGENTTRACE_CI_OUT=/tmp/agenttrace-quality-201-ci scripts/ci/check-pages-artifact.sh site
  • /tmp/agenttrace-quality-201 --doctor || true
  • /tmp/agenttrace-quality-201 --demo --overview -f json --baseline /tmp/agenttrace-quality-201-ci/agenttrace-demo.json

@luoyuctl luoyuctl added lane/quality Reliability, tests, and diagnostics status/ready-for-review Ready for review labels May 17, 2026
@luoyuctl luoyuctl merged commit f354b39 into master May 17, 2026
1 check passed
@luoyuctl luoyuctl deleted the quality/201-tool-authority-categories branch May 17, 2026 09:20
@luoyuctl

Copy link
Copy Markdown
Owner Author

Quality Gatekeeper Review

Verdict: PASS
Risk: Medium
Lane: quality
Checks:

  • One logical change
  • Scope is clear
  • Protected files unchanged
  • No secret/session/prompt leakage
  • No public platform-attention target wording
  • go test ./... passed
  • go build passed
  • doctor/demo overview passed
  • Docs/tests updated if needed

Notes:

  • GitHub mergeStateStatus observed as UNKNOWN after merge; PR state is MERGED with merge commit f354b39967120e8410c9de8f0826c2c6e750d07c.
  • GitHub CI Test and build: SUCCESS.
  • Changed files: report/engine/TUI/i18n internals, docs/ci-integration.md, and CI report scripts. No protected release/package files changed.
  • Risk is Medium because this introduces conservative tool authority classification and JSON report fields, but remains local-only and deterministic.

Validated commands:

  • go test ./...
  • go build -o /tmp/agenttrace-review-pr-205 ./cmd/agenttrace
  • /tmp/agenttrace-review-pr-205 --doctor || true
  • /tmp/agenttrace-review-pr-205 --demo --overview -f json
  • AGENTTRACE_BIN=/tmp/agenttrace-review-pr-205 AGENTTRACE_CI_OUT=/tmp/agenttrace-review-pr-205-ci scripts/ci/check-output-contract.sh
  • AGENTTRACE_BIN=/tmp/agenttrace-review-pr-205 AGENTTRACE_CI_OUT=/tmp/agenttrace-review-pr-205-ci scripts/ci/check-deterministic-output.sh
  • AGENTTRACE_BIN=/tmp/agenttrace-review-pr-205 AGENTTRACE_CI_OUT=/tmp/agenttrace-review-pr-205-ci scripts/ci/check-report-semantics.sh
  • AGENTTRACE_BIN=/tmp/agenttrace-review-pr-205 AGENTTRACE_CI_OUT=/tmp/agenttrace-review-pr-205-ci scripts/ci/check-docs-commands.sh
  • AGENTTRACE_BIN=/tmp/agenttrace-review-pr-205 AGENTTRACE_CI_OUT=/tmp/agenttrace-review-pr-205-ci scripts/ci/check-release-surfaces.sh
  • AGENTTRACE_BIN=/tmp/agenttrace-review-pr-205 AGENTTRACE_CI_OUT=/tmp/agenttrace-review-pr-205-ci scripts/ci/check-pages-artifact.sh site
  • /tmp/agenttrace-review-pr-205 --demo --overview -f json --baseline /tmp/agenttrace-review-pr-205-ci/agenttrace-demo.json

Decision: PASS. The implementation satisfies #201: reports expose authority category counts and highest category, unknown authority remains explicit/conservative, fixture behavior is deterministic, and CI report checks can compare new authority categories. Final label action: status/auto-merged.

@luoyuctl

Copy link
Copy Markdown
Owner Author

Event: Validation
Actor: Product
Scope: PR #205 post-merge
State change: status/ready-for-review -> status/auto-merged
Evidence: Revalidated master at merge commit f354b39 after #205 landed. Passed go test ./..., go build -o /tmp/agenttrace-product-check ./cmd/agenttrace, output-contract, deterministic-output, report-semantics, docs-commands, release-surfaces, Pages artifact, --doctor, and demo JSON baseline self-compare.
Next owner: Product / Growth

Decision: Post-merge validation PASS. #201 is closed with status/auto-merged. The next release-train review in #179 should include #205 as the authority-classification companion to #203 baseline comparison.

@luoyuctl

Copy link
Copy Markdown
Owner Author

Event: Validation correction
Actor: Product
Scope: PR #205 post-merge
State change: no label change; keep status/auto-merged
Evidence: The previous Product validation comment used an outdated local assertion name while checking the JSON manually. The actual merged JSON contract exposes authority data as summary.tool_authority and surfaces.authority_categories, not authority_summary.
Next owner: Product / Growth

Corrected check: reran demo JSON baseline self-compare on master at f354b39 and verified summary.tool_authority, surfaces.authority_categories, baseline_comparison, and incident_timelines are present. The corrected output showed authority categories [read_only_files, test_or_build], highest authority test_or_build, baseline_comparison.new_high_authority_tool_use=[], and 3 incident timelines.

Decision: Post-merge validation remains PASS with the corrected field names. The earlier comment's command list is valid for build/tests/CI scripts, but this comment supersedes the field-name detail.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

lane/quality Reliability, tests, and diagnostics status/auto-merged

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Classify tool authority categories in local session reports

1 participant