Skip to content

Classify tool authority categories in local session reports #201

Description

@luoyuctl

Background

Issue #198 defines tool authority/safety categories so reports can show whether an agent session crossed higher-risk tool boundaries.

Evidence

User value

Users reviewing an agent session should be able to see the highest-risk boundary reached and whether a new run used broader or riskier tools than expected.

Adoption rationale

Tool authority summaries improve trust in local reports and make CI review more actionable without turning agenttrace into a hosted security product.

Suggested scope

  • Add a shared best-effort tool authority classifier for local reports.
  • Classify tool calls into the initial categories from Design incident timeline and tool-authority signals for agent debugging #198.
  • Surface counts by category and the highest category reached in report/TUI data where appropriate.
  • Treat unknown or ambiguous tool calls conservatively as unknown_authority rather than under-classifying.
  • Leave source-specific parser mapping as follow-up if a source lacks enough tool metadata.

Non-goals

  • Do not claim security certification or policy enforcement.
  • Do not block commands by default.
  • Do not upload logs or tool data to a hosted service.
  • Do not require perfect classification for every source in the first pass.

Acceptance criteria

  • Reports expose tool authority category counts and highest category reached when tool metadata is available.
  • Unknown or ambiguous tool calls are represented explicitly.
  • Classification is deterministic for fixtures.
  • CI/report checks can later compare whether a run introduced a new high-authority category.

Suggested lane

lane/quality

Risk

Medium. Classification can be misleading if it is too confident. Prefer conservative categories and explicit unknowns.

Source

Follow-up split from #198 and Discussion #2 feedback: #2 (comment)

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions