chainsaw version Version

v0.2.13

Description

When exporting Chainsaw results as JSON, the report includes timings and names for tests, steps, and operations, but it does not include outcome/status fields (pass/fail/error/skipped) or a message/reason. This makes it hard to use the JSON as a single source of truth for analytics, dashboards, or traces, forcing consumers to cross-reference JUnit just to know outcomes. This problem does not exist when exporting as XML, JUNIT, etc.

Current behavior

• JSON contains: name, startTime, endTime, nested tests[] → steps[] → operations[]
• JSON omits status/outcome fields and failure/error messages at all levels
• Consumers cannot programmatically determine whether a test/step/operation passed or failed from JSON alone

Why this matters

• Dashboards & time series: We want to store JSON in BigQuery and visualize in Looker/Grafana. Without status, trend KPIs (pass rate, flakiness) require joining JUnit
• Tracing: The JSON structure maps perfectly to spans (test -> step -> operation). Without status, OpenTelemetry traces (Tempo/Grafana) can't show red/errored spans
• Root-cause analysis: Step/operation-level outcomes and messages enable "slow or failing section" insights and critical-path analysis directly from JSON

Steps to reproduce

1. Run Chainsaw with flag: --report-format JSON

Expected behavior

JSON report contains status, e.g. passed | failed | error | skipped | cancelled | unknown

A message fields would be helpful as well, a human-readable message describing the failure/error.

Screenshots

No response

Logs

Slack discussion

No response

Troubleshooting

• I have searched other issues in this repository and mine is not recorded.