Skip to content

feat: add diagnose_ci tool for CI failure diagnosis#192

Merged
ichoosetoaccept merged 1 commit intomainfrom
02-19-feat-add-diagnose-ci-tool
Feb 19, 2026
Merged

feat: add diagnose_ci tool for CI failure diagnosis#192
ichoosetoaccept merged 1 commit intomainfrom
02-19-feat-add-diagnose-ci-tool

Conversation

@ichoosetoaccept
Copy link
Member

@ichoosetoaccept ichoosetoaccept commented Feb 19, 2026

Description

Add a new diagnose_ci MCP tool that lets agents quickly diagnose GitHub Actions CI failures in a single call, replacing the typical 3-5 sequential gh commands.

Problem

Agents currently need to run multiple gh commands to diagnose CI failures:

  1. gh run list → find failed runs
  2. gh run view <id> --json jobs → find failed jobs
  3. gh run view <id> --log-failed → get error logs
  4. Manually parse noisy log output to find actual errors

Solution

The diagnose_ci tool collapses this into one call that returns structured results:

  • Finds the latest failed run for a PR/branch (or accepts a specific run_id)
  • Identifies failed jobs and the specific failed step within each
  • Extracts actionable error lines from logs, filtering out CI noise (timestamps, group markers, cleanup steps)

New files

  • src/codereviewbuddy/tools/ci.py — business logic (diagnose_ci, log parsing, error extraction)
  • tests/test_ci.py — 23 tests covering helpers and integration scenarios

Changes to existing files

  • src/codereviewbuddy/models.py — add CIJobFailure and CIDiagnosisResult models
  • src/codereviewbuddy/server.py — register diagnose_ci tool with prerequisite validation
  • tests/test_mcp_integration.py — add diagnose_ci to expected tools set, bump count to 13

Bug fix included

Fixed prerequisite validation: when run_id is provided without repo and no workspace is detected, the tool now correctly requires repo to be provided (previously skipped the check entirely).

Checklist

Copy link
Member Author

ichoosetoaccept commented Feb 19, 2026

@greptile-apps
Copy link

greptile-apps bot commented Feb 19, 2026

Greptile Summary

Adds a new diagnose_ci MCP tool that consolidates CI failure diagnosis into a single call, replacing the typical 3-5 sequential gh commands agents currently need to run. The tool finds failed workflow runs, identifies failed jobs and steps, and extracts actionable error lines from logs while filtering out CI noise like timestamps and cleanup messages.

Key changes:

  • New business logic in src/codereviewbuddy/tools/ci.py with helper functions for log parsing, error extraction, and run discovery
  • CIJobFailure and CIDiagnosisResult models added to models.py for structured output
  • Tool registration in server.py with prerequisite validation that correctly requires repo when run_id is provided without workspace detection
  • Comprehensive test coverage with 23 tests covering unit helpers and integration scenarios

Issue found:

  • Line 257 in ci.py has error matching logic that may duplicate all error lines across multiple failed jobs when no job-specific matches are found (the or error_lines fallback). This could make the diagnosis output confusing when multiple jobs fail with generic error messages that don't contain job names.

Confidence Score: 4/5

  • Safe to merge with minor logic improvement recommended
  • The implementation is well-structured with comprehensive test coverage and follows existing patterns. The bug fix for prerequisite validation is correct. The only issue is the error line duplication logic which could cause confusion but won't break functionality.
  • Pay close attention to src/codereviewbuddy/tools/ci.py:257 - the error matching fallback logic

Important Files Changed

Filename Overview
src/codereviewbuddy/tools/ci.py New CI diagnosis tool with comprehensive log parsing, error extraction, and helper functions - well-structured implementation
src/codereviewbuddy/server.py Registered diagnose_ci tool with prerequisite validation logic; validation logic has subtle edge case

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[diagnose_ci tool call] --> B{run_id provided?}
    B -->|Yes| C[Use specified run_id]
    B -->|No| D[Find latest failed run]
    
    D --> E{pr_number provided?}
    E -->|Yes| F[Get branch from PR]
    E -->|No| G[Use current branch]
    
    F --> H[gh run list --branch]
    G --> H
    H --> I[Filter for failed runs]
    
    C --> J[Get run details]
    I --> J
    
    J --> K[Extract failed jobs]
    K --> L{Failed jobs found?}
    
    L -->|No| M[Return error: no jobs failed]
    L -->|Yes| N[Get failed logs]
    
    N --> O[Clean log lines]
    O --> P[Extract error lines with context]
    P --> Q{Multiple failed jobs?}
    
    Q -->|Yes| R[Match errors to job names]
    Q -->|No| S[Assign all errors to job]
    
    R --> T[Build CIJobFailure objects]
    S --> T
    T --> U[Return CIDiagnosisResult]
Loading

Last reviewed commit: 47bd4c3

@ichoosetoaccept ichoosetoaccept changed the title fix: validate repo prerequisite when run_id is provided without repo feat: add diagnose_ci tool for CI failure diagnosis Feb 19, 2026
Copy link
Member Author

ichoosetoaccept commented Feb 19, 2026

Merge activity

  • Feb 19, 12:46 PM UTC: A user started a stack merge that includes this pull request via Graphite.
  • Feb 19, 12:47 PM UTC: Graphite rebased this pull request as part of a merge.
  • Feb 19, 12:48 PM UTC: @ichoosetoaccept merged this pull request with Graphite.

@ichoosetoaccept ichoosetoaccept changed the base branch from 02-19-chore-update-copier-template-to-0.29.3 to graphite-base/192 February 19, 2026 12:46
@ichoosetoaccept ichoosetoaccept changed the base branch from graphite-base/192 to main February 19, 2026 12:46
@ichoosetoaccept ichoosetoaccept force-pushed the 02-19-feat-add-diagnose-ci-tool branch from 1c6a854 to 47bd4c3 Compare February 19, 2026 12:47
@ichoosetoaccept ichoosetoaccept merged commit d06bd93 into main Feb 19, 2026
7 checks passed
@ichoosetoaccept ichoosetoaccept deleted the 02-19-feat-add-diagnose-ci-tool branch February 19, 2026 12:48
Copy link

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

5 files reviewed, 1 comment

Edit Code Review Agent Settings | Greptile

@blacksmith-sh
Copy link

blacksmith-sh bot commented Feb 19, 2026

Found 1 test failure on Blacksmith runners:

Failure

Test View Logs
Required test coverage of 90.0% not reached. Total coverage: 89.89%/
Required test coverage of 90.0% not reached. Total coverage: 89.89%
View Logs

Fix in Cursor

ichoosetoaccept added a commit that referenced this pull request Feb 24, 2026
## Description

Adds 70 lines of new test coverage for `tools/ci.py` to bring the CI module above the fail-under threshold. Covers error paths, edge cases in log parsing, and the `run_id` vs `pr_number` detection branches.

## Changes

- **`tests/test_ci.py`** — new test cases for CI diagnosis edge cases (empty logs, missing steps, malformed output, run ID lookup)

## Checklist

- [x] Tests added/updated
- [x] `poe check` passes
- [x] Commit messages follow [conventional commits](../CONTRIBUTING.md#commit-message-convention)

## Related Issues

Fixes CI coverage regression from #192 (`diagnose_ci` tool addition).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant