Skip to content

Rename azdo-helix-failures to ci-analysis and improve SKILL.md#124151

Merged
lewing merged 11 commits intomainfrom
lewing/helix-skill-improvements
Feb 9, 2026
Merged

Rename azdo-helix-failures to ci-analysis and improve SKILL.md#124151
lewing merged 11 commits intomainfrom
lewing/helix-skill-improvements

Conversation

@lewing
Copy link
Member

@lewing lewing commented Feb 8, 2026

Renames and improves the CI analysis skill (formerly azdo-helix-failures).

Rename:

  • Directory: azdo-helix-failuresci-analysis
  • Script: Get-HelixFailures.ps1Get-CIStatus.ps1
  • Frontmatter name: azdo-helix-failuresci-analysis
  • The old name was too narrow — the skill does full CI analysis, not just failure reporting

SKILL.md improvements (skill-builder patterns):

  • Critical rules — No PR approval/blocking (--approve/--request-changes), no branch switching (git checkout/gh pr checkout)
  • Expanded trigger keywords — Added "why is CI red", "test failures", "retry CI", "rerun tests"
  • Script Limitations section — Clarifies the script targets AzDO/Helix, but analysis patterns apply broadly
  • Three Modes decision table — Distinguishes PR analysis vs Build ID vs Helix Job modes with -WorkItem optional
  • Split "What the Script Does" — Per-mode sections with accurate step coverage (Build ID mode doesn't do Build Analysis or PR correlation)
  • Inline anti-patterns — Warnings at key decision points:
    • Don't guess the mode — match input to mode
    • Canceled ≠ Failed — check Helix results before dismissing
    • Be cautious labeling failures as "infrastructure" — if Build Analysis didn't flag it, investigate before assuming transient
    • Missing packages on flow PRs — may be a code change, not feed infrastructure (e.g., CoreCLR vs Mono runtime pack)
  • Heuristic trust warning — Don't blindly trust the script's recommendation over evidence
  • gh pr checks tip — Valid --json fields documented (no conclusion — use state)

Script fixes:

  • PowerShell 5.1 compatibility — Replaced em-dash characters with ASCII dashes, added UTF-8 BOM
  • Cache directory — Renamed from helix-failures-cache to ci-analysis-cache

cc @dotnet/runtime-infrastructure

Copilot AI review requested due to automatic review settings February 8, 2026 20:46
@lewing lewing added the area-skills Agent Skills label Feb 8, 2026
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Documentation-only update improving the .github/skills/azdo-helix-failures skill guide to better steer users toward the right analysis mode (PR vs Build ID vs Helix job) and to add decision-point warnings/anti-patterns for interpreting AzDO/Helix results.

Changes:

  • Expanded trigger phrases and added a “When NOT to Use” section to reduce misuse (e.g., GitHub Actions vs AzDO/Helix).
  • Added a “Three Modes” decision table and reorganized “What the Script Does” by mode.
  • Added inline warnings about canceled jobs, infra-vs-code classification, flow PR missing-package scenarios, and heuristic retry guidance.

@lewing lewing changed the title Improve azdo-helix-failures SKILL.md structure Rename azdo-helix-failures to ci-analysis and improve SKILL.md Feb 8, 2026
Copilot AI review requested due to automatic review settings February 8, 2026 20:50
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 5 changed files in this pull request and generated no new comments.

Copilot AI review requested due to automatic review settings February 8, 2026 21:04
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 5 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (1)

.github/skills/ci-analysis/scripts/Get-CIStatus.ps1:1

  • The file now starts with a UTF-8 BOM character (shown as an invisible char before <#). Other PowerShell scripts in this repo (e.g., .github/skills/vmr-codeflow-status/scripts/Get-CodeflowStatus.ps1) start directly with <# without a BOM. Please remove the BOM to avoid odd diffs/tooling issues when running or parsing the script cross-platform.

Copilot AI review requested due to automatic review settings February 8, 2026 21:18
@lewing lewing requested a review from steveisok February 8, 2026 21:20
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 6 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (2)

.github/skills/ci-analysis/scripts/Get-CIStatus.ps1:8

  • The script has been renamed/broadened to CI status analysis, but the .SYNOPSIS still says it only “Retrieves test failures…”. Please update the synopsis/description so it matches the current scope (build status + failures + Helix job/work item queries).
    .github/skills/ci-analysis/scripts/Get-CIStatus.ps1:165
  • -ClearCache only removes the new ci-analysis-cache directory. Since this is a rename from helix-failures-cache, users may end up with orphaned old cache data that never gets cleaned up. Consider clearing both directories (old + new) for backwards compatibility, or at least printing where the previous cache lived so users can remove it.

Copilot AI review requested due to automatic review settings February 8, 2026 21:55
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 6 changed files in this pull request and generated no new comments.

Copy link
Member

@steveisok steveisok left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice!

@lewing lewing merged commit 4f65fec into main Feb 9, 2026
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area-skills Agent Skills

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants