Skip to content

Add new workflow for updating VR snapshots#6541

Merged
PavelVanecek merged 4 commits into
mainfrom
vr-ci
Oct 31, 2025
Merged

Add new workflow for updating VR snapshots#6541
PavelVanecek merged 4 commits into
mainfrom
vr-ci

Conversation

@PavelVanecek

@PavelVanecek PavelVanecek commented Oct 31, 2025

Copy link
Copy Markdown
Collaborator

I was going to test it on the failed VR test that made it to main branch but you fixed it faster Coltin. I don't want to run the VR update on pre-push because it takes 10+ minutes sometimes.

Summary by CodeRabbit

  • New Features

    • Added automated visual regression snapshot updates via /update-snapshots comment on pull requests
  • Documentation

    • Enhanced pull request comments with instructions for updating baseline screenshots

@coderabbitai

coderabbitai Bot commented Oct 31, 2025

Copy link
Copy Markdown
Contributor

Walkthrough

Two workflow files modified: .github/workflows/ci.yml extended with a contextual note in PR comments instructing users to comment /update-snapshots for automatic snapshot updates. A new .github/workflows/update-vr-snapshots.yml workflow added to automate visual regression snapshot updates via manual trigger or PR comments with conditional logic handling multiple scenarios.

Changes

Cohort / File(s) Summary
PR Comment Enhancement
.github/workflows/ci.yml
Added informational text to PR comment bodies following artifact download links, with a separator and instruction to use /update-snapshots command for automatic baseline screenshot updates. Applied to both integrated test artifact and VR report comment sections.
Visual Regression Snapshot Automation
.github/workflows/update-vr-snapshots.yml
New workflow triggered by manual dispatch or PR comment /update-snapshots command. Fetches PR metadata, checks out code, builds Docker image, updates VR snapshots, detects changes, and creates commits/branches/PRs based on context (main branch, open PR, closed PR, or direct comment trigger).

Sequence Diagram(s)

sequenceDiagram
    autonumber
    actor Trigger as Manual Dispatch or<br/>PR Comment with /update-snapshots
    participant Workflow as Update VR<br/>Snapshots Workflow
    participant Git as Git & GitHub API
    participant Docker as Docker Build<br/>& Test
    
    Trigger->>Workflow: Trigger event
    
    alt Issue Comment Trigger
        Workflow->>Git: Fetch PR metadata<br/>(head ref, SHA, state, base)
    else Manual Dispatch
        Note over Workflow: Use default context
    end
    
    Workflow->>Git: Checkout repository<br/>(full history, PR branch if applicable)
    Workflow->>Docker: Setup Node.js, install deps
    Workflow->>Docker: Build Docker image
    Workflow->>Docker: Run VR snapshot updates
    Workflow->>Git: Detect changes via git status
    
    alt Changes Detected
        Workflow->>Git: Configure Git user
        
        alt On main branch
            Workflow->>Git: Create new branch
            Workflow->>Git: Commit & push changes
            Workflow->>Git: Create PR against main
        else On open PR
            Workflow->>Git: Commit & push on PR branch
            alt Triggered by comment
                Workflow->>Git: Post success comment on PR
            end
        else On closed PR
            Workflow->>Git: Create new branch from base
            Workflow->>Git: Commit & push changes
            Workflow->>Git: Create PR against original base
            Workflow->>Git: Comment new PR URL on original PR
        end
    else No Changes
        alt Triggered by comment
            Workflow->>Git: Post informational comment on PR
        else Manual trigger
            Workflow->>Workflow: Log no-op message
        end
    end
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

  • .github/workflows/update-vr-snapshots.yml — New workflow with intricate conditional branching across multiple scenarios (manual vs. comment trigger, main branch vs. open PR vs. closed PR). Requires careful verification of:
    • Conditional trigger logic and env variable handling
    • Git operations and branch management correctness
    • PR creation/comment logic for different states
    • Error handling and edge cases
  • .github/workflows/ci.yml — Minimal impact; text content additions only, requires brief verification for grammar/clarity

Pre-merge checks and finishing touches

❌ Failed checks (2 warnings)
Check name Status Explanation Resolution
Description Check ⚠️ Warning The PR description provided is largely incomplete and does not follow the required template structure. The author's description is informal and conversational, mentioning context about testing and pre-push timing, but it lacks all of the required sections including "Related Issue," "Motivation and Context" (in proper section format), "How Has This Been Tested?," "Types of changes" checkboxes, and the final checklist. While the description does hint at motivation (avoiding long pre-push runtime), it fails to provide a formal, structured description of the changes being made, the related issues, or testing approach as specified by the template. The PR description should be updated to follow the provided template structure. Please add proper section headings and fill in all required sections: a detailed description of the workflow changes, a link to any related issue (or mention if there isn't one), the motivation and context in a formal section, details on how the changes were tested, appropriate checkboxes under "Types of changes" (this appears to be a new feature), and completion of the checklist items at the end.
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. You can run @coderabbitai generate docstrings to improve docstring coverage.
✅ Passed checks (1 passed)
Check name Status Explanation
Title Check ✅ Passed The PR title "Add new workflow for updating VR snapshots" clearly and specifically describes the primary change in the pull request—the addition of a new GitHub Actions workflow file (.github/workflows/update-vr-snapshots.yml). While the changeset also includes modifications to the existing ci.yml workflow, the new workflow file represents the main contribution with significantly higher code review effort (High vs. Low), making the title appropriately focused on this core addition. The title is concise, avoids vague terminology, and conveys sufficient information for a developer scanning the history to understand what was added.
✨ Finishing touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch vr-ci

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (2)
.github/workflows/update-vr-snapshots.yml (2)

118-124: Specify branch explicitly in git push for robustness.

Line 123 uses git push without specifying the branch. While this may work if git tracking is configured, it's safer and more explicit to specify the branch or use git push origin HEAD:refs/heads/$(git symbolic-ref --short HEAD) or simply git push -u origin HEAD. This ensures the push succeeds even if tracking is not configured or the branch was just created in the same job.

Apply this diff:

      - name: Commit and Push on PR Branch
        if: steps.check_changes.outputs.has_changes == 'true' && github.event_name == 'issue_comment' && steps.get_pr_branch.outputs.is_open == 'true'
        run: |
          git add .
          git commit -m "chore: update visual regression snapshots"
-         git push
+         git push origin HEAD

125-142: Use gh pr comment for consistency with ci.yml.

Line 142 uses gh issue comment to comment on the original PR. While this works (PRs are issues in GitHub's model), ci.yml line 284 uses gh pr comment for the same purpose. For consistency, semantic clarity, and reduced confusion, use gh pr comment:

          # Comment on the original closed PR with link to new PR
-         gh issue comment ${{ github.event.issue.number }} --body "✅ Visual regression snapshots have been updated in a new PR: $PR_URL"
+         gh pr comment ${{ github.event.issue.number }} --body "✅ Visual regression snapshots have been updated in a new PR: $PR_URL"
📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between dbf3e61 and a4a453c.

📒 Files selected for processing (2)
  • .github/workflows/ci.yml (1 hunks)
  • .github/workflows/update-vr-snapshots.yml (1 hunks)
🧰 Additional context used
🧠 Learnings (2)
📓 Common learnings
Learnt from: CR
Repo: recharts/recharts PR: 0
File: DEVELOPING.md:0-0
Timestamp: 2025-10-25T07:34:46.549Z
Learning: Applies to test-vr/__snapshots__/** : Commit newly recorded or updated visual regression snapshot files under test-vr/__snapshots__ to the repository
📚 Learning: 2025-10-25T07:34:46.549Z
Learnt from: CR
Repo: recharts/recharts PR: 0
File: DEVELOPING.md:0-0
Timestamp: 2025-10-25T07:34:46.549Z
Learning: Applies to test-vr/__snapshots__/** : Commit newly recorded or updated visual regression snapshot files under test-vr/__snapshots__ to the repository

Applied to files:

  • .github/workflows/update-vr-snapshots.yml
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (2)
  • GitHub Check: Build, Test, Pack
  • GitHub Check: Analyze (javascript-typescript)
🔇 Additional comments (10)
.github/workflows/ci.yml (1)

278-281: Instruction text integrates well with VR failure reporting.

The added separator and /update-snapshots instruction text clearly guides users to the new workflow. The formatting and placement in the comment are appropriate.

.github/workflows/update-vr-snapshots.yml (9)

60-68: Verify ref checkout evaluation logic with conditional inputs.

The ref parameter uses a ternary operator that depends on step outputs only available when github.event_name == 'issue_comment':

ref: ${{ github.event_name == 'issue_comment' && steps.get_pr_branch.outputs.ref || github.ref }}

When triggered via workflow_dispatch, steps.get_pr_branch does not execute, so steps.get_pr_branch.outputs.ref may not exist. While the short-circuit AND operator (&&) prevents evaluation if the first condition is false, this pattern could be fragile. If get_pr_branch.outputs.ref evaluates to an empty string on issue_comment triggers (due to timing or other issues), it would silently fall back to github.ref.

Consider using explicit conditionals for clarity:

- name: Checkout Repository
  uses: actions/checkout@v4
  with:
    fetch-depth: 0
    token: ${{ secrets.GITHUB_TOKEN }}
    ref: ${{ steps.get_pr_branch.outputs.ref || github.ref }}

This relies on the GitHub Actions context to only populate the outputs when the step runs, which is more explicit. Alternatively, verify that the current pattern handles the workflow_dispatch trigger correctly by testing with that trigger to ensure github.ref is used as fallback.


85-101: Change detection and git configuration are well-guarded.

The workflow correctly detects changes and only configures git when needed. All commit/push steps properly check the has_changes flag, ensuring git config runs before any git operations.


25-29: Trigger condition guards correctly against non-PR comments.

The workflow condition properly checks that issue_comment events are only processed when they occur on a PR (github.event.issue.pull_request) and contain the /update-snapshots phrase. This prevents unnecessary workflow runs.


102-142: Three execution paths are well-designed and properly separated.

The workflow correctly handles three scenarios: automatic snapshot updates on main (creates new PR), updates on open PRs (commits to existing branch), and updates on closed PRs (creates new PR and comments original). Each path has appropriate logic and messaging. The timestamped branch names prevent collisions if multiple updates occur.


1-19: Workflow structure and environment setup are sound.

The workflow uses appropriate Node.js version pinning (20.x), proper permissions for contents and PRs, and standard GitHub Actions. The trigger setup correctly distinguishes between manual dispatch and comment-based automation.


31-41: Emoji reaction provides good UX feedback.

The workflow reacts to the triggering comment with a rocket emoji, giving users immediate feedback that their request was recognized and is processing.


43-58: PR data fetching via github-script is correct and comprehensive.

The workflow fetches all necessary PR details (head ref, head SHA, open state, base branch) needed for the three execution scenarios. Step only runs for issue_comment triggers, avoiding unnecessary API calls.


82-83: Snapshot update and Docker preparation steps are appropriately minimal.

The workflow delegates to npm run scripts (test-vr:prepare and test-vr:update), maintaining separation of concerns and keeping the workflow maintainable. These scripts are standard in the recharts repo based on ci.yml.


144-166: Comment strategy provides clear feedback to users.

The workflow offers three distinct messages: success with snapshots updated (145-154), no changes needed (156-166), and creation of new PR for closed PRs (142). This keeps users informed at each step.

@PavelVanecek

Copy link
Copy Markdown
Collaborator Author

/update-snapshots

@ckifer

ckifer commented Oct 31, 2025

Copy link
Copy Markdown
Member

sick, now I can be even more lazy

@PavelVanecek

Copy link
Copy Markdown
Collaborator Author

Hmmm that didn't work. I will try to merge it and see if it runs from main.

@PavelVanecek
PavelVanecek merged commit 392719b into main Oct 31, 2025
6 of 9 checks passed
@PavelVanecek
PavelVanecek deleted the vr-ci branch October 31, 2025 00:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants