feat: add idempotent manual testing workflow by subsy · Pull Request #242 · subsy/ralph-tui

subsy · 2026-01-30T13:34:38Z

Add testing infrastructure for repeatable end-to-end workflow testing:

testing/TESTING.md: Comprehensive guide for manual testing
testing/test-prd.json: Test PRD with 5 tasks and dependency graph
testing/setup-test-workspace.sh: Creates fresh git repo for testing
testing/reset-test.sh: Resets all state for re-running tests

The test PRD exercises:

Parallel task execution (TEST-001, 002, 003 have no deps)
Dependency resolution (TEST-004 depends on 001 and 002)
Final aggregation (TEST-005 depends on 003 and 004)

https://claude.ai/code/session_019Q1N2XskiaS1omXVN3Xrt2

Summary by CodeRabbit

Documentation
- Added a comprehensive manual testing guide detailing end-to-end workflows, quick-start steps, architecture rationale, reset procedures, scenarios (parallel execution, dependency resolution, error recovery) and troubleshooting.
Chores
- Introduced repeatable testing infrastructure: automated workspace setup, idempotent reset and cleanup procedures, a structured test plan, and a helper to persist the test workspace path.
Minor
- Added an ignore entry for the manual test workspace.

_{✏️ Tip: You can customize this high-level summary in your review settings.}

Add testing infrastructure for repeatable end-to-end workflow testing: - testing/TESTING.md: Comprehensive guide for manual testing - testing/test-prd.json: Test PRD with 5 tasks and dependency graph - testing/setup-test-workspace.sh: Creates fresh git repo for testing - testing/reset-test.sh: Resets all state for re-running tests The test PRD exercises: - Parallel task execution (TEST-001, 002, 003 have no deps) - Dependency resolution (TEST-004 depends on 001 and 002) - Final aggregation (TEST-005 depends on 003 and 004) https://claude.ai/code/session_019Q1N2XskiaS1omXVN3Xrt2

Move test workspace to ~/.cache/ralph-tui/test-workspace to avoid: - Nested git repository issues - Accidental commits of test state - Confusion for contributors Changes: - setup-test-workspace.sh: Creates workspace in ~/.cache by default - reset-test.sh: Reads saved workspace path or uses default - TESTING.md: Updated documentation for external workspace - .gitignore: Track .test-workspace-path instead of test-workspace/ Contributors clone the repo and run setup to create their own isolated test environment. https://claude.ai/code/session_019Q1N2XskiaS1omXVN3Xrt2

Update all testing documentation and script outputs to use the --cwd parameter instead of requiring users to cd first. This is cleaner and more explicit about where ralph-tui will operate. https://claude.ai/code/session_019Q1N2XskiaS1omXVN3Xrt2

Assume user is in ralph-tui directory using dev builds. Removes redundant options and cd commands for cleaner docs. https://claude.ai/code/session_019Q1N2XskiaS1omXVN3Xrt2

vercel · 2026-01-30T13:34:45Z

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment

Project	Deployment	Actions	Updated (UTC)
ralph-tui	Ignored	Preview	Jan 30, 2026 1:42pm

coderabbitai · 2026-01-30T13:34:55Z

Walkthrough

Adds a manual end-to-end testing suite for Ralph‑TUI: documentation, an isolated test workspace setup script, an idempotent reset script, a JSON PRD-style test plan, and a .gitignore entry for storing the test workspace path.

Changes

Cohort / File(s)	Summary
Configuration `\.gitignore`	Add ignore entry for `testing/.test-workspace-path` to avoid committing local test workspace location.
Testing Documentation `testing/TESTING.md`	New comprehensive manual testing guide: quick start, architecture rationale, PRD test plan, detailed workflows, reset procedures, troubleshooting, and CI guidance.
Workspace Setup Script `testing/setup-test-workspace.sh`	New script to create an isolated test workspace (default under `~/.cache/ralph-tui/test-workspace`), initialise Git, populate initial files, tag a reset point, and record workspace path.
Reset / Cleanup Script `testing/reset-test.sh`	New idempotent reset script: resolves test workspace path, resets `test-prd.json` passes, removes generated outputs and session state, provides Git reset guidance and a suggested run command.
Test Plan Data `testing/test-prd.json`	New PRD-style JSON test plan with five user stories (TEST-001…TEST-005), dependencies, acceptance criteria, expected output files and metadata.

Sequence Diagram(s)

mermaid
sequenceDiagram
rect rgba(200,200,255,0.5)
participant Tester
end
rect rgba(200,255,200,0.5)
participant SetupScript as Setup Script
end
rect rgba(255,200,200,0.5)
participant Workspace as Test Workspace (Git)
end
rect rgba(255,255,200,0.5)
participant Ralph as Ralph‑TUI
end
rect rgba(200,255,255,0.5)
participant ResetScript as Reset Script
end

Tester->>SetupScript: run `setup-test-workspace.sh` (create workspace, init Git, seed files, tag)
SetupScript->>Workspace: create repo, initial commit, create tag `test-start`
Tester->>Ralph: run tests against Workspace (e.g. `bun run ... --workspace <path>`)
Ralph->>Workspace: read `test-prd.json`, execute tasks, write outputs
Tester->>ResetScript: run `reset-test.sh` (reset PRD passes, clean outputs, clear session)
ResetScript->>Workspace: remove generated files, reset session state, print Git reset commands

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

🚥 Pre-merge checks | ✅ 3

✅ Passed checks (3 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The PR title 'feat: add idempotent manual testing workflow' directly and concisely summarizes the main change: adding testing infrastructure for repeatable, idempotent manual workflow testing.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing touches

📝 Generate docstrings

🧪 Generate unit tests (beta)

Create PR with unit tests
Post copyable unit tests in a comment
Commit unit tests in branch claude/ralph-tui-testing-workflow-CT50f

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

coderabbitai

Actionable comments posted: 2

🤖 Fix all issues with AI agents

In `@testing/reset-test.sh`:
- Around line 46-50: The fallback sed invocation (sed -i 's/"passes":
true/"passes": false/g' "$SCRIPT_DIR/test-prd.json") is not portable on macOS;
update the script to perform an in-place, cross-platform replace (e.g., use perl
-pi -e 's/"passes": true/"passes": false/g' "$SCRIPT_DIR/test-prd.json" or
detect Darwin and call sed -i '' on macOS) and keep the echo line (echo -e
"${GREEN}  PRD reset (via sed): all tasks set to passes: false${NC}") unchanged;
alternatively add a short comment or error recommending jq be installed and used
for the preferred path.

In `@testing/TESTING.md`:
- Around line 32-35: Update the example command in TESTING.md to use the correct
relative path to the script by changing the invocation from
"./setup-test-workspace.sh /custom/path/to/workspace" to
"./testing/setup-test-workspace.sh /custom/path/to/workspace" so it matches the
actual script location and other docs; locate the example string in the file and
replace the path accordingly.

🧹 Nitpick comments (1)

testing/setup-test-workspace.sh (1)
14-19: Unused colour variable.

The RED variable is defined but never used in this script. Consider removing it for cleanliness, or keep it for future error message additions.
♻️ Optional: Remove unused variable
 # Colors
-RED='\033[0;31m'
 GREEN='\033[0;32m'
 YELLOW='\033[1;33m'
 BLUE='\033[0;34m'
 NC='\033[0m'

testing/reset-test.sh

testing/TESTING.md

sed -i behaves differently on macOS (requires empty string argument). perl -pi -e is portable across both macOS and Linux. https://claude.ai/code/session_019Q1N2XskiaS1omXVN3Xrt2

Use ./testing/setup-test-workspace.sh to match other docs. https://claude.ai/code/session_019Q1N2XskiaS1omXVN3Xrt2

…flow-CT50f feat: add idempotent manual testing workflow

claude added 4 commits January 29, 2026 13:55

docs: simplify testing workflow for dev builds

0a61173

Assume user is in ralph-tui directory using dev builds. Removes redundant options and cd commands for cleaner docs. https://claude.ai/code/session_019Q1N2XskiaS1omXVN3Xrt2

coderabbitai bot reviewed Jan 30, 2026

View reviewed changes

testing/reset-test.sh Show resolved Hide resolved

testing/TESTING.md Show resolved Hide resolved

claude added 2 commits January 30, 2026 13:41

fix: correct script path in TESTING.md example

64dba60

Use ./testing/setup-test-workspace.sh to match other docs. https://claude.ai/code/session_019Q1N2XskiaS1omXVN3Xrt2

subsy merged commit 046a6eb into main Jan 30, 2026
9 checks passed

subsy deleted the claude/ralph-tui-testing-workflow-CT50f branch January 30, 2026 14:45

sakaman pushed a commit to sakaman/ralph-tui that referenced this pull request Feb 15, 2026

Merge pull request subsy#242 from subsy/claude/ralph-tui-testing-work…

14dffec

…flow-CT50f feat: add idempotent manual testing workflow

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: add idempotent manual testing workflow#242

feat: add idempotent manual testing workflow#242
subsy merged 6 commits intomainfrom
claude/ralph-tui-testing-workflow-CT50f

subsy commented Jan 30, 2026 •

edited by coderabbitai bot

Loading

Uh oh!

vercel bot commented Jan 30, 2026 •

edited

Loading

Uh oh!

coderabbitai bot commented Jan 30, 2026 •

edited

Loading

Uh oh!

coderabbitai bot left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Comments

Conversation

subsy commented Jan 30, 2026 • edited by coderabbitai bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary by CodeRabbit

Uh oh!

vercel bot commented Jan 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

coderabbitai bot commented Jan 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Sequence Diagram(s)

Estimated code review effort

Uh oh!

coderabbitai bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Comments

subsy commented Jan 30, 2026 •

edited by coderabbitai bot

Loading

vercel bot commented Jan 30, 2026 •

edited

Loading

coderabbitai bot commented Jan 30, 2026 •

edited

Loading