Problem

bin/dev is a critical user-facing script that is completely untested in CI. Every bin/dev bug goes straight to users.

Evidence

Recent bugs that made it to releases:

• PR Fix bin/dev failing with 'Unknown argument' when using --route flag #2273 - --route flag parsing broke bin/dev immediately after rails generate react_on_rails:install
• This is at least the 2nd or 3rd bin/dev issue that passed CI but broke for users

Why This Happens

CI builds assets directly without using bin/dev:

# integration-tests.yml
- run: bundle exec rake react_on_rails:generate_packs
- run: bin/shakapacker  # Direct webpack, bypasses bin/dev entirely
- run: bundle exec rake run_rspec:all_dummy

# playwright.yml  
- run: bundle exec rake react_on_rails:generate_packs
- run: pnpm run build:test  # Direct webpack
- run: pnpm run test:e2e

The dummy app has a bin/dev script, but it's never executed in CI.

What Should Be Tested

1. Argument parsing - --route, --verbose, commands like static, kill, help
2. Script loading - No NameError, LoadError, SyntaxError
3. Full startup flow - bin/dev starts, webpack compiles successfully
4. Server responds - Rails serves pages correctly

Proposed Solution

Layer 1: Unit Tests (fast, every PR)

Following Rails' CLI testing pattern:

describe ".run_from_command_line" do
  before do
    allow(described_class).to receive(:start)  # Mock execution
  end
  
  it "parses --route with separate value" do
    expect(described_class).to receive(:start).with(
      :development, "Procfile.dev", hash_including(route: "hello_world")
    )
    described_class.run_from_command_line(["--route", "hello_world"])
  end
end

Layer 2: Integration Test (thorough, on master or bin/dev changes)

- name: Test bin/dev starts and compiles
  run: |
    ./bin/dev > /tmp/bin_dev.log 2>&1 &
    # Wait for "compiled successfully" or fail on errors
    # Then kill the process

Acceptance Criteria

• Unit tests for ServerManager.run_from_command_line argument parsing
• Unit tests for extract_command_from_args helper
• Integration test that actually runs bin/dev and waits for webpack compile
• CI workflow that runs integration test on relevant file changes
• The --route bug (PR Fix bin/dev failing with 'Unknown argument' when using --route flag #2273) would have been caught by these tests

Related

• PR Fix bin/dev failing with 'Unknown argument' when using --route flag #2273 - Fix for --route parsing bug (symptom)
• PR Add automatic precompile hook coordination in bin/dev #2092 - Introduced the --route bug (precompile hook coordination)