Severity

Medium for project quality. The repo already has the test infrastructure to enforce this — it just isn't used.

Location

• CLAUDE.md — "MANDATORY: Live Integration Testing" section
• crates/librefang-api/tests/api_integration_test.rs — 4,173 lines, real axum + reqwest test harness already in place

Finding

CLAUDE.md describes an 8-step "Live Integration Testing" workflow that contributors are expected to run manually after every endpoint, feature, or wiring change. The steps include:

1. tasklist | grep -i librefang && taskkill //PID <pid> //F
2. cargo build --release -p librefang-cli
3. GROQ_API_KEY=... target/release/librefang.exe start &
4. curl http://127.0.0.1:4545/api/<endpoint>
5. POST/PUT real payloads
6. Verify metering / cost side-effects via additional curl
7. curl ... | grep -c newComponentName to verify dashboard HTML
8. taskkill //PID <pid> //F

This is a manual checklist. It is not part of cargo test, not part of CI, and not enforced by any pre-merge gate. The default outcome is: contributors skip steps that are inconvenient, and regressions land.

The most common bug it would catch — "endpoint added in routes.rs but not registered in server.rs" — is the same class of bug CLAUDE.md repeatedly warns about. But there's no automation to actually catch it.

Wasteful aspect

api_integration_test.rs already has a TestServer that spawns real axum + reqwest. Steps 4–6 of the manual workflow are trivial to express as #[tokio::test] cases against TestServer. The barrier is not capability; it's that the workflow was written as a checklist instead of as code.

Step 7 is fragile regardless

curl ... | grep -c newComponentName to verify dashboard HTML breaks the moment Vite minifies a class name or React renames a component. This was never a robust check; the modern equivalent is Playwright or Vitest+JSDOM hitting the SPA.

Suggested fix

Phase 1 — automate the dead-route audit:

Add to api_integration_test.rs:

#[tokio::test]
async fn dead_route_audit() {
    let server = TestServer::start().await;
    let routes = server.app.routes();  // axum 0.7+: Router::layer-listing
    for path in routes {
        // GET should not panic and should not return 404 if listed
        let r = server.client.get(path).send().await.unwrap();
        assert_ne!(r.status(), StatusCode::NOT_FOUND, "route listed but returns 404: {path}");
    }
}

If axum doesn't expose route enumeration, maintain a lightweight routes! macro that records each route at registration time, and audit against that list. This catches the #1 documented gotcha (added route in routes.rs without server.rs).

Phase 2 — convert curl steps 4–6 to tests:

For each new endpoint, the contributor adds an integration test in api_integration_test.rs that:

1. Hits the endpoint with a fixture payload
2. Asserts the response shape
3. For write endpoints, hits a follow-up read endpoint and asserts the side effect

Reviewers gate PRs on the presence of this test for new endpoints. CI runs them on every push. CLAUDE.md's manual steps 4–6 disappear.

Phase 3 — Dashboard validation via Playwright (optional but preferred):

Replace step 7 with a Playwright spec under crates/librefang-api/dashboard/e2e/. Run in CI on a smoke-test job.

Phase 4 — Live LLM test via #[ignore]:

For step 5 (real Groq call), keep the test in the suite but mark #[ignore]. Run via cargo test -- --include-ignored in a nightly job that has access to API keys via secrets. Don't burn budget on every PR.

Acceptance criteria

• cargo test -p librefang-api includes a dead-route audit test that fails when a route is registered but unreachable
• At least 80% of the curl-equivalent assertions in CLAUDE.md's "Step 4" are present as #[tokio::test] cases
• CLAUDE.md's "Live Integration Testing" section is rewritten to point at api_integration_test.rs as the canonical enforcement, not as a manual checklist
• CI gate: PR cannot land if dead_route_audit fails

Related

• The two CLAUDE.md gotchas this addresses: "Missing route registrations in server.rs" and "Endpoints that compile but return wrong/empty data".