Skip to content

Commit 7e305f5

Browse files
pshort05claude
andcommitted
fix(tests): stop the flaky unit suite by running it in one process
- root cause is Node's own test-runner bug (nodejs/node#64061): the parent intermittently fails to deserialize a child subprocess's V8 message stream ("Unable to deserialize cloned data"), so the failing file was random and every file passed in isolation — not our test logic - run test:unit with --experimental-test-isolation=none: single process, no child->parent IPC channel for the bug to occur in (14/14 clean vs ~1-in-4 base failure; verified isolation-safe — env-mutating tests save/restore distinct vars) - the TODO's ENOENT-rename hypothesis was wrong: that "Failed to persist confirmations" line is benign caught noise present in passing runs too - clean it up anyway so it stops misleading: expireOne's fire-and-forget persist (lazy expiry sweep) raced the tests' rmSync; add ConfirmationGateService.whenIdle() and await it in afterEach + the two short-expiry tests so the atomic write-then-rename finishes first - moved the TODO item to COMPLETED verified: tsc clean, npm run test:unit 2138/2138, 0 deserialize failures and 0 confirmation-gate ENOENT noise across 14 consecutive full-suite runs Co-Authored-By: Claude Opus 4.8 <[email protected]>
1 parent ac9b776 commit 7e305f5

5 files changed

Lines changed: 19 additions & 9 deletions

File tree

‎docs/COMPLETED.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,8 @@ Items moved here from [TODO.md](TODO.md) as they are finished. Newest first. Eac
44

55
## 2026-07-12
66

7+
- **Unit suite flaky under parallel `node --test` [surfaced during premise-intake testing].** _Original TODO: "the same tree passes or fails run to run, and the failing file moves … Every failure surfaces as `ENOENT … rename '/tmp/bc-cg-XXXXXX/confirmations.json.tmp' …`, each file passes in isolation."_ Root-caused via systematic debugging: the TODO's ENOENT hypothesis was **wrong** — that `Failed to persist confirmations` line is benign console.error noise (caught inside `ConfirmationGateService.persist()`) that appears in *passing* runs too. The actual failure is a known Node.js test-runner bug ([nodejs/node#64061](https://github.com/nodejs/node/issues/64061), still open through 24.x): the parent runner intermittently fails to deserialize a child test-subprocess's V8-serialized message stream (`Unable to deserialize cloned data due to invalid or unsupported version`, stack entirely in `node:internal/test_runner/runner` `#processRawBuffer`/`parseMessage`), so the "failing file" is whichever child's stream got misparsed — which is why every file passes in isolation. **Fix (primary):** run the 323-file suite in a single process — `test:unit` now passes `--experimental-test-isolation=none`, which removes the per-file child→parent IPC channel the bug lives in (verified isolation-safe: the handful of env-mutating tests all save/restore distinct vars). Base rate was ~1 fail in 4 runs; **14/14 clean** with the flag. **Fix (secondary, cleanup):** eliminated the misleading ENOENT noise that mis-led the diagnosis — `expireOne`'s fire-and-forget `persist()` (scheduled by the lazy expiry sweep in `get()`/`list()`) raced the tests' synchronous `rmSync(ws)`; added `ConfirmationGateService.whenIdle()` (drains the persist chain) and awaited it in `afterEach` + the two short-expiry tests, so the atomic write-then-rename completes while its temp dir still exists. **Verified:** `tsc --noEmit` clean; full `npm run test:unit` 2138/2138 with 0 deserialize failures and 0 ENOENT noise across 14 consecutive full-suite runs.
8+
79
- **AuthorAgent Tier-4 feature port (8 features, #10–#17) [owner ask 2026-07-11].** Ported the eight Tier-4 items from `docs/AUTHORAGENT-PORT-ANALYSIS-2026-07-11.md`, each grounded in the `Ckokoski/AuthorAgent` fork's actual code and adapted to BookClaw, via spec → plan → 7 parallel disjoint-file TDD implementers → 2 parallel Wave-2 wiring clusters → Opus whole-branch code review (1 Important + Minors, Important fixed) → smoke → Mercury deploy. Three items were honestly **scoped down** where the fork's feature was redundant against a stronger BookClaw implementation. **(#10) Vault key-format validation** (`_shared.ts` `validateKeyFormat`, `settings.routes.ts`): non-blocking `warning` on the `/api/vault` response when a saved key doesn't match its provider slot (never blocks the save). **(#11) Onboarding status** (`onboarding-status.ts`, `core.routes.ts`): read-only `GET /api/onboarding/status` first-run checklist (provider/voice/soul/project/telegram/discord), each signal fail-soft, `firstRun` gated on provider+project. **(#12) Character motivation critique** (`character-motivation.ts`, `POST /api/projects/:id/motivation-critique`, MCP `motivation_critique`): AI check for lines that contradict a character's established motivation/arc — **scoped to motivation only** (knowledge-horizon dropped as redundant vs `continuity-check.ts` `evaluateKnowledge`; off-voice dropped vs `character-voices.ts`). **(#13) Translation execution** (`translation-pipeline.ts` `executeTranslation`, `POST /api/translation/execute`): completes the half-built feature (planning already shipped) — paragraph-boundary chunking with one-retry-then-error-marker fail-soft, gated at the route via `requireApprovedConfirmation`/`recordOutcome` (fork's in-service gate dropped for BookClaw's route convention), per-chunk cost recorded model-aware. **(#14) Model-aware pricing** (`ai/pricing.ts`, `costs.ts`): per-model `LLM_PRICING` table + `getLLMPrice`; `CostTracker.record()` now **overrides** the router's flat estimate for known non-OpenRouter models (fixes the acknowledged `#35b` Opus underpricing — Opus was billed ~40% cheap), threaded through 11 `record()` call sites. **(#15) Path-safety consolidation** (`security/paths.ts`, `sandbox.ts`, `memory.ts`, `_shared.ts`): one hardened `resolveWithin`/`safeResolveWithin`/`sanitizeSegment` module (null-byte, Windows-reserved, `..`); the 28 `safePath` sites collapse to a one-line alias; **`SandboxGuard`'s `realpathSync` symlink-escape defense preserved** (the fork's port dropped it — a regression we avoided), guarded by a symlink regression test. **(#16) Dialogue-parser deduplication** (`dialogue-parser.ts`, `character-voices.ts`, `audiobook-prep.ts`): one shared parser fixes a confirmed untrimmed-name bug (`buildNameLookup` trims key+value) that silently dropped dialogue lines / mis-voiced TTS; `dialogue-auditor.ts` left untouched (architecturally distinct); code review caught and fixed a reverse-tag quote-anchor regression in `character-voices`. **(#17) Revision-orchestrator** (`revision-orchestrator.ts`, `POST /api/projects/:id/revision-report`, MCP `revision_report`): thin aggregator over craft/dialogue/continuity/voice/mechanical detectors into one severity-ranked report (no new detection; continuity reads persisted per-step flags — zero extra AI; route resolves character names from the entity DB). **Verified:** 8 feature unit-test files + 2 reverse-tag regression tests all green, full suite 2137/2137, `tsc --noEmit` clean (root + mcp), mcp builds; new `tests/tier4-features-smoke.sh` passes 7/7 against Mercury; Tier-1/2 smokes non-regressed. Spec: `docs/superpowers/specs/2026-07-11-authoragent-tier4-features-design.md`; plan: `docs/superpowers/plans/2026-07-11-authoragent-tier4-features-plan.md`. Deployed to Mercury. _(Owner ask 2026-07-11.)_
810

911
## 2026-07-11

‎docs/TODO.md‎

Lines changed: 0 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -4,13 +4,6 @@ Tracking list of items surfaced while writing CLAUDE.md. Grouped by effort.
44

55
Anything currently being worked on must appear in this list. When an item is finished, move it to [COMPLETED.md](COMPLETED.md) with a `YYYY-MM-DD` completion date — don't just check the box and leave it here.
66

7-
## Next up (2026-07-12)
8-
9-
- [ ] **Unit suite is flaky under parallel `node --test`** — the same tree passes or fails run to run, and the failing file *moves*. Observed on three consecutive full runs: run 1 `book-phase-hook.test.ts` failed, run 2 `ConfirmationGateService` failed, run 3 passed 2138/2138. Every failure so far surfaces as `ENOENT: no such file or directory, rename '/tmp/bc-cg-XXXXXX/confirmations.json.tmp' -> '.../confirmations.json'`, i.e. a test's temp dir is gone while that test is still writing. Each file passes 17/17 in isolation, so it is cross-file interference, not test logic.
10-
- **Repro:** `node --import tsx --test $(ls tests/unit/*.test.ts | grep -vE 'studio-build|chat-build')` — run it several times; roughly 2 in 3 runs fail somewhere.
11-
- **Likely cause:** tests share a temp-dir root or `rmSync` cleanup races against another file's still-open service (the atomic write-then-rename in the persist path is the victim, not the culprit). Suspect a shared `/tmp` prefix, an unawaited async persist in a service that outlives its test, or an `afterEach` cleaning a dir another test still holds.
12-
- **Why it matters:** an intermittently-red suite trains everyone to ignore it, and it is the only automated gate on ~2,138 assertions. Not a product bug — pre-existing, unrelated to the premise-intake fix that surfaced it.
13-
147
## Targeted feature roadmap (product, consolidated 2026-06-21)
158

169
A single ranked, deduplicated list of the **differentiating product features** distilled from [STRATEGY-LEADING-AI-WRITING-ASSISTANT.md](STRATEGY-LEADING-AI-WRITING-ASSISTANT.md) + the six tool reviews ([NARRATIVE-ENGINE](NARRATIVE-ENGINE-INTEGRATION.md), [STORYTHREAD](STORYTHREAD-STUDIO-INTEGRATION.md), [MIRRORSHARD](MIRRORSHARD-INTEGRATION.md), [NOVELMINT-TOOLS](NOVELMINT-TOOLS-REVIEW.md), [CLAUDE-CODE-METHODS](CLAUDE-CODE-WRITING-METHODS-REVIEW.md), [LONEWRITER](LONEWRITER-REVIEW.md)). **Detailed scoping for each item lives in the linked doc** — this is the canonical index; the scattered strategy/review bullets that fed it were removed from "Larger items" below so each feature appears once. The **multi-author/multi-book studio** (North Star, below) is the umbrella platform these build on; it is not re-listed here.

‎gateway/src/services/confirmation-gate.ts‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -119,6 +119,17 @@ export class ConfirmationGateService {
119119
this.auditFn = fn;
120120
}
121121

122+
/**
123+
* Resolve once every queued persist has settled. Lazy expiry (reached through
124+
* the synchronous get()/list() sweep) schedules a fire-and-forget persist, so
125+
* a caller about to delete the workspace directory awaits this first — that
126+
* lets the atomic write-then-rename finish while its target dir still exists,
127+
* instead of racing the teardown and logging a spurious ENOENT.
128+
*/
129+
async whenIdle(): Promise<void> {
130+
await this.persistChain;
131+
}
132+
122133
async initialize(): Promise<void> {
123134
await mkdir(join(this.filePath, '..'), { recursive: true });
124135
if (existsSync(this.filePath)) {

‎package.json‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,7 @@
2121
"security-audit": "node scripts/security-check.js",
2222
"export-book": "bash scripts/export-book.sh",
2323
"test": "npm run test:unit && npm run test:api && npm run test:smoke",
24-
"test:unit": "npm run build:frontend && node --import tsx --test tests/unit/*.test.ts",
24+
"test:unit": "npm run build:frontend && node --import tsx --test --experimental-test-isolation=none tests/unit/*.test.ts",
2525
"test:api": "bash tests/api/api-test.sh",
2626
"test:smoke": "npm run build:frontend && bash tests/smoke-test.sh",
2727
"test:intake-smoke": "bash tests/romance-premise-intake-smoke.sh",

‎tests/unit/confirmation-gate.test.ts‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -32,7 +32,9 @@ describe('ConfirmationGateService', () => {
3232
svc = new ConfirmationGateService(ws);
3333
await svc.initialize();
3434
});
35-
afterEach(() => { rmSync(ws, { recursive: true, force: true }); });
35+
// Drain any fire-and-forget persist (scheduled by lazy expiry in get()/list())
36+
// before removing the temp dir, so the atomic rename can't race the teardown.
37+
afterEach(async () => { await svc.whenIdle(); rmSync(ws, { recursive: true, force: true }); });
3638

3739
test('createRequest returns a pending request with an id and copied fields', async () => {
3840
const req = await svc.createRequest(baseInput());
@@ -191,6 +193,7 @@ describe('ConfirmationGateService', () => {
191193
// checkDecision surfaces the expiry via the lazy sweep in get()
192194
assert.equal(fast.checkDecision(req.id).status, 'expired');
193195
await assert.rejects(() => fast.approve(req.id), /expired/);
196+
await fast.whenIdle();
194197
});
195198

196199
test('list filters by status and service; get returns the stored request', async () => {
@@ -214,5 +217,6 @@ describe('ConfirmationGateService', () => {
214217
await new Promise(r => setTimeout(r, 40));
215218
assert.equal(fast.list({ status: 'pending' }).length, 0);
216219
assert.equal(fast.list({ status: 'expired' }).length, 1);
220+
await fast.whenIdle();
217221
});
218222
});

0 commit comments

Comments
 (0)