Skip to content

Commit 02fc6b4

Browse files
Rejnyxclaude
andcommitted
feat(tests): Tier 4 — hook contract (block-destructive + 5 critical hooks)
Pre-Hermes HARD GATE #1 lands. Hermes runtime will spawn cortex-x hooks on every tool call, every session, every prompt — without contract tests on these, a regression to block-destructive (e.g., a regex breaks and stops blocking dangerous bash commands) ships silently and damages users. Five components, 92 tests, all green on Win native Node 25.0.0: 1. tests/_helpers/run-hook.cjs — spawn hook with stdin payload + capture stdout/stderr/exit. Cross-platform (process.execPath bypasses shebang/+x), 5s default timeout, signal-based timeout detection (matches Node 22+ behavior — no timedOut flag exposed by spawnSync, per nodejs/node#51561). 2. tests/unit/hooks/block-destructive.test.cjs (28 tests) — runtime safety contract: - 6 filesystem destruction patterns blocked + 2 allow-patterns confirmed - 9 git destructive ops blocked (force-push, hard reset, force-clean, force-branch-delete, blanket checkout/restore, stash destruction) + 4 allow-git patterns - 4 database destruction blocks (DROP TABLE, DROP DATABASE, TRUNCATE, supabase db reset) + SELECT allowed - 4 fail-open scenarios (malformed JSON, empty stdin, non-Bash tool, 5s timeout budget) Block path uses JSON-on-stdout + exit 0 (the post-Claude-Code-v2.1.90 modern contract per code.claude.com/docs/en/changelog — the older exit-2 path was buggy in v2.1.32-2.1.89). 3. tests/unit/hooks/session-start.test.cjs (7 tests) — context-injection contract: - Output schema (hookSpecificOutput, hookEventName=SessionStart, additionalContext as string) - Bare project (no docs/git) returns valid output - 5s budget respected - CLAUDE.md / PROGRESS.md detection surfaces in additionalContext - CORTEX_DATA_HOME env override resolves projects-library entry - PII guard (no Dave-specific path leaks) 4. tests/unit/hooks/auto-orchestrate.test.cjs (16 tests) — UserPromptSubmit soft-gate: - 5 trigger patterns (cs+en) emit guidance via hookSpecificOutput - 5 skip patterns return silent continue=true - 5 fail-open scenarios (malformed JSON, empty stdin, missing prompt field, 5s timeout, CORTEX_BUDGET_DISABLED suppresses USD figures) - PII guard 5. tests/unit/hooks/pre-compact.test.cjs (6 tests) — state-snapshot: - Writes .claude/compact-state.md on bare project - Captures Sprint name from PROGRESS.md - Mentions CLAUDE.md when present - mkdirSync .claude/ on first run (no manual setup required) - Overwrites stale state on re-run (idempotent) - 5s budget 6. tests/contract/hook-shape.test.cjs (35 tests) — generic invariants across ALL 7 hooks (block-destructive, session-start, pre-compact, auto-orchestrate, pre-tool-use, post-tool-use, tirith-scan): - Hook script exists as regular file - Respects 5s timeout on empty stdin (catches reads-stdin-forever hangs) - Doesn't leak Dave-specific paths - Fails open on malformed JSON (no crash) Plus a critical-hook-present hard-coded list — if any of the 6 Hermes-runtime-required hooks gets renamed/removed, this fires loudly before Hermes lands. 7. verify-install.cjs extended — now requires pre-compact, auto-orchestrate, pre-tool-use, post-tool-use as blocker-severity files (was just session-start + block-destructive before). Field tests would not have caught a missing pre-tool-use because nothing exercised it. Local: npm test = 183/183 pass on Win native, full suite ~8s Local: npm run test:smoke = 36/36 checks (was 32) on live install Research grounding (3 parallel web research agents on 2026-05-07): - code.claude.com/docs/en/hooks (canonical hook reference, May 2026) - code.claude.com/docs/en/changelog (v2.1.90 = floor for JSON-stdout block contract; v2.1.105 = PreCompact officially added) - karanb192/claude-code-hooks (active 2026 OSS test suite using node --test, same shape as cortex-x) - nodejs/node#51561 (canonical timeout-detection pattern) - push-based/node-cli-testing (spawnSync wrapper reference) Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
1 parent 87937f2 commit 02fc6b4

7 files changed

Lines changed: 847 additions & 0 deletions

File tree

‎tests/_helpers/run-hook.cjs‎

Lines changed: 78 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,78 @@
1+
// run-hook.cjs — invoke a cortex-x hook script with mock stdin payload.
2+
//
3+
// Hooks are stdin-driven Node CJS scripts under shared/hooks/<name>.cjs.
4+
// Claude Code spawns them per event (PreToolUse, SessionStart, PostToolUse,
5+
// UserPromptSubmit, PreCompact) and pipes a JSON payload on stdin. The hook
6+
// writes structured JSON or human messages to stdout/stderr and exits.
7+
//
8+
// This helper standardizes test invocations:
9+
//
10+
// const { runHook } = require('../_helpers/run-hook.cjs');
11+
// const r = runHook('block-destructive', {
12+
// tool_name: 'Bash',
13+
// tool_input: { command: 'rm -rf /' },
14+
// });
15+
// assert.equal(r.exitCode, 0); // fail-open contract
16+
// assert.match(r.stdout, /permissionDecision.*deny/);
17+
//
18+
// Cross-platform: spawnSync invokes hook via process.execPath (the running
19+
// node binary), bypassing the shebang line and OS-level execute permissions.
20+
// Works on Linux, macOS, Windows native, Git Bash without modification.
21+
22+
'use strict';
23+
24+
const path = require('node:path');
25+
const { spawnSync } = require('node:child_process');
26+
27+
const REPO_ROOT = path.resolve(__dirname, '..', '..');
28+
const HOOKS_DIR = path.join(REPO_ROOT, 'shared', 'hooks');
29+
30+
const DEFAULT_TIMEOUT_MS = 5000;
31+
32+
function runHook(name, payload, options = {}) {
33+
const hookPath = path.join(HOOKS_DIR, `${name}.cjs`);
34+
const input =
35+
payload === undefined || payload === null
36+
? ''
37+
: typeof payload === 'string'
38+
? payload
39+
: JSON.stringify(payload);
40+
41+
const result = spawnSync(process.execPath, [hookPath], {
42+
encoding: 'utf8',
43+
input,
44+
timeout: options.timeout || DEFAULT_TIMEOUT_MS,
45+
cwd: options.cwd || REPO_ROOT,
46+
env: {
47+
...process.env,
48+
...(options.env || {}),
49+
},
50+
});
51+
52+
return {
53+
exitCode: result.status,
54+
signal: result.signal,
55+
stdout: result.stdout || '',
56+
stderr: result.stderr || '',
57+
timedOut: result.signal === 'SIGTERM',
58+
error: result.error,
59+
};
60+
}
61+
62+
function parseHookOutput(stdout) {
63+
const trimmed = (stdout || '').trim();
64+
if (!trimmed) return null;
65+
try {
66+
return JSON.parse(trimmed);
67+
} catch {
68+
return null;
69+
}
70+
}
71+
72+
module.exports = {
73+
runHook,
74+
parseHookOutput,
75+
REPO_ROOT,
76+
HOOKS_DIR,
77+
DEFAULT_TIMEOUT_MS,
78+
};

‎tests/contract/hook-shape.test.cjs‎

Lines changed: 123 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,123 @@
1+
// hook-shape.test.cjs — contract every shared/hooks/*.cjs must satisfy.
2+
//
3+
// Generic invariants — applied to every hook, regardless of event type.
4+
// These catch regressions that unit tests miss because unit tests focus on
5+
// hook-specific behavior. The contract tests check the *boundary* every
6+
// hook crosses.
7+
//
8+
// Invariants:
9+
// 1. Hook script exists and is a regular file
10+
// 2. Hook script can be require()'d without crashing (syntax-clean)
11+
// EXCEPT: stdin-driven hooks that immediately invoke process.exit on
12+
// empty stdin will be skipped here (covered by spawnSync tests)
13+
// 3. Hook honors the 5s timeout (always returns within budget)
14+
// 4. Hook does NOT leak Dave-specific paths in stdout/stderr
15+
// 5. Hook fails open: malformed JSON / empty stdin → exit 0 (no crash)
16+
//
17+
// The list of hooks is derived from shared/hooks/*.cjs (excluding _lib/).
18+
19+
'use strict';
20+
21+
const { test, describe } = require('node:test');
22+
const assert = require('node:assert/strict');
23+
const fs = require('node:fs');
24+
const path = require('node:path');
25+
const { runHook, HOOKS_DIR } = require('../_helpers/run-hook.cjs');
26+
27+
function listHooks() {
28+
return fs
29+
.readdirSync(HOOKS_DIR)
30+
.filter((f) => f.endsWith('.cjs') && !f.endsWith('.test.cjs'))
31+
.map((f) => f.replace(/\.cjs$/, ''));
32+
}
33+
34+
const ALL_HOOKS = listHooks();
35+
36+
// Hooks that operate on cwd-side-effects (write files) — give them a tmp
37+
// cwd so they don't pollute the repo.
38+
const NEEDS_TMP_CWD = new Set(['pre-compact', 'session-start']);
39+
40+
const os = require('node:os');
41+
42+
function tempCwd() {
43+
return fs.mkdtempSync(path.join(os.tmpdir(), 'cortex-hook-shape-'));
44+
}
45+
46+
describe('hook-shape contract: all shared/hooks/*.cjs', () => {
47+
test('inventory non-empty', () => {
48+
assert.ok(ALL_HOOKS.length >= 5, `expected >= 5 hooks; found ${ALL_HOOKS.length}: ${ALL_HOOKS.join(', ')}`);
49+
});
50+
51+
for (const hookName of ALL_HOOKS) {
52+
test(`${hookName}: file exists and is regular file`, () => {
53+
const p = path.join(HOOKS_DIR, `${hookName}.cjs`);
54+
assert.ok(fs.statSync(p).isFile());
55+
});
56+
57+
test(`${hookName}: respects 5s timeout (no stdin payload)`, () => {
58+
const cwd = NEEDS_TMP_CWD.has(hookName) ? tempCwd() : undefined;
59+
try {
60+
const r = runHook(hookName, '', { timeout: 5000, cwd });
61+
assert.equal(r.timedOut, false, `${hookName} timed out on empty stdin — likely waiting for input it'll never get`);
62+
// exit code 0 OR 1 acceptable (1 = non-blocking error per Anthropic
63+
// hook contract); 2 = blocking, also acceptable for some hook types
64+
assert.ok(
65+
[0, 1, 2, null].includes(r.exitCode),
66+
`${hookName} exited with unexpected code ${r.exitCode}; stderr: ${r.stderr}`
67+
);
68+
} finally {
69+
if (cwd) fs.rmSync(cwd, { recursive: true, force: true });
70+
}
71+
});
72+
73+
test(`${hookName}: does not leak Dave-specific paths`, () => {
74+
const cwd = NEEDS_TMP_CWD.has(hookName) ? tempCwd() : undefined;
75+
try {
76+
const r = runHook(hookName, '', { timeout: 5000, cwd });
77+
const combined = (r.stdout + r.stderr).toLowerCase();
78+
assert.ok(
79+
!combined.includes('/c/users/david/') && !combined.includes('c:\\users\\david\\'),
80+
`${hookName} leaked Dave-specific path; combined output:\n${(r.stdout + r.stderr).slice(0, 500)}`
81+
);
82+
} finally {
83+
if (cwd) fs.rmSync(cwd, { recursive: true, force: true });
84+
}
85+
});
86+
87+
test(`${hookName}: fail-open on malformed JSON stdin`, () => {
88+
const cwd = NEEDS_TMP_CWD.has(hookName) ? tempCwd() : undefined;
89+
try {
90+
const r = runHook(hookName, '{ this is not valid JSON }', { timeout: 5000, cwd });
91+
// Either exit 0 (graceful continue) OR null/SIGTERM (timeout — bad)
92+
assert.ok(r.exitCode === 0 || r.exitCode === 1,
93+
`${hookName} crashed on malformed JSON; exit ${r.exitCode}, stderr: ${r.stderr.slice(0, 300)}`);
94+
assert.equal(r.timedOut, false);
95+
} finally {
96+
if (cwd) fs.rmSync(cwd, { recursive: true, force: true });
97+
}
98+
});
99+
}
100+
});
101+
102+
describe('hook-shape contract: critical hooks must be present', () => {
103+
// Hard-coded list of hooks Hermes runtime will depend on — if any of these
104+
// is renamed/removed, this test fires loudly.
105+
const CRITICAL_HOOKS = [
106+
'block-destructive',
107+
'session-start',
108+
'pre-compact',
109+
'auto-orchestrate',
110+
'pre-tool-use',
111+
'post-tool-use',
112+
];
113+
114+
for (const hookName of CRITICAL_HOOKS) {
115+
test(`critical hook present: ${hookName}.cjs`, () => {
116+
const p = path.join(HOOKS_DIR, `${hookName}.cjs`);
117+
assert.ok(
118+
fs.existsSync(p),
119+
`Critical hook missing: ${hookName}. Hermes runtime depends on this.`
120+
);
121+
});
122+
}
123+
});

‎tests/smoke/verify-install.cjs‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -211,6 +211,11 @@ function runChecks() {
211211
checkFileExists('shared.agents.planner', path.join(SHARED, 'agents', 'planner.md'));
212212
checkFileExists('shared.hooks.session-start', path.join(SHARED, 'hooks', 'session-start.cjs'));
213213
checkFileExists('shared.hooks.block-destructive', path.join(SHARED, 'hooks', 'block-destructive.cjs'));
214+
// Tier 4 — additional critical hooks Hermes runtime depends on
215+
checkFileExists('shared.hooks.pre-compact', path.join(SHARED, 'hooks', 'pre-compact.cjs'));
216+
checkFileExists('shared.hooks.auto-orchestrate', path.join(SHARED, 'hooks', 'auto-orchestrate.cjs'));
217+
checkFileExists('shared.hooks.pre-tool-use', path.join(SHARED, 'hooks', 'pre-tool-use.cjs'));
218+
checkFileExists('shared.hooks.post-tool-use', path.join(SHARED, 'hooks', 'post-tool-use.cjs'));
214219

215220
// User-discoverable agents (Claude Code reads ~/.claude/agents/, NOT ~/.claude/shared/agents/)
216221
// Field test #5 caught this: install copied to staging only, default agents
Lines changed: 153 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,153 @@
1+
// auto-orchestrate.test.cjs — UserPromptSubmit soft-gate hook contract.
2+
//
3+
// auto-orchestrate.cjs intercepts UserPromptSubmit, detects "new
4+
// implementation" intents (implement feature / build / wire up / etc.),
5+
// and injects 3-fronta orchestration guidance + research cache state +
6+
// budget into the turn.
7+
//
8+
// Contract (per source: shared/hooks/auto-orchestrate.cjs):
9+
// - Input on stdin: { prompt, session_id, cwd, ... } (UserPromptSubmit)
10+
// - No trigger match → stdout: { continue: true } (silent pass-through)
11+
// - Trigger match → stdout: { continue: true, hookSpecificOutput: {
12+
// hookEventName: 'UserPromptSubmit',
13+
// additionalContext: <guidance string> } }
14+
// - Always exit 0 (fail-open contract)
15+
// - NEVER blocks the turn
16+
17+
'use strict';
18+
19+
const { test, describe } = require('node:test');
20+
const assert = require('node:assert/strict');
21+
const { runHook, parseHookOutput } = require('../../_helpers/run-hook.cjs');
22+
23+
function userPrompt(prompt, extra = {}) {
24+
return { prompt, session_id: 'test-session', cwd: process.cwd(), ...extra };
25+
}
26+
27+
function expectSilentPassthrough(result) {
28+
assert.equal(result.exitCode, 0, `auto-orchestrate exited ${result.exitCode}; stderr: ${result.stderr}`);
29+
const parsed = parseHookOutput(result.stdout);
30+
assert.ok(parsed, `expected JSON output; got: ${result.stdout}`);
31+
assert.equal(parsed.continue, true);
32+
assert.equal(parsed.hookSpecificOutput, undefined,
33+
`silent pass-through should not emit hookSpecificOutput; got: ${JSON.stringify(parsed.hookSpecificOutput)}`);
34+
}
35+
36+
function expectGuidanceInjected(result) {
37+
assert.equal(result.exitCode, 0);
38+
const parsed = parseHookOutput(result.stdout);
39+
assert.ok(parsed);
40+
assert.equal(parsed.continue, true);
41+
assert.ok(parsed.hookSpecificOutput, 'expected hookSpecificOutput on trigger match');
42+
assert.equal(parsed.hookSpecificOutput.hookEventName, 'UserPromptSubmit');
43+
assert.equal(typeof parsed.hookSpecificOutput.additionalContext, 'string');
44+
assert.ok(parsed.hookSpecificOutput.additionalContext.length > 10,
45+
'guidance string should be non-trivial');
46+
return parsed.hookSpecificOutput.additionalContext;
47+
}
48+
49+
describe('auto-orchestrate: trigger patterns', () => {
50+
test('triggers on Czech "implementuj novou feature"', () => {
51+
const r = runHook('auto-orchestrate', userPrompt('implementuj novou feature pro správu uživatelů'));
52+
expectGuidanceInjected(r);
53+
});
54+
55+
test('triggers on English "implement a new endpoint"', () => {
56+
const r = runHook('auto-orchestrate', userPrompt('implement a new endpoint for the users API'));
57+
expectGuidanceInjected(r);
58+
});
59+
60+
test('triggers on "build new component"', () => {
61+
const r = runHook('auto-orchestrate', userPrompt('build a new component for the dashboard'));
62+
expectGuidanceInjected(r);
63+
});
64+
65+
test('triggers on "wire up payments"', () => {
66+
const r = runHook('auto-orchestrate', userPrompt('wire up payments via Stripe'));
67+
expectGuidanceInjected(r);
68+
});
69+
70+
test('triggers on "vytvoř novou stránku"', () => {
71+
const r = runHook('auto-orchestrate', userPrompt('vytvoř novou stránku pro registraci'));
72+
expectGuidanceInjected(r);
73+
});
74+
});
75+
76+
describe('auto-orchestrate: skip patterns', () => {
77+
test('skips "fix typo"', () => {
78+
const r = runHook('auto-orchestrate', userPrompt('fix typo in README — implement → implements'));
79+
expectSilentPassthrough(r);
80+
});
81+
82+
test('skips "quick fix"', () => {
83+
const r = runHook('auto-orchestrate', userPrompt('quick fix the broken link'));
84+
expectSilentPassthrough(r);
85+
});
86+
87+
test('skips "skip research"', () => {
88+
const r = runHook('auto-orchestrate', userPrompt('skip research and just implement the feature'));
89+
expectSilentPassthrough(r);
90+
});
91+
92+
test('silent on simple greeting', () => {
93+
const r = runHook('auto-orchestrate', userPrompt('ahoj, jak se máš?'));
94+
expectSilentPassthrough(r);
95+
});
96+
97+
test('silent on simple question', () => {
98+
const r = runHook('auto-orchestrate', userPrompt('what does this function do?'));
99+
expectSilentPassthrough(r);
100+
});
101+
});
102+
103+
describe('auto-orchestrate: fail-open contract', () => {
104+
test('exit 0 on malformed JSON input', () => {
105+
const r = runHook('auto-orchestrate', '{ malformed');
106+
assert.equal(r.exitCode, 0);
107+
const parsed = parseHookOutput(r.stdout);
108+
assert.ok(parsed);
109+
assert.equal(parsed.continue, true);
110+
});
111+
112+
test('exit 0 on empty stdin', () => {
113+
const r = runHook('auto-orchestrate', '');
114+
assert.equal(r.exitCode, 0);
115+
const parsed = parseHookOutput(r.stdout);
116+
assert.ok(parsed);
117+
assert.equal(parsed.continue, true);
118+
});
119+
120+
test('exit 0 on missing prompt field', () => {
121+
const r = runHook('auto-orchestrate', { session_id: 'test', cwd: '/tmp' });
122+
assert.equal(r.exitCode, 0);
123+
expectSilentPassthrough(r);
124+
});
125+
126+
test('respects 5s timeout', () => {
127+
const r = runHook('auto-orchestrate', userPrompt('hello'), { timeout: 5000 });
128+
assert.equal(r.timedOut, false);
129+
});
130+
131+
test('CORTEX_BUDGET_DISABLED suppresses budget guidance', () => {
132+
// When budget is disabled, guidance should still emit but without USD figures
133+
const r = runHook(
134+
'auto-orchestrate',
135+
userPrompt('implement a new feature'),
136+
{ env: { CORTEX_BUDGET_DISABLED: '1' } }
137+
);
138+
const ctx = expectGuidanceInjected(r);
139+
assert.ok(!/\$\d+\.\d{2}/.test(ctx),
140+
`with CORTEX_BUDGET_DISABLED=1, guidance should not contain dollar figures; got: ${ctx.slice(0, 300)}`);
141+
});
142+
});
143+
144+
describe('auto-orchestrate: PII / Dave-path leak guard', () => {
145+
test('output does not leak Dave-specific paths', () => {
146+
const r = runHook('auto-orchestrate', userPrompt('implement a new feature'));
147+
const stdout = r.stdout.toLowerCase();
148+
assert.ok(
149+
!stdout.includes('/c/users/david/') && !stdout.includes('c:\\users\\david\\'),
150+
`auto-orchestrate hook leaked Dave-specific path; output:\n${r.stdout}`
151+
);
152+
});
153+
});

0 commit comments

Comments
 (0)