Skip to content

Commit 53e4be5

Browse files
thymikeeclaude
andauthored
Remove SkillGym suite and repo-health snapshot infrastructure (#1480)
* chore: drop SkillGym and the repo-health aggregator (#1412 descope) Remove the SkillGym harness (test/skillgym/), its check-affected lane, package scripts, and devDependency — the help-conformance bench is now the single non-gating small-model oracle. skills/ markdown classifies as docs in the affected-check selector instead of failing open. Remove scripts/repo-health: its only gating assertion duplicated the Layering Guard job, its case-count metric imported the deleted SkillGym suite, and its sole planned consumer (#1424 / PR #1477) was closed with the Track C descope on #1412. Verified: check-affected node --test suites, oxfmt, oxlint, tsc, check:layering, fallow audit vs origin/main, and the full unit suite (unit-core + subprocess-stub) all pass. Co-Authored-By: Claude Fable 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01FUv7bvbWNryuXgSBuqTtep * fix(scripts): fold slow-test budgets into the reporter for production-exports The Fallow production-exports gate flagged all three budget exports: their in-file consumer (SLOW_TEST_RATCHET) and the repo-health entry point that kept the module reachable were both removed in the descope, leaving the config-loaded reporter as the only consumer — invisible to --production analysis. The data-only module's second consumer is gone, so per the boundaries-are-earned norm the constants move into the reporter instead of gaining a suppression. Co-Authored-By: Claude Fable 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01FUv7bvbWNryuXgSBuqTtep * docs: align skills/ format policy and purge last SkillGym mention Address both P2 review findings on #1480: the testing-matrix row and the selector's formatGate both still claimed oxfmt covers skills/, while selectChecks classifies skills/*.md docs-only (oxfmt ignores **/*.md, so the claim was a no-op even before). The matrix now states the docs-only policy and formatGate drops the dead underSkills fact. The merged examples/README.md index (from #1469) loses its skillgym mention. Co-Authored-By: Claude Fable 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01FUv7bvbWNryuXgSBuqTtep --------- Co-authored-by: Claude <[email protected]>
1 parent 6ab5963 commit 53e4be5

35 files changed

Lines changed: 187 additions & 2418 deletions

‎.fallowrc.json‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -64,7 +64,7 @@
6464
},
6565
{
6666
"comment": "Tool config default exports, loaded by the tool rather than imported.",
67-
"file": "{tsdown.config.ts,vitest.mutation.config.ts,website/rspress.config.ts,test/skillgym/skillgym.config.ts,test/skillgym/suites/*.ts}",
67+
"file": "{tsdown.config.ts,vitest.mutation.config.ts,website/rspress.config.ts}",
6868
"exports": ["default"]
6969
},
7070
{
@@ -88,7 +88,7 @@
8888
"exports": ["ownedTestFiles"]
8989
},
9090
{
91-
"comment": "Help-benchmark conformance seams: helpTopicIds feeds the topic-coverage gate and PRIVATE_AX_RECOVERY_SAMPLE feeds the skillgym suite; both consumers are test-tree files outside --production analysis.",
91+
"comment": "Help-benchmark conformance seams: helpTopicIds feeds the topic-coverage gate and PRIVATE_AX_RECOVERY_SAMPLE feeds the bench sample producers; both consumers are test-tree files outside --production analysis.",
9292
"file": "{src/cli/parser/cli-help.ts,scripts/help-conformance-sample-outputs.mjs}",
9393
"exports": ["helpTopicIds", "PRIVATE_AX_RECOVERY_SAMPLE"]
9494
},

‎.gitignore‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -35,7 +35,6 @@ xcuserdata/
3535
*.app
3636
*.xctestrun
3737
*.xcarchive
38-
.skillgym-results/
3938
android/snapshot-helper/build/
4039
android/snapshot-helper/dist/
4140
android/ime-helper/build/

‎.oxfmtrc.json‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,6 @@
77
"ignorePatterns": [
88
"dist/**",
99
"node_modules/**",
10-
"**/.skillgym-results/**",
1110
"**/*.md",
1211
"scripts/maestro-conformance/corpus/**",
1312
"fallow-baselines/**",

‎AGENTS.md‎

Lines changed: 2 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -233,10 +233,8 @@ connect errors, retry policy, or command typing, start in
233233
layer after writing is how the same contract ends up duplicated across two of them.
234234
`docs/agents/cli-flags.md` walks the layers for the flag case.
235235
- Decide docs impact with the change, not after. For behavior/CLI-surface changes: update
236-
help/metadata, README or `website/docs/**` when user-facing, and a SkillGym case in
237-
`test/skillgym/suites/agent-device-smoke-suite.ts` when command-planning guidance changes.
238-
- Keep SkillGym cases behavioral and command-planning oriented: assert the user-visible contract and
239-
expected command family, forbid known bad patterns, avoid brittle exact output.
236+
help/metadata, README or `website/docs/**` when user-facing, and a help-conformance bench case
237+
(`scripts/help-conformance-*.mjs`) when command-planning guidance changes.
240238
- State in the final summary whether docs/skills were updated, and why not if they weren't.
241239

242240
When guidance conflicts, Hard Rules win, then scope, then testing, then style.

‎docs/agents/pull-requests.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@
1414
observes the artifact/state the feature claims to handle. If live verification is blocked, state
1515
the blocker and the exact command/device needed, and downgrade the PR to residual risk rather than
1616
calling it ready.
17-
- Command-surface changes preserve CLI, Node.js, daemon, MCP, help, docs, and SkillGym coverage
17+
- Command-surface changes preserve CLI, Node.js, daemon, MCP, help, and docs coverage
1818
where that surface is affected, without duplicating command contracts across layers.
1919
- Runtime output stays agent-friendly: compact defaults, top offenders first for diagnostics/perf,
2020
bounded arrays in JSON, artifact paths for large raw data, progressive lookup for deeper detail.

‎docs/agents/testing.md‎

Lines changed: 3 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -18,9 +18,8 @@ The mapping it encodes, for when you need to run a gate directly or reason about
1818
| iOS runner / Swift | `pnpm build:xcuitest` |
1919
| CLI help/guidance (`src/cli/parser/cli-help.ts`, `src/cli-schema/`) | `pnpm exec vitest run src/cli/parser/__tests__ src/cli-schema/command-schema-guards.test.ts scripts/__tests__` — the `scripts/__tests__` gates enforce help-topic benchmark coverage and pin the bench's quoted CLI samples to the real renderers |
2020
| Help benchmark cases (`scripts/help-conformance-*.mjs`) | `pnpm exec vitest run scripts/__tests__` (deterministic gates); model-backed: `pnpm bench:help-conformance` (paid LLM calls, local only) |
21-
| SkillGym prompts/assertions | `pnpm test:skillgym:case <case-id>` (broad: `pnpm test:skillgym`, filter with `-- --tag fixture-smoke` or `-- --tag skill-guidance`) — agentic routing + local-help-consumption proof only; command-planning knowledge checks belong in the help bench |
2221
| `.ad` grammar (`src/replay/script.ts`, gesture arity, replay vars) | `pnpm exec vitest run --project unit-core test/replay-compat` — the frozen replay-compat corpus asserts which released script surfaces still parse; a flipped verdict is edited in `test/replay-compat/manifest.ts`, never in the script. Adding or re-pinning a corpus entry also runs `pnpm check:replay-compat`, which re-derives each entry from its release tag in git history |
23-
| Anything in `src/`, `test/`, `skills/` | `pnpm format` |
22+
| Anything in `src/`, `test/` | `pnpm format` (`skills/` is Markdown-only guidance: oxfmt ignores `**/*.md`, and the affected-check selector classifies it docs-only) |
2423
| A decision kernel or its tests (`src/kernel/errors.ts`, `src/daemon/ref-frame.ts`, `src/commands/interaction/runtime/settle.ts`, `src/utils/scroll-edge-state.ts`, `src/selectors/`) | `pnpm mutation:affected --base origin/main` (minutes; GitHub runs it per PR — see the mutation ratchet section) |
2524

2625
Two traps worth naming:
@@ -88,8 +87,8 @@ hand-maintained path map:
8887
import ownership. Dynamic-import relationships remain outside Vitest's
8988
analysis; GitHub's authoritative full suites still cover that boundary.
9089
- **Non-Vitest suites** retain explicit ownership. Root
91-
`test/integration/*.ts` files use the Node integration lane, SkillGym owns its
92-
harness and skill guidance, and platform/build tools keep their native gates.
90+
`test/integration/*.ts` files use the Node integration lane, and
91+
platform/build tools keep their native gates.
9392
- **Always-on gates** (`lint`, `typecheck`, `layering`, `fallow`, `format`) fire
9493
for their input categories and are never silently skipped. Platform source
9594
also selects the provider-integration and coverage gates required by the
@@ -99,9 +98,6 @@ hand-maintained path map:
9998
- A **small explicit build-ownership layer** covers the paths whose owning build
10099
cannot be derived: Swift runner, Android helpers, macOS helper, MCP metadata,
101100
and the public package surface (itself derived from `package.json` `exports`).
102-
- **SkillGym ownership** covers skill guidance (`skills/`) and the SkillGym
103-
harness (`test/skillgym/`) — those changes select the (local-only) SkillGym
104-
suite, and their Markdown is treated as skill/harness input, not inert docs.
105101

106102
Changed-file discovery folds working-tree state into the local plan: in the
107103
default local mode (`--head HEAD`) it unions the committed `base..HEAD` diff with

‎examples/README.md‎

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -33,6 +33,5 @@ way a Node consumer or the agent-device-cloud bridge would. Each file:
3333

3434
## test-app/
3535

36-
[`test-app/`](./test-app) is the Expo dogfood fixture used for `agent-device` and `skillgym`
37-
experiments (see its own [README](./test-app/README.md)) — it is a test fixture, not an SDK usage
38-
example.
36+
[`test-app/`](./test-app) is the Expo dogfood fixture used for `agent-device` experiments (see
37+
its own [README](./test-app/README.md)) — it is a test fixture, not an SDK usage example.

‎examples/test-app/README.md‎

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,12 @@
11
# Agent Device Tester
22

3-
`Agent Device Tester` is a minimal Expo Router fixture app for `agent-device` and `skillgym` experiments.
3+
`Agent Device Tester` is a minimal Expo Router fixture app for `agent-device` experiments.
44

55
It is intentionally small, but each surface is dense with durable accessibility targets so a few screens cover a large share of the workflows we care about.
66

77
## Why this app exists
88

99
- It gives `agent-device` a stable React Native target that we control.
10-
- It makes `skillgym` prompts concrete: the agent can inspect real app files instead of answering against an imagined UI.
1110
- It keeps the number of screens low while still covering roughly 50 practical interaction and verification cases.
1211

1312
## Screens

‎examples/test-app/src/screens/FormScreen.tsx‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -140,7 +140,7 @@ export function FormScreen(props: FormScreenProps) {
140140
title="Android IME capture"
141141
testID="android-ime-capture-fixture"
142142
>
143-
{/* SkillGym fixture: static diagnostic copy, not live state. */}
143+
{/* Fixture: static diagnostic copy, not live state. */}
144144
<TextField
145145
accessibilityLabel="Android IME target field"
146146
autoCapitalize="none"

‎fallow-baselines/health.json‎

Lines changed: 0 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -570,11 +570,6 @@
570570
"crap_moderate": {
571571
"count": 1
572572
}
573-
},
574-
"test/skillgym/suites/agent-device-smoke-suite.ts": {
575-
"crap_moderate": {
576-
"count": 1
577-
}
578573
}
579574
},
580575
"runtime_coverage_findings": [],

0 commit comments

Comments
 (0)