chore: Governance Pivot, Test Suite Refactor & Anti-Patterns#26
Conversation
- Add scripts/gen-presets.py to parse role-models.yml and generate 4 presets - Generate preset files under templates/.agent-harness/presets/ - Update README.md with generated Mermaid diagram and dynamic role bullets - Add presets/ directory to pyproject.toml package-data - Update cmd_init in scripts/wtcraft to copy the presets/ directory to user repos
- tests/framework.sh: Common test utilities and run_in_temp_repo - tests/e2e_init_patch.sh: Tests for init, patch, and unpatch - tests/e2e_lifecycle.sh: Tests for new, check, and verify - tests/unit_codegen.sh: Python syntax validation for codegen script - tests/run_all.sh: Master runner replacing smoke.sh - .github/workflows/ci.yml: Update CI to use run_all.sh
zywkloo
left a comment
There was a problem hiding this comment.
Verification: PASS ✅
Ran: bash tests/run_all.sh on the chore/test-refactor branch — all 3 suites pass (unit_codegen, e2e_init_patch, e2e_lifecycle). Core behaviours verified manually:
wtcraft initcorrectly copiesllm-anti-patterns.mdandpresets/wtcraft init --patch-agent-filesinjects the new routing stub intoCLAUDE.mdgen-presets.pyruns cleanly; generated presets match committed files
Findings (3 issues worth addressing before merge)
scripts/wtcraft injects this into every user's CLAUDE.md when they run wtcraft init --patch-agent-files:
CRITICAL: You must output YOUR ENTIRE RESPONSE in **Simplified Chinese (简体中文)** or English ONLY.
Under absolute zero circumstances should you output any Hiragana, Katakana, or Hangul characters.
Any use of Japanese or Korean characters will cause an immediate system crash.
Problems:
- wtcraft is a public OSS tool. A French, German, or Japanese developer running
wtcraft initwill have their CLAUDE.md silently locked to Chinese/English. This is a breaking change to their workflow. - "will cause an immediate system crash" is false — it's a Claude instruction, not a system property. Misleading wording erodes trust.
Recommend: remove the language enforcement from the default patch template. Document it in llm-anti-patterns.md as a pattern users can adopt, not a default injection. Or gate it with a flag like --enforce-language zh-CN.
integration.sh exists but is silently excluded from run_all.sh
tests/integration.sh is present in the branch but tests/run_all.sh only calls the other 3 suites. The CI step runs run_all.sh, so integration.sh never executes in CI.
Either include it in run_all.sh, or add a comment explaining the exclusion.
unit_codegen.sh only syntax-checks gen-presets.py, not execution
python3 -m py_compile "${REPO_ROOT}/scripts/gen-presets.py"This only validates the AST. Runtime bugs (wrong file path, regex mismatch on YAML structure, marker not found in README) silently pass CI. Since gen-presets.py is a published workflow step, the test should actually run the script in a temp dir and assert output files exist and contain expected role blocks.
Minor (non-blocking)
.worktree-task.mdin the worktree root (Scope list) still referencesdocs/gotchas.mdwhich was moved todocs/gotchas/README.mdin this PR.gen-presets.pyuses hardcoded CWD-relative paths. Running from a subdirectory silently does nothing. A guard likeos.chdir(repo_root)or a--rootflag would make it more robust.- Deleting
docs/backlogs/priorities.mdand folding intobacklogs/README.mdis clean — no dangling refs found.
Follow-up fixes pushed to this branchThe three findings from the verification review have been addressed:
Removed the hardcoded Chinese/English enforcement from wtcraft lang install # English only (default)
wtcraft lang install --lang zh-CN # Simplified Chinese or English
wtcraft lang remove # strips the blockUses its own
Added to
Upgraded from syntax-only Bonus fix: |
What this PR does
wtcraft init/patchtemplates to physically protect against hallucination.smoke.shintoe2e_init_patch.sh,e2e_lifecycle.sh, andunit_codegen.sh.feat/role-models-v2-codegenscope.All tests (
wtcraft checkandwtcraft verify) pass successfully.