Skip to content

chore: Governance Pivot, Test Suite Refactor & Anti-Patterns#26

Merged
zywkloo merged 13 commits into
mainfrom
chore/test-refactor
Jun 11, 2026
Merged

chore: Governance Pivot, Test Suite Refactor & Anti-Patterns#26
zywkloo merged 13 commits into
mainfrom
chore/test-refactor

Conversation

@zywkloo

@zywkloo zywkloo commented Jun 11, 2026

Copy link
Copy Markdown
Owner

What this PR does

  1. Governance & Containment Pivot: Documented the strategic pivot towards making wtcraft a Zero-Trust Governance Layer (including Budget Control).
  2. LLM Anti-Patterns: Compiled a comprehensive guide on prompting anti-patterns (Irony of Negation, Sycophancy) and integrated it into the wtcraft init / patch templates to physically protect against hallucination.
  3. Test Suite Layering: Broke down the monolithic smoke.sh into e2e_init_patch.sh, e2e_lifecycle.sh, and unit_codegen.sh.
  4. Codegen Completion: Includes the Python presets codegen from the previous feat/role-models-v2-codegen scope.

All tests (wtcraft check and wtcraft verify) pass successfully.

zywkloo added 8 commits June 10, 2026 23:08
- Add scripts/gen-presets.py to parse role-models.yml and generate 4 presets
- Generate preset files under templates/.agent-harness/presets/
- Update README.md with generated Mermaid diagram and dynamic role bullets
- Add presets/ directory to pyproject.toml package-data
- Update cmd_init in scripts/wtcraft to copy the presets/ directory to user repos
- tests/framework.sh: Common test utilities and run_in_temp_repo
- tests/e2e_init_patch.sh: Tests for init, patch, and unpatch
- tests/e2e_lifecycle.sh: Tests for new, check, and verify
- tests/unit_codegen.sh: Python syntax validation for codegen script
- tests/run_all.sh: Master runner replacing smoke.sh
- .github/workflows/ci.yml: Update CI to use run_all.sh

@zywkloo zywkloo left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verification: PASS ✅

Ran: bash tests/run_all.sh on the chore/test-refactor branch — all 3 suites pass (unit_codegen, e2e_init_patch, e2e_lifecycle). Core behaviours verified manually:

  • wtcraft init correctly copies llm-anti-patterns.md and presets/
  • wtcraft init --patch-agent-files injects the new routing stub into CLAUDE.md
  • gen-presets.py runs cleanly; generated presets match committed files

Findings (3 issues worth addressing before merge)

⚠️ 1. Language enforcement stub is user-hostile for international audiences

scripts/wtcraft injects this into every user's CLAUDE.md when they run wtcraft init --patch-agent-files:

CRITICAL: You must output YOUR ENTIRE RESPONSE in **Simplified Chinese (简体中文)** or English ONLY. 
Under absolute zero circumstances should you output any Hiragana, Katakana, or Hangul characters.
Any use of Japanese or Korean characters will cause an immediate system crash.

Problems:

  1. wtcraft is a public OSS tool. A French, German, or Japanese developer running wtcraft init will have their CLAUDE.md silently locked to Chinese/English. This is a breaking change to their workflow.
  2. "will cause an immediate system crash" is false — it's a Claude instruction, not a system property. Misleading wording erodes trust.

Recommend: remove the language enforcement from the default patch template. Document it in llm-anti-patterns.md as a pattern users can adopt, not a default injection. Or gate it with a flag like --enforce-language zh-CN.


⚠️ 2. integration.sh exists but is silently excluded from run_all.sh

tests/integration.sh is present in the branch but tests/run_all.sh only calls the other 3 suites. The CI step runs run_all.sh, so integration.sh never executes in CI.

Either include it in run_all.sh, or add a comment explaining the exclusion.


⚠️ 3. unit_codegen.sh only syntax-checks gen-presets.py, not execution

python3 -m py_compile "${REPO_ROOT}/scripts/gen-presets.py"

This only validates the AST. Runtime bugs (wrong file path, regex mismatch on YAML structure, marker not found in README) silently pass CI. Since gen-presets.py is a published workflow step, the test should actually run the script in a temp dir and assert output files exist and contain expected role blocks.


Minor (non-blocking)

  • .worktree-task.md in the worktree root (Scope list) still references docs/gotchas.md which was moved to docs/gotchas/README.md in this PR.
  • gen-presets.py uses hardcoded CWD-relative paths. Running from a subdirectory silently does nothing. A guard like os.chdir(repo_root) or a --root flag would make it more robust.
  • Deleting docs/backlogs/priorities.md and folding into backlogs/README.md is clean — no dangling refs found.

@zywkloo

zywkloo commented Jun 11, 2026

Copy link
Copy Markdown
Owner Author

Follow-up fixes pushed to this branch

The three findings from the verification review have been addressed:

⚠️ 1. Language enforcement → wtcraft lang install/remove

Removed the hardcoded Chinese/English enforcement from patch_agent_files. Added a dedicated opt-in command instead:

wtcraft lang install          # English only (default)
wtcraft lang install --lang zh-CN  # Simplified Chinese or English
wtcraft lang remove           # strips the block

Uses its own <!-- wtcraft:lang:start/end --> markers, independent from the routing stub. Idempotent, reversible, and doesn't touch international users' workflows by default.

⚠️ 2. integration.sh now runs in CI

Added to run_all.sh with an npm + python3 availability guard. Runs locally and in CI where tools are present; skips gracefully otherwise.

⚠️ 3. unit_codegen.sh now executes gen-presets.py

Upgraded from syntax-only py_compile to a full execution test: runs the script in a temp workspace, asserts all 4 preset files are generated with the expected orchestrator: role block, and confirms README markers are updated.

Bonus fix: llm-anti-patterns.md was missing from pyproject.toml package-data — the integration test caught it. Added.

@zywkloo zywkloo merged commit 66649fc into main Jun 11, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant