Add automated documentation validation to CI#734
Conversation
Implements a Python script that validates documentation sync between code and docs by checking: - CLI flags are documented in docs/src/cli/ - Cheatcodes are in interface at docs/src/cheatcodes/cheatcodes_overview.md - Config fields (with json tags) are documented in docs/src/project_configuration/ Features: - Heuristic-based exclusions (no manual allowlist) - Automatically excludes nested config structs - Strips JSON tag modifiers like ,omitempty - Simple regex-based parsing for maintainability Adds docs-check job to CI workflow that runs on PRs and master. Currently detects 6 documentation gaps including the missing 'label' cheatcode. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| all-checks: | ||
| if: always() | ||
| needs: [lint, test, build, release] | ||
| needs: [lint, test, build, release, docs-check] | ||
|
|
There was a problem hiding this comment.
The new docs-check job is wired into all-checks with no allowed skips, but scripts/check_docs.py currently exits non‑zero on the existing tree (e.g., missing docs for --rpc-url, --verbosity, --rpc-block, label, coverageExclusions, contractAddressOverrides). With this change, every PR and the default branch will go red until those docs are added, effectively breaking CI rather than surfacing warnings.
Useful? React with 👍 / 👎.
Documentation fixes: - Add label cheatcode to interface in cheatcodes_overview.md - Document --verbosity long form in fuzz.md CLI docs - Add --rpc-url and --rpc-block flag documentation - Document coverageExclusions config field Script improvements: - Exclude fields with ,omitempty tag from validation (optional/internal fields) - These fields are not required in user-facing documentation All validation checks now pass (31 cheatcodes, 21 CLI flags, 44 config fields). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>
- Create docs/src/cheatcodes/label.md with description, example, and signature - Add label entry to SUMMARY.md navigation The label cheatcode sets a human-readable label for an address, making it easier to identify specific contracts or accounts in execution traces and error messages during testing. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>
1 participant
Summary
Adds automated documentation validation to prevent drift between code and documentation during PR reviews.
Changes
New script:
scripts/check_docs.pyvalidates:docs/src/cli/*.mddocs/src/cheatcodes/cheatcodes_overview.mdjsontags) are documented indocs/src/project_configuration/*.mdCI integration: Added
docs-checkjob to.github/workflows/ci.ymlall-checksgateValidation Strategy
,omitemptyCurrent Status
The script currently detects 6 documentation gaps:
labelcheatcode in interface (known issue)--verbosity,--rpc-url,--rpc-blockcoverageExclusions,contractAddressOverridesThese can be fixed in follow-up PRs. The goal of this PR is to establish the validation infrastructure to prevent future drift.
Test Plan
labelcheatcode🤖 Generated with Claude Code