Skip to content

Commit 5e6609b

Browse files
steipetesteipete
committed
docs: require crabbox e2e bug proof
1 parent bed8f49 commit 5e6609b

2 files changed

Lines changed: 56 additions & 0 deletions

File tree

‎.agents/skills/crabbox/SKILL.md‎

Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -131,6 +131,55 @@ unclear:
131131
blacksmith testbox list
132132
```
133133

134+
## Efficient Bug E2E Verification
135+
136+
Use the smallest Crabbox lane that proves the reported user path, not just the
137+
touched code. Aim for one after-fix E2E proof before commenting, closing, or
138+
opening a PR for a user-visible bug.
139+
140+
Pick the lane by symptom:
141+
142+
- Docker/setup/install bug: build a package tarball and run the matching
143+
`scripts/e2e/*-docker.sh` or package script. This proves npm packaging,
144+
install paths, runtime deps, config writes, and container behavior.
145+
- Provider/model/auth bug: use a live lane when a `.profile`/Testbox profile key
146+
is available; otherwise use the repo's mock provider lane and state clearly
147+
that live provider auth was not exercised.
148+
- Channel delivery bug: use the channel Docker/live lane when available; include
149+
setup, config, gateway start, send/receive or agent-turn proof, and redacted
150+
logs.
151+
- Gateway/session/tool bug: prefer an end-to-end CLI or Gateway RPC command that
152+
creates real state and inspects the resulting files/API output.
153+
- Pure parser/config bug: targeted tests may be enough, but still run a
154+
Crabbox command when OS, package, Docker, secrets, or service lifecycle could
155+
change behavior.
156+
157+
Efficient flow:
158+
159+
1. Reproduce or prove the pre-fix symptom when feasible. If the issue cannot be
160+
reproduced, capture the exact command and observed behavior instead.
161+
2. Patch locally and run narrow local tests for edit speed.
162+
3. Run one Crabbox E2E command that starts from the user-facing entrypoint:
163+
package install, Docker setup, onboarding, channel add, gateway start, or
164+
agent turn as appropriate.
165+
4. Record proof as: Testbox id, command, environment shape, redacted secret
166+
source, and copied success/failure output.
167+
5. If the issue says "cannot reproduce", ask for the missing config/log fields
168+
that would distinguish the tested path from the reporter's path.
169+
170+
Keep it efficient:
171+
172+
- Reuse existing E2E scripts and helper assertions before writing ad hoc shell.
173+
- Use one-shot Crabbox for a single proof; use a reusable Testbox only when
174+
several commands must share built images, installed packages, or live state.
175+
- Prefer `OPENCLAW_CURRENT_PACKAGE_TGZ` with Docker/package lanes when testing a
176+
candidate tarball; prefer the repo's package helper instead of direct source
177+
execution when the bug might be packaging/install related.
178+
- Keep secrets redacted. It is fine to report key presence, source, and length;
179+
never print secret values.
180+
- Include `--timing-json` on broad or flaky runs when command duration or sync
181+
behavior matters.
182+
134183
## Reuse And Keepalive
135184

136185
For most Blacksmith-backed Crabbox calls, one-shot is enough. Use reuse only

‎.agents/skills/openclaw-pr-maintainer/SKILL.md‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -132,13 +132,20 @@ Output only qualifying candidates, with: ref, surface, proof, cause, fix sketch,
132132
## Enforce the bug-fix evidence bar
133133

134134
- Never merge a bug-fix PR based only on issue text, PR text, or AI rationale.
135+
- Whenever feasible, use Crabbox (`$crabbox`) for end-to-end verification before
136+
commenting that a bug is unreproducible, closing an issue, or opening/landing
137+
a fix PR. Prefer a real packaged/Docker/live lane that exercises the reported
138+
user flow over unit-only proof.
135139
- Before landing, require:
136140
1. symptom evidence such as a repro, logs, or a failing test
137141
2. a verified root cause in code with file/line
138142
3. a fix that touches the implicated code path
139143
4. a regression test when feasible, or explicit manual verification plus a reason no test was added
140144
- If the claim is unsubstantiated or likely wrong, request evidence or changes instead of merging.
141145
- If the linked issue appears outdated or incorrect, correct triage first. Do not merge a speculative fix.
146+
- If Crabbox/E2E proof is blocked, say exactly why and use the closest available
147+
local, Docker, mocked, or targeted proof. Do not present unit tests as real
148+
behavior proof.
142149

143150
## Close low-signal manual PRs carefully
144151

0 commit comments

Comments
 (0)