Skip to content

Commit 5425f53

Browse files
sahilsatralkarsahilsatralkar
committed
Merge remote-tracking branch 'origin/main' into fix/issue-62395-tui-streaming-gap-reset
# Conflicts: # src/tui/tui-event-handlers.test.ts # src/tui/tui.ts
2 parents 2040974 + 6370013 commit 5425f53

1,447 files changed

Lines changed: 68226 additions & 20997 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.agents/skills/blacksmith-testbox/SKILL.md‎

Lines changed: 340 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,340 @@
1+
---
2+
name: blacksmith-testbox
3+
description: >
4+
Validate code changes against real CI when local execution is not
5+
enough. Use for CI-parity checks, secrets/services, migrations, or
6+
builds/tests that cannot run reliably on the local machine. Do not
7+
replace repo-documented local test/build loops just because this
8+
skill exists.
9+
---
10+
11+
# Blacksmith Testbox
12+
13+
## Scope
14+
15+
Use Testbox when you need remote CI parity, injected secrets, hosted services,
16+
or an OS/runtime image that your local machine cannot provide cheaply.
17+
18+
Do not default to Testbox for every local test/build loop. If the repo has
19+
documented local commands for normal iteration, use those first so you keep
20+
warm caches, local build state, and fast feedback.
21+
22+
Testbox is the expensive path. Reach for it deliberately.
23+
24+
## Install the CLI
25+
26+
If `blacksmith` is not installed, install it:
27+
28+
curl -fsSL https://get.blacksmith.sh | sh
29+
30+
For the canary channel (bleeding-edge):
31+
32+
BLACKSMITH_CHANNEL=canary sh -c 'curl -fsSL https://get.blacksmith.sh | sh'
33+
34+
Then authenticate:
35+
36+
blacksmith auth login
37+
38+
## Agent-triggered browser auth (non-interactive)
39+
40+
When an agent needs to ensure the user is authenticated before running testbox
41+
commands (e.g. warmup, run), use browser-based auth with non-interactive mode.
42+
This opens the browser for the user to sign in; the agent does not interact with
43+
the browser. The org selector in the dashboard is skipped, so the user only sees
44+
the sign-in flow.
45+
46+
**Required command** (`--organization` is required with `--non-interactive`):
47+
48+
blacksmith auth login --non-interactive --organization <org-slug>
49+
50+
The org slug can come from `BLACKSMITH_ORG` env var or the `--org` global flag.
51+
If neither is set, the agent should use the project's known org (e.g. from repo
52+
config or user context). Example:
53+
54+
blacksmith auth login --non-interactive --organization acme-corp
55+
blacksmith --org acme-corp auth login --non-interactive --organization acme-corp
56+
57+
**Flow**: The CLI starts a local callback server, opens the browser to the
58+
dashboard auth page, and blocks for up to 2 minutes. The user completes sign-in
59+
and authorization in the browser. The dashboard redirects to localhost with the
60+
token; the CLI saves credentials and exits. The agent then proceeds.
61+
62+
**Do not use** `--api-token` for this flow — that is for headless/token-based
63+
auth. This skill focuses on browser-based auth when the user prefers signing in
64+
via the web UI.
65+
66+
Optional flags:
67+
68+
- `--dashboard-url <url>` — Override dashboard URL (e.g. for staging)
69+
70+
## Decide first: local or Testbox
71+
72+
Before warming anything up, check the repo's own instructions.
73+
74+
Prefer local commands when:
75+
76+
- the repo documents a supported local test/build workflow
77+
- you are iterating on unit tests, lint, typecheck, formatting, or other
78+
local-only validation
79+
- the value comes from warm local caches and fast repeat runs
80+
- the command does not need remote secrets, hosted services, or CI-only images
81+
82+
Prefer Testbox when:
83+
84+
- the repo explicitly requires CI-parity or remote validation
85+
- the command needs secrets, service containers, or provisioned infra
86+
- you are reproducing CI-only failures
87+
- you need the exact workflow image/job environment from GitHub Actions
88+
89+
For OpenClaw specifically, normal local iteration should stay local:
90+
91+
- `pnpm check:changed`
92+
- `pnpm test:changed`
93+
- `pnpm test <path-or-filter>`
94+
- `pnpm test:serial`
95+
- `pnpm build`
96+
97+
Only use Testbox in OpenClaw when the user explicitly wants CI-parity or the
98+
check truly depends on remote secrets/services that the local repo loop cannot
99+
provide.
100+
101+
## Setup: Warmup before coding
102+
103+
If you decided Testbox is actually warranted, warm one up early. This returns
104+
an ID instantly and boots the CI environment in the background while you work:
105+
106+
blacksmith testbox warmup ci-check-testbox.yml
107+
# → tbx_01jkz5b3t9...
108+
109+
Save this ID. You need it for every `run` command.
110+
111+
Warmup dispatches a GitHub Actions workflow that provisions a VM with the
112+
full CI environment: dependencies installed, services started, secrets
113+
injected, and a clean checkout of the repo at the default branch.
114+
115+
Options:
116+
117+
--ref <branch> Git ref to dispatch against (default: repo's default branch)
118+
--job <name> Specific job within the workflow (if it has multiple)
119+
--idle-timeout <min> Idle timeout in minutes (default: 30)
120+
121+
## CRITICAL: Always run from the repo root
122+
123+
ALWAYS invoke `blacksmith testbox` commands from the **root of the git
124+
repository**. The CLI syncs the current working directory to the testbox
125+
using rsync with `--delete`. If you run from a subdirectory (e.g.
126+
`cd backend && blacksmith testbox run ...`), rsync will mirror only that
127+
subdirectory and **delete everything else** on the testbox — wiping other
128+
directories like `dashboard/`, `cli/`, etc.
129+
130+
# CORRECT — run from repo root, use paths in the command
131+
blacksmith testbox run --id <ID> "cd backend && php artisan test"
132+
blacksmith testbox run --id <ID> "cd dashboard && npm test"
133+
134+
# WRONG — do NOT cd into a subdirectory before invoking the CLI
135+
cd backend && blacksmith testbox run --id <ID> "php artisan test"
136+
137+
If your shell is in a subdirectory, `cd` back to the repo root first:
138+
139+
cd "$(git rev-parse --show-toplevel)"
140+
blacksmith testbox run --id <ID> "cd backend && php artisan test"
141+
142+
## Running commands
143+
144+
blacksmith testbox run --id <ID> "<command>"
145+
146+
The `run` command automatically waits for the testbox to become ready if
147+
it is still booting, so you can call `run` immediately after warmup without
148+
needing to check status first.
149+
150+
## Downloading files from a testbox
151+
152+
Use the `download` command to retrieve files or directories from a running
153+
testbox to your local machine. This is useful for fetching build artifacts,
154+
test results, coverage reports, or any output generated on the testbox.
155+
156+
blacksmith testbox download --id <ID> <remote-path> [local-path]
157+
158+
The remote path is relative to the testbox working directory (same as `run`).
159+
If no local path is specified, the file is saved to the current directory
160+
using the same base name.
161+
162+
To download a directory, append a trailing `/` to the remote path — this
163+
triggers recursive mode:
164+
165+
# Download a single file
166+
blacksmith testbox download --id <ID> coverage/report.html
167+
168+
# Download a file to a specific local path
169+
blacksmith testbox download --id <ID> build/output.tar.gz ./output.tar.gz
170+
171+
# Download an entire directory
172+
blacksmith testbox download --id <ID> test-results/ ./results/
173+
174+
Options:
175+
176+
--ssh-private-key <path> Path to SSH private key (if warmup used --ssh-public-key)
177+
178+
## How file sync works
179+
180+
Understanding this model is critical for using Testbox correctly.
181+
182+
When you call `run`, the CLI performs a **delta sync** of your local changes
183+
to the remote testbox before executing your command:
184+
185+
1. The testbox VM starts from a clean `actions/checkout` at the warmup ref.
186+
The workflow's setup steps (e.g. `npm install`, `pip install`, `composer install`)
187+
run during warmup and populate dependency directories on the remote VM.
188+
189+
2. On each `run`, the CLI uses **git** to detect which files changed locally
190+
since the last sync. It syncs ONLY tracked files and untracked non-ignored
191+
files (i.e. files that `git ls-files` reports).
192+
193+
3. **`.gitignore`'d directories are never synced.** This means directories
194+
like `node_modules/`, `vendor/`, `.venv/`, `build/`, `dist/`, etc. are
195+
NOT transferred from your local machine. The testbox uses its own copies
196+
of those directories, populated during the warmup workflow steps.
197+
198+
4. If nothing has changed since the last sync (same git commit and working
199+
tree state), the sync is skipped entirely for speed.
200+
201+
### Why this matters
202+
203+
- **Changing dependencies**: If you modify `package.json`, `requirements.txt`,
204+
`composer.json`, `go.mod`, or similar dependency manifests, the lock/manifest
205+
file will be synced but the actual dependency directory will NOT. You must
206+
re-run the install command on the testbox:
207+
208+
blacksmith testbox run --id <ID> "npm install && npm test"
209+
blacksmith testbox run --id <ID> "pip install -r requirements.txt && pytest"
210+
blacksmith testbox run --id <ID> "composer install && phpunit"
211+
212+
- **Generated/build artifacts**: If your tests depend on a build step (e.g.
213+
`npm run build`, `make`), and you changed source files that affect the build
214+
output, re-run the build on the testbox before testing.
215+
216+
- **New untracked files**: New files you create locally ARE synced (as long as
217+
they are not gitignored). You do not need to `git add` them first.
218+
219+
- **Deleted files**: Files you delete locally are also deleted on the remote
220+
testbox. The sync model keeps the remote in lockstep with your local managed
221+
file set.
222+
223+
## CRITICAL: Do not ban local tests
224+
225+
Do not assume local validation is forbidden. Many repos intentionally invest in
226+
fast, warm local loops, and forcing every run through Testbox destroys that
227+
advantage.
228+
229+
Use Testbox for the checks that actually need it: remote parity, secrets,
230+
services, CI-only runners, or reproducibility against the workflow image.
231+
232+
If the repo says local tests/builds are the normal path, follow the repo.
233+
234+
## When to use
235+
236+
Use Testbox when:
237+
238+
- running database migrations or destructive environment checks
239+
- running commands that depend on secrets or environment variables not present locally
240+
- reproducing CI-only failures or validating against the workflow image
241+
- validating behavior that needs provisioned services or remote runners
242+
- doing a final parity check before commit/push when the repo or user wants that
243+
244+
Trim that list based on repo guidance. If the repo documents supported local
245+
tests/builds, prefer local for routine iteration and keep Testbox for the
246+
checks that need parity or remote state.
247+
248+
## Workflow
249+
250+
1. Decide whether the repo's local loop is the right default.
251+
2. Only if Testbox is warranted, warm up early:
252+
`blacksmith testbox warmup ci-check-testbox.yml` → save the ID
253+
3. Write code while the testbox boots in the background.
254+
4. Run the remote command when needed:
255+
`blacksmith testbox run --id <ID> "npm test"`
256+
5. If tests fail, fix code and re-run against the same warm box.
257+
6. If you changed dependency manifests (package.json, etc.), prepend
258+
the install command: `blacksmith testbox run --id <ID> "npm install && npm test"`
259+
7. If you need artifacts (coverage reports, build outputs, etc.), download them:
260+
`blacksmith testbox download --id <ID> coverage/ ./coverage/`
261+
8. Once green, commit and push.
262+
263+
## OpenClaw full test suite
264+
265+
For OpenClaw, use the repo package manager and the measured stable full-suite
266+
profile below. It keeps six Vitest project shards active while limiting each
267+
shard to one worker to avoid worker OOMs on Testbox:
268+
269+
blacksmith testbox run --id <ID> "env NODE_OPTIONS=--max-old-space-size=4096 OPENCLAW_TEST_PROJECTS_PARALLEL=6 OPENCLAW_VITEST_MAX_WORKERS=1 pnpm test"
270+
271+
Observed full-suite time on Blacksmith Testbox is about 3-4 minutes:
272+
273+
- 173-180s on a warmed box
274+
- 219s on a fresh 32-vCPU box
275+
276+
When validating before commit/push, run `pnpm check:changed` first when
277+
appropriate, then the full suite with the profile above if broad confidence is
278+
needed.
279+
280+
## Examples
281+
282+
blacksmith testbox warmup ci-check-testbox.yml
283+
# → tbx_01jkz5b3t9...
284+
285+
# Run tests
286+
blacksmith testbox run --id <ID> "npm test -- --testPathPattern=handler.test"
287+
blacksmith testbox run --id <ID> "go test ./pkg/api/... -run TestHandler -v"
288+
blacksmith testbox run --id <ID> "python -m pytest tests/test_api.py -k test_auth"
289+
290+
# Re-install deps after changing package.json, then test
291+
blacksmith testbox run --id <ID> "npm install && npm test"
292+
293+
# Build and test
294+
blacksmith testbox run --id <ID> "npm run build && npm test"
295+
296+
# Download artifacts from the testbox
297+
blacksmith testbox download --id <ID> coverage/lcov-report/ ./coverage/
298+
blacksmith testbox download --id <ID> build/output.tar.gz
299+
300+
## Waiting for the testbox to be ready
301+
302+
The `run` command automatically waits for the testbox, so explicit waiting is
303+
usually unnecessary. If you do need to check readiness separately (e.g. before
304+
a series of runs), use the `--wait` flag. Do NOT use a sleep-and-recheck loop.
305+
306+
Correct: block until ready with a timeout:
307+
308+
blacksmith testbox status --id <ID> --wait [--wait-timeout 5m]
309+
310+
Wrong: never use sleep + status in a loop:
311+
312+
# BAD — do not do this
313+
sleep 30 && blacksmith testbox status --id <ID>
314+
while ! blacksmith testbox status --id <ID> | grep ready; do sleep 5; done
315+
316+
`--wait` polls the status and exits as soon as the testbox is ready (or when the
317+
timeout is reached). Default timeout is 5m; use `--wait-timeout` for longer
318+
(e.g. `10m`, `1h`).
319+
320+
## Managing testboxes
321+
322+
# Check status of a specific testbox
323+
blacksmith testbox status --id <ID>
324+
325+
# List all active testboxes for the current repo
326+
blacksmith testbox list
327+
328+
# Stop a testbox when you're done (frees resources)
329+
blacksmith testbox stop --id <ID>
330+
331+
Testboxes automatically shut down after being idle (default: 30 minutes).
332+
If you need a longer session, increase the timeout at warmup time:
333+
334+
blacksmith testbox warmup ci-check-testbox.yml --idle-timeout 60
335+
336+
## With options
337+
338+
blacksmith testbox warmup ci-check-testbox.yml --ref main
339+
blacksmith testbox warmup ci-check-testbox.yml --idle-timeout 60
340+
blacksmith testbox run --id <ID> "go test ./..."

‎.agents/skills/optimizetests/SKILL.md‎

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
---
2+
name: optimizetests
3+
description: Optimize OpenClaw test runtime end to end. Use when the user asks for /optimizetests, slow-test review, import optimization, deduping tests, moving misplaced core coverage to extensions, or reducing CI/test wall time without adding shards or dropping coverage.
4+
---
5+
6+
# Optimize Tests
7+
8+
Goal: real OpenClaw test/runtime speedups with coverage intact. Do not add shards,
9+
skip assertions, weaken gates, or tune runner flags as the main fix.
10+
11+
## Runbook
12+
13+
1. Read `docs/help/testing.md`, `docs/ci.md`, and the scoped `AGENTS.md` files
14+
for any subtree you will edit.
15+
2. Establish evidence before edits:
16+
- Full ranking: `pnpm test:perf:groups --full-suite --allow-failures --output .artifacts/test-perf/<name>.json`
17+
- Targeted file: `timeout 240 /usr/bin/time -l pnpm test <file> --maxWorkers=1 --reporter=verbose`
18+
- Import suspicion: add `OPENCLAW_VITEST_IMPORT_DURATIONS=1 OPENCLAW_VITEST_PRINT_IMPORT_BREAKDOWN=1`
19+
3. Attack highest-return hotspots first:
20+
- broad barrels or `importActual()` in hot tests
21+
- per-test `vi.resetModules()` plus fresh imports
22+
- expensive gateway/server/client setup where reset/reuse proves same behavior
23+
- core tests asserting extension-owned behavior
24+
- duplicated fixture construction or contract assertions
25+
4. Prefer production-quality fixes:
26+
- narrow runtime seams over broad mocks
27+
- pure helpers for static parsing/metadata
28+
- injected deps over module resets
29+
- extension-owned tests for bundled plugin/provider/channel behavior
30+
5. After each change, rerun the same benchmark and the proving test lane. Record
31+
before/after wall time, Vitest duration, and max RSS when available.
32+
6. Run `pnpm check:changed`; run broader gates (`pnpm check`, `pnpm test`,
33+
`pnpm build`) when touched surfaces require them.
34+
7. Commit scoped changes with `scripts/committer "<conventional message>" <paths...>`.
35+
Push when requested. If CI is red, inspect with `gh run list/view`, fix, push,
36+
repeat until current CI is green or a blocker is proven unrelated.
37+
38+
## Output
39+
40+
End with the pushed commit(s), before/after timings, gates run, current CI state,
41+
and any remaining tail lanes that need separate optimization.

‎.agents/skills/optimizetests/agents/openai.yaml‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
interface:
2+
display_name: "Optimize Tests"
3+
short_description: "Benchmark and speed up OpenClaw tests"
4+
default_prompt: "Use $optimizetests to benchmark slow OpenClaw tests, optimize imports and duplicated setup, move misplaced core coverage to extensions, verify gates, commit scoped changes, push, and keep CI green without adding shards or dropping coverage."
5+
policy:
6+
allow_implicit_invocation: false

0 commit comments

Comments
 (0)