Skip to content

Commit ec24fc5

Browse files
authored
Merge branch 'main' into fix/embedding-batch-jsonl-utf8
2 parents a17e457 + cad4e39 commit ec24fc5

73 files changed

Lines changed: 1812 additions & 141 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

docs/.generated/plugin-sdk-api-baseline.sha256

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -136,7 +136,7 @@ e1b72e4c2d926d39d216df80775e0a5f44705d04f5040c01fe1043ad7a2c2fe4 module/heartbe
136136
21a233e9008f0701df1efbefe521a9ced45c61da6e297e788bdd86895d793128 module/image-generation-runtime
137137
f6e3c44e7d1090a97aca554a3c247219b8de78b3cb4399cac5efde8a0a6c1156 module/inbound-envelope
138138
d3102bd433eea32ca2a761030788fe18294aa34266f2fb2d01d74268cbf8fb54 module/inbound-reply-dispatch
139-
a5b9f46c1d3e5af2ffe9c68806c6de4309b1d2e83f66ff288b384eba4abcf97b module/infra-runtime
139+
30b0b311f2488a12bdfb3eceb70a14327c9dfa2833519f24666141a734f9e7d1 module/infra-runtime
140140
3d0431e2575abf67c7fd9e9763b08b7a46018e01c290ffea8d4af9874d3c3fc4 module/inline-image-data-url-runtime
141141
74fee62a94618d830a5282a7816a73b85570b2e4ce17b90078078734510511d7 module/interactive-runtime
142142
9353103e334009222e5ae4fe4a2992a1d00552735013a69f93f17f74f14eb9ee module/json-schema-runtime

docs/ci.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -70,7 +70,7 @@ replace the separate strict, App-owned test-merge check against current `main`.
7070
A later pending or failed rerun does not erase an earlier successful result for
7171
that unchanged head during the freshness window.
7272

73-
The default-branch ruleset requires the GitHub Actions-owned `openclaw/ci-gate` check without a repository-role bypass. The separate strict App-owned test-merge check still binds the head to current `main`. A maintainer therefore cannot manually merge a head whose selected CI lanes failed, as happened before this rule was enforced.
73+
The default-branch ruleset requires the GitHub Actions-owned `openclaw/ci-gate` check. Repository maintainers and admins have an audited break-glass bypass intended only for signed direct fast-forward landings; the organization ruleset still blocks deletion and non-fast-forward updates. Normal pull-request merges should continue to use the gate rather than bypass failed CI. The separate strict App-owned test-merge check still binds the head to current `main`.
7474

7575
GitHub may mark superseded pull-request jobs as `cancelled` when a newer head lands. Treat that as CI noise unless the newest run for the same PR is also failing. Canonical `main` runs are not canceled after admission; when merge traffic arrives, GitHub replaces only the older pending run with the newest tip. Matrix jobs use `fail-fast: false`, and `build-artifacts` reports embedded channel, core-support-boundary, and gateway-watch failures directly instead of queuing tiny verifier jobs. The automatic CI concurrency key is versioned (`CI-v7-*`) so a GitHub-side zombie in an old queue group cannot indefinitely block newer main runs. Manual full-suite runs use `CI-manual-v1-*` and do not cancel in-progress runs. The plugin-list startup-memory guard keeps a 350 MiB ceiling on self-hosted Blacksmith Linux and allows 425 MiB on GitHub-hosted Linux, whose RSS baseline is higher for the same built CLI.
7676

docs/cli/onboard.md

Lines changed: 11 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -46,6 +46,7 @@ openclaw onboard --import-from hermes --import-source ~/.hermes
4646
openclaw onboard --skip-bootstrap
4747
openclaw onboard recommendations --json
4848
openclaw onboard recommendations acknowledge
49+
openclaw onboard recommendations refresh
4950
openclaw onboard --mode remote --remote-url wss://gateway-host:18789
5051
```
5152

@@ -55,8 +56,9 @@ the first-run bootstrap. The command does not rescan installed apps or call a
5556
model. Its output contains only validated install IDs, source, and tier; it
5657
intentionally omits untrusted marketplace prose, model reasons, and local app
5758
labels. After the recommendation offer has been answered, the command returns
58-
an empty list and future onboarding runs skip the step entirely. A future
59-
explicit refresh command can clear that accepted state and run discovery again.
59+
an empty list and future onboarding runs skip the step entirely.
60+
`openclaw onboard recommendations refresh` clears the stored offer so the next
61+
onboarding run rescans installed apps and creates a new offer.
6062

6163
Fresh workspaces defer the recommendation choice to the bootstrap conversation.
6264
After that conversation handles the user's choices,
@@ -122,13 +124,13 @@ the same plan the conversational `openclaw setup` chat would apply on "yes" —
122124
then offers plugin and skill recommendations from installed apps; app names
123125
are matched through your configured model and ClawHub search, and the step can
124126
be disabled with [`wizard.appRecommendations`](/gateway/configuration-reference#wizard).
125-
On a macOS console, it then opens the authenticated Control UI dashboard and
126-
waits up to 60 seconds for the browser client to connect. Over SSH, it prints a
127-
prominent copy-pasteable dashboard URL, including an SSH port-forward command
128-
for a loopback Gateway, and waits up to five minutes. A successful connection
129-
continues in the browser; an unreachable Gateway or a timeout falls back to the
130-
same terminal hatch as before. Pass `--tui` to skip the browser handoff and
131-
force that terminal hatch. Other platforms currently keep the terminal hatch.
127+
In a macOS, Linux, or Windows desktop session, it then opens the authenticated
128+
Control UI dashboard and waits up to 60 seconds for the browser client to
129+
connect. On headless Linux or over SSH, it prints a prominent copy-pasteable
130+
dashboard URL, including an SSH port-forward command for a loopback Gateway,
131+
and waits up to five minutes. A successful connection continues in the browser;
132+
an unreachable Gateway or a timeout falls back to the same terminal hatch as
133+
before. Pass `--tui` to skip the browser handoff and force that terminal hatch.
132134
If applying setup fails, onboarding falls back to the conversational OpenClaw
133135
chat to finish interactively. Channels, agents,
134136
plugins, and other optional features remain OpenClaw chat territory: run

docs/docs_map.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8649,6 +8649,10 @@ Do not edit it by hand; run `pnpm docs:map:gen`.
86498649
- H2: Agent schema history
86508650
- H2: State schema history
86518651
- H2: Integrity checks
8652+
- H2: Troubleshooting
8653+
- H3: Why you cannot go back after updating to 2026.7.2
8654+
- H3: The Gateway refuses to start with a newer schema version error
8655+
- H3: A database is quarantined after integrity verification failed
86528656
- H2: Downgrades are unsupported
86538657
- H3: Example: agent schema 11 to 9
86548658

docs/gateway/opentelemetry.md

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -469,8 +469,12 @@ content classes you opted into.
469469

470470
## Diagnostic event catalog
471471

472-
The events below back the metrics and spans above. Plugins can also
473-
subscribe to them directly without OTLP export.
472+
The events below back the metrics and spans above or are available for direct
473+
plugin subscription. `run.progress` and `run.execution_phase` are direct-only
474+
lifecycle signals; the diagnostics-otel plugin does not export them as
475+
standalone OTLP signals. Event kinds and `run.execution_phase.phase` values are
476+
additive. TypeScript consumers should keep default branches instead of assuming
477+
either union is permanently exhaustive.
474478

475479
**Model usage**
476480

@@ -490,6 +494,7 @@ subscribe to them directly without OTLP export.
490494
- `queue.lane.enqueue` / `queue.lane.dequeue`
491495
- `session.state` / `session.long_running` / `session.stalled` / `session.stuck`
492496
- `run.attempt` / `run.progress`
497+
- `run.execution_phase` (public, session-correlated embedded-runner startup milestones)
493498
- `diagnostic.heartbeat` (aggregate counters: webhooks/queue/session)
494499

495500
**Harness lifecycle**

docs/help/testing-live.md

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -320,14 +320,26 @@ Docker notes:
320320
- Default thinking: `low`
321321
- Model override: `OPENCLAW_LIVE_CODEX_HARNESS_MODEL=openai/<model>`
322322
- Thinking override: `OPENCLAW_LIVE_CODEX_HARNESS_THINKING=<level>`
323+
- Non-default model effort assertion:
324+
`OPENCLAW_LIVE_CODEX_HARNESS_EXPECTED_EFFORT=<level>`
323325
- Matrix override: `OPENCLAW_LIVE_CODEX_HARNESS_TARGETS=<model>=<thinking>,...`
324326
- Auth mode: `OPENCLAW_LIVE_CODEX_HARNESS_AUTH=codex-auth` (default) uses the
325327
copied Codex login; `api-key` uses `OPENAI_API_KEY` through Codex app-server.
326328
- Optional image probe: `OPENCLAW_LIVE_CODEX_HARNESS_IMAGE_PROBE=1`
327329
- Optional MCP/tool probe: `OPENCLAW_LIVE_CODEX_HARNESS_MCP_PROBE=1`
328330
- Optional Guardian probe: `OPENCLAW_LIVE_CODEX_HARNESS_GUARDIAN_PROBE=1`
331+
- Optional resume stress: `OPENCLAW_LIVE_CODEX_HARNESS_RESUME_STRESS=1` adds
332+
four history turns, then closes and restarts the Gateway and Codex app-server
333+
three times while requiring the same native thread id and conversation
334+
history. Override the bounded counts with
335+
`OPENCLAW_LIVE_CODEX_HARNESS_RESUME_STRESS_HISTORY_TURNS` (1-20) and
336+
`OPENCLAW_LIVE_CODEX_HARNESS_RESUME_STRESS_RESTARTS` (1-10).
329337
- Optional loop-relay opt-out probe:
330338
`OPENCLAW_LIVE_CODEX_HARNESS_DISABLE_LOOP_RELAY=1`
339+
- The requested thinking preference may map to the nearest effort advertised
340+
by Codex for that model. For example, Luna maps `minimal` to `low`.
341+
- Known Codex catalog models derive that exact native effort automatically.
342+
Unknown model overrides must state the expected mapped effort.
331343
- The smoke forces provider/model `agentRuntime.id: "codex"` so a broken Codex
332344
harness cannot pass by silently falling back to OpenClaw.
333345
- Auth: Codex app-server auth from the local Codex subscription login, or
@@ -351,6 +363,13 @@ Docker recipe:
351363
pnpm test:docker:live-codex-harness
352364
```
353365

366+
Restart and history stress:
367+
368+
```bash
369+
OPENCLAW_LIVE_CODEX_HARNESS_RESUME_STRESS=1 \
370+
pnpm test:docker:live-codex-harness
371+
```
372+
354373
GPT-5.6 native Codex matrix:
355374

356375
```bash

docs/reference/database-schemas.md

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -66,6 +66,28 @@ Version 3 was an unshipped development step folded into version 4.
6666
The Gateway preflight reads schema headers only. The background verifier owns the slower full scan for databases that do not need migration.
6767
Quarantine decisions live in a dedicated `openclaw-quarantine.sqlite` store, so they survive damage to the databases being quarantined. The global `database_verifications` table remains verification history.
6868

69+
## Troubleshooting
70+
71+
### Why you cannot go back after updating to 2026.7.2
72+
73+
Every release through `v2026.7.1` used agent schema 1 and state schema 1. The 2026.7.2 release train (starting with `v2026.7.2-beta.1`) migrates your databases forward on first start. That migration is one-way: the data is rewritten into the newer schema, and installing an older OpenClaw afterwards does not undo it. The older build refuses to start with a `newer schema version` error that names the build that owns the database.
74+
75+
Downgrading the binary never downgrades the data. If you must run a release older than 2026.7.2 after updating, you have three options:
76+
77+
1. Restore a backup taken before the update. [Create and verify backups](/cli/backup) before major updates.
78+
2. Run the older build against a separate state directory (`OPENCLAW_STATE_DIR`). It starts fresh; your migrated data stays untouched for when you return to the newer build.
79+
3. Follow the manual downgrade procedure below. It is unsupported and risks data loss without a verified backup.
80+
81+
Since 2026.7.2, `openclaw update` refuses to install a release that cannot open your current databases, so the updater will not put you in this situation. Installing an older version manually through npm bypasses that guard; the databases still refuse the old binary, but only after it is installed.
82+
83+
### The Gateway refuses to start with a newer schema version error
84+
85+
A newer OpenClaw build wrote your databases, and the running build is older. The error and the Gateway startup log name the build that owns the database (`app_version`). Install that version or newer, or use one of the options above. Do not edit the database to silence the error.
86+
87+
### A database is quarantined after integrity verification failed
88+
89+
The background verifier proved the file is corrupt, and every open now fails fast instead of rescanning. Restore the database from a backup or repair it, then run `openclaw doctor --fix` to clear the quarantine record. Doctor reports an explicit error if the quarantine record itself cannot be cleared; rerun it until it reports clean.
90+
6991
## Downgrades are unsupported
7092

7193
Manual schema downgrades are for agents and operators who accept the risk. [Create and verify a backup](/cli/backup) before editing any database. Stop the Gateway and every process that can open the database.

extensions/diagnostics-otel/src/service-events.ts

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -148,6 +148,8 @@ export function createDiagnosticsEventHandler(params: {
148148
break;
149149
case "run.progress":
150150
break;
151+
case "run.execution_phase":
152+
break;
151153
case "diagnostic.heartbeat":
152154
recordHeartbeat(evt);
153155
return;
Lines changed: 104 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,104 @@
1+
// Nostr tests cover outbound relay failover behavior.
2+
import { beforeEach, describe, expect, it, vi } from "vitest";
3+
import { startNostrBus } from "./nostr-bus.js";
4+
5+
const BAD_RELAY = "wss://bad-relay.example";
6+
const GOOD_RELAY = "wss://good-relay.example";
7+
const RECIPIENT_PUBKEY = "b".repeat(64);
8+
9+
const mocks = vi.hoisted(() => ({
10+
poolPublish: vi.fn(),
11+
close: vi.fn(),
12+
}));
13+
14+
vi.mock("nostr-tools", () => {
15+
class MockSimplePool {
16+
subscribeMany() {
17+
return { close: vi.fn() };
18+
}
19+
20+
publish(relays: string[]) {
21+
return mocks.poolPublish(relays);
22+
}
23+
24+
close(relays: string[]) {
25+
mocks.close(relays);
26+
}
27+
}
28+
29+
return {
30+
SimplePool: MockSimplePool,
31+
finalizeEvent: vi.fn((event: unknown) => event),
32+
getPublicKey: vi.fn(() => "a".repeat(64)),
33+
verifyEvent: vi.fn(() => true),
34+
nip19: {
35+
decode: vi.fn(),
36+
npubEncode: vi.fn(),
37+
},
38+
};
39+
});
40+
41+
vi.mock("nostr-tools/nip04", () => ({
42+
decrypt: vi.fn(),
43+
encrypt: vi.fn(() => "ciphertext"),
44+
}));
45+
46+
vi.mock("./nostr-state-store.js", () => ({
47+
readNostrBusState: vi.fn(async () => null),
48+
writeNostrBusState: vi.fn(async () => {}),
49+
computeSinceTimestamp: vi.fn(() => 0),
50+
readNostrProfileState: vi.fn(async () => null),
51+
writeNostrProfileState: vi.fn(async () => {}),
52+
}));
53+
54+
vi.mock("./nostr-profile.js", () => ({
55+
publishProfile: vi.fn(),
56+
}));
57+
58+
describe("Nostr outbound relay failover", () => {
59+
beforeEach(() => {
60+
mocks.poolPublish
61+
.mockReset()
62+
.mockImplementation((relays: string[]) => [
63+
Promise.resolve(
64+
relays[0] === BAD_RELAY ? "connection failure: connection failed" : "saved",
65+
),
66+
]);
67+
mocks.close.mockReset();
68+
});
69+
70+
it("tries the next relay when the first relay cannot connect", async () => {
71+
const bus = await startNostrBus({
72+
privateKey: "1".repeat(64),
73+
relays: [BAD_RELAY, GOOD_RELAY],
74+
onMessage: vi.fn(async () => {}),
75+
onMetric: () => {},
76+
});
77+
78+
await bus.sendDm(RECIPIENT_PUBKEY, "hello");
79+
80+
expect(mocks.poolPublish.mock.calls.map(([relays]) => relays)).toEqual([
81+
[BAD_RELAY],
82+
[GOOD_RELAY],
83+
]);
84+
bus.close();
85+
});
86+
87+
it("preserves string connection failures when every relay fails", async () => {
88+
const onError = vi.fn();
89+
const bus = await startNostrBus({
90+
privateKey: "1".repeat(64),
91+
relays: [BAD_RELAY],
92+
onMessage: vi.fn(async () => {}),
93+
onError,
94+
onMetric: () => {},
95+
});
96+
97+
await expect(bus.sendDm(RECIPIENT_PUBKEY, "hello")).rejects.toThrow(
98+
"Failed to publish to any relay: connection failed",
99+
);
100+
expect(onError).toHaveBeenCalledWith(expect.any(Error), `publish to ${BAD_RELAY}`);
101+
102+
bus.close();
103+
});
104+
});

extensions/nostr/src/nostr-bus.ts

Lines changed: 3 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,7 @@ import {
2323
readNostrProfileState,
2424
writeNostrProfileState,
2525
} from "./nostr-state-store.js";
26+
import { publishNostrEventToRelay } from "./relay-publish.js";
2627
import { createSeenTracker, type SeenTracker } from "./seen-tracker.js";
2728

2829
// ============================================================================
@@ -776,12 +777,7 @@ async function sendEncryptedDm(
776777

777778
const startTime = Date.now();
778779
try {
779-
const publishPromises = pool.publish([relay], reply);
780-
if (publishPromises.length === 0) {
781-
throw new Error(`Failed to create publish promise for relay ${relay}`);
782-
}
783-
const publishPromise = publishPromises[0];
784-
await publishPromise;
780+
await publishNostrEventToRelay(pool, relay, reply);
785781
const latency = Date.now() - startTime;
786782

787783
// Record success
@@ -790,7 +786,7 @@ async function sendEncryptedDm(
790786

791787
return reply.id;
792788
} catch (err) {
793-
lastError = err as Error;
789+
lastError = err instanceof Error ? err : new Error(String(err));
794790
const latency = Date.now() - startTime;
795791

796792
// Record failure

0 commit comments

Comments
 (0)