Skip to content

Commit 4dd1874

Browse files
vincentkocvincentkoc
authored
fix(models): mark local Ollama rows available
1 parent b5390f5 commit 4dd1874

48 files changed

Lines changed: 3663 additions & 173 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎docs/.generated/config-baseline.sha256‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
f5a5855ddd7aa8c23a732f257eceaa20fd163b1d5f342c909f4aef15aa8643cf config-baseline.json
2-
b8dffdb1a328aaf728a0707ab04d21c65f1a225a2360042e10832aa608699716 config-baseline.core.json
1+
26afb40d889df462d5886e6fa0ecd09ce6bae12387836b59a2489c696a8ff3a1 config-baseline.json
2+
00242f93938579d3ff3130f8d42e08a423e43dc7a73e3b5e5afb45b80c78d25a config-baseline.core.json
33
671979e86e4c4f59415d0a20879e838f9bbd883b3d29eeb02cb5131db8d187fe config-baseline.channel.json
4-
94529978588d6e3776a86780b22cf9ff46a6f9957f2f178d3829403fad451ca7 config-baseline.plugin.json
4+
21ce3d97ac3a83323fa29ddc35045c04f6eae3040c2c5515077f543b4608fd12 config-baseline.plugin.json

‎docs/cli/plugins.md‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -54,6 +54,9 @@ openclaw plugins update <id-or-npm-spec>
5454
openclaw plugins update --all
5555
openclaw plugins marketplace list <marketplace>
5656
openclaw plugins marketplace list <marketplace> --json
57+
openclaw plugins marketplace refresh
58+
openclaw plugins marketplace refresh --feed-profile clawhub-public --json
59+
openclaw plugins marketplace refresh --feed-url https://clawhub.ai/v1/feeds/plugins --expected-sha256 <sha256>
5760
openclaw plugins init my-tool --name "My Tool"
5861
openclaw plugins init my-provider --name "My Provider" --type provider
5962
openclaw plugins init my-provider --name "My Provider" --type provider --directory ./my-provider
@@ -521,10 +524,26 @@ Use `plugins registry` to inspect whether the persisted registry is present, cur
521524
```bash
522525
openclaw plugins marketplace list <source>
523526
openclaw plugins marketplace list <source> --json
527+
openclaw plugins marketplace refresh
528+
openclaw plugins marketplace refresh --feed-profile <name>
529+
openclaw plugins marketplace refresh --feed-url <url>
530+
openclaw plugins marketplace refresh --expected-sha256 <sha256> --json
524531
```
525532

526533
Marketplace list accepts a local marketplace path, a `marketplace.json` path, a GitHub shorthand like `owner/repo`, a GitHub repo URL, or a git URL. `--json` prints the resolved source label plus the parsed marketplace manifest and plugin entries.
527534

535+
Marketplace refresh loads a hosted OpenClaw marketplace feed and persists the
536+
validated response as the local hosted-feed snapshot. Without options, it uses
537+
the configured default feed profile. Use `--feed-profile <name>` to refresh a
538+
specific configured profile, `--feed-url <url>` to refresh an explicit hosted
539+
feed URL, `--expected-sha256 <sha256>` to require a matching payload checksum
540+
(`sha256:<hex>` or a bare 64-character hex digest), and `--json` for
541+
machine-readable output. Explicit hosted feed URLs must not include
542+
credentials, query strings, or fragments. Unpinned refreshes can report a
543+
hosted snapshot or bundled fallback result without failing the command. Pinned
544+
refreshes fail unless they accept a fresh hosted payload, and successful hosted
545+
refreshes fail if OpenClaw cannot persist the validated snapshot.
546+
528547
## Related
529548

530549
- [Building plugins](/plugins/building-plugins)

‎docs/plugins/sdk-runtime.md‎

Lines changed: 2 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -131,11 +131,9 @@ two-party event loops that do not go through the shared inbound reply runner.
131131
await api.runtime.agent.ensureAgentWorkspace(cfg);
132132

133133
// Run an embedded agent turn
134-
const agentDir = api.runtime.agent.resolveAgentDir(cfg);
135134
const result = await api.runtime.agent.runEmbeddedAgent({
136135
sessionId: "my-plugin:task-1",
137136
runId: crypto.randomUUID(),
138-
sessionFile: path.join(agentDir, "sessions", "my-plugin-task-1.jsonl"),
139137
workspaceDir: api.runtime.agent.resolveAgentWorkspaceDir(cfg),
140138
prompt: "Summarize the latest changes",
141139
timeoutMs: api.runtime.agent.resolveAgentTimeoutMs(cfg),
@@ -166,9 +164,9 @@ two-party event loops that do not go through the shared inbound reply runner.
166164

167165
Prefer `getSessionEntry(...)`, `listSessionEntries(...)`, `patchSessionEntry(...)`, or `upsertSessionEntry(...)` for session workflows. These helpers address sessions by agent/session identity so plugins do not depend on the legacy `sessions.json` storage shape. Use `preserveActivity: true` for metadata-only patches that should not refresh session activity, and `replaceEntry: true` only when the callback returns a complete entry and deleted fields must stay deleted.
168166

169-
For transcript reads and writes, import `openclaw/plugin-sdk/session-transcript-runtime` and use `resolveSessionTranscriptIdentity(...)`, `resolveSessionTranscriptTarget(...)`, `readSessionTranscriptEvents(...)`, `appendSessionTranscriptMessageByIdentity(...)`, `publishSessionTranscriptUpdateByIdentity(...)`, or `withSessionTranscriptWriteLock(...)` with `{ agentId, sessionKey, sessionId }`. These APIs let plugins identify a transcript, read its events, append messages, publish updates, and run related operations under the same transcript write lock. Pass `sessionFile` only when adapting code that already receives an active transcript artifact and needs each helper to operate on that same artifact.
167+
For transcript reads and writes, import `openclaw/plugin-sdk/session-transcript-runtime` and use `resolveSessionTranscriptIdentity(...)`, `resolveSessionTranscriptTarget(...)`, `readSessionTranscriptEvents(...)`, `appendSessionTranscriptMessageByIdentity(...)`, `publishSessionTranscriptUpdateByIdentity(...)`, or `withSessionTranscriptWriteLock(...)` with `{ agentId, sessionKey, sessionId }`. These APIs let plugins identify a transcript, read its events, append messages, publish updates, and run related operations under the same transcript write lock. Passing `sessionFile`, using `resolveSessionTranscriptLegacyFileTarget(...)`, or importing low-level `appendSessionTranscriptMessage(...)` / `emitSessionTranscriptUpdate(...)` from `openclaw/plugin-sdk/agent-harness-runtime` is deprecated; those paths exist only for legacy code that already receives an active transcript artifact.
170168

171-
`loadSessionStore(...)`, `saveSessionStore(...)`, `updateSessionStore(...)`, and `resolveSessionFilePath(...)` are compatibility helpers for plugins that still intentionally depend on the legacy whole-store or transcript-file shape. New plugin code must not use those helpers, and existing callers should migrate to entry helpers.
169+
`loadSessionStore(...)`, `saveSessionStore(...)`, `updateSessionStore(...)`, `resolveSessionFilePath(...)`, and `resolveAndPersistSessionFile(...)` are deprecated compatibility helpers for plugins that still intentionally depend on the legacy whole-store or transcript-file shape. New plugin code must not use those helpers, and existing callers should migrate to entry helpers and transcript identity helpers.
172170

173171
</Accordion>
174172
<Accordion title="api.runtime.agent.defaults">

‎docs/providers/opencode.md‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,7 @@ as one OpenCode setup.
2121

2222
<Tabs>
2323
<Tab title="Zen catalog">
24-
**Best for:** the curated OpenCode multi-model proxy (Claude, GPT, Gemini).
24+
**Best for:** the curated OpenCode multi-model proxy (Claude, GPT, Gemini, GLM).
2525

2626
<Steps>
2727
<Step title="Run onboarding">
@@ -92,10 +92,10 @@ as one OpenCode setup.
9292

9393
### Zen
9494

95-
| Property | Value |
96-
| ---------------- | ----------------------------------------------------------------------- |
97-
| Runtime provider | `opencode` |
98-
| Example models | `opencode/claude-opus-4-6`, `opencode/gpt-5.5`, `opencode/gemini-3-pro` |
95+
| Property | Value |
96+
| ---------------- | --------------------------------------------------------------------------------------------- |
97+
| Runtime provider | `opencode` |
98+
| Example models | `opencode/claude-opus-4-6`, `opencode/gpt-5.5`, `opencode/gemini-3.1-pro`, `opencode/glm-5.2` |
9999

100100
### Go
101101

‎extensions/discord/src/internal/rest.test.ts‎

Lines changed: 72 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -692,6 +692,78 @@ describe("RequestClient", () => {
692692
expect(metrics.invalidRequestCountByStatus).toEqual({ 403: 1 });
693693
});
694694

695+
it("bounds oversized REST response bodies instead of buffering them unbounded", async () => {
696+
const encoder = new TextEncoder();
697+
let pullCount = 0;
698+
let cancelCount = 0;
699+
const fetchSpy = vi.fn(
700+
async () =>
701+
new Response(
702+
new ReadableStream<Uint8Array>({
703+
pull(controller) {
704+
pullCount += 1;
705+
// Flood far past the cap so an unbounded reader would OOM.
706+
controller.enqueue(encoder.encode("x".repeat(4 * 1024 * 1024)));
707+
},
708+
cancel() {
709+
cancelCount += 1;
710+
},
711+
}),
712+
{ status: 200 },
713+
),
714+
);
715+
const client = new RequestClient("test-token", { fetch: fetchSpy, queueRequests: false });
716+
717+
await expect(client.get("/channels/c1/messages")).rejects.toThrow(
718+
/Discord REST response body exceeds 8388608 bytes/,
719+
);
720+
// The reader was cancelled at the cap rather than draining the whole flood:
721+
// only a handful of 4 MiB chunks are pulled before the cap is hit.
722+
expect(cancelCount).toBe(1);
723+
expect(pullCount).toBeLessThanOrEqual(4);
724+
});
725+
726+
it("aborts stalled REST response bodies after the idle timeout", async () => {
727+
const encoder = new TextEncoder();
728+
let cancelReason: unknown;
729+
const fetchSpy = vi.fn(
730+
async () =>
731+
new Response(
732+
new ReadableStream<Uint8Array>({
733+
start(controller) {
734+
// Emit a partial chunk, then stall forever so the idle timeout
735+
// (request timeout) must fire and cancel the stream.
736+
controller.enqueue(encoder.encode("partial payload"));
737+
},
738+
cancel(reason) {
739+
cancelReason = reason;
740+
},
741+
}),
742+
{ status: 200 },
743+
),
744+
);
745+
const client = new RequestClient("test-token", {
746+
fetch: fetchSpy,
747+
queueRequests: false,
748+
timeout: 50,
749+
});
750+
751+
await expect(client.get("/channels/c1/messages")).rejects.toThrow(
752+
"Discord REST response stalled: no data received for 50ms",
753+
);
754+
expect(cancelReason).toBeInstanceOf(Error);
755+
expect((cancelReason as Error).message).toBe(
756+
"Discord REST response stalled: no data received for 50ms",
757+
);
758+
});
759+
760+
it("still parses normal-sized REST response payloads under the cap", async () => {
761+
const fetchSpy = vi.fn(async () => createJsonResponse({ id: "channel", name: "general" }));
762+
const client = new RequestClient("test-token", { fetch: fetchSpy, queueRequests: false });
763+
764+
await expect(client.get("/channels/c1")).resolves.toEqual({ id: "channel", name: "general" });
765+
});
766+
695767
it("serializes message multipart uploads with payload_json", () => {
696768
const headers = new Headers();
697769
const body = serializeRequestBody(

‎extensions/discord/src/internal/rest.ts‎

Lines changed: 20 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@ import {
66
parseFiniteNumber,
77
resolveTimerTimeoutMs,
88
} from "openclaw/plugin-sdk/number-runtime";
9+
import { readResponseWithLimit } from "openclaw/plugin-sdk/response-limit-runtime";
910
import { serializeRequestBody } from "./rest-body.js";
1011
import {
1112
DiscordError,
@@ -89,6 +90,24 @@ const defaultLaneOptions: Record<RestRequestPriority, { staleAfterMs?: number; w
8990
background: { staleAfterMs: 20_000, weight: 1 },
9091
};
9192

93+
// Cap the REST response body well above any legitimate Discord JSON payload
94+
// (bulk message/member fetches stay in the low hundreds of KB) so a controlled
95+
// or hijacked endpoint cannot flood the body into an unbounded buffer (OOM).
96+
const DISCORD_REST_RESPONSE_BODY_MAX_BYTES = 8 * 1024 * 1024;
97+
98+
async function readResponseBodyText(response: Response, idleTimeoutMs: number): Promise<string> {
99+
const buffer = await readResponseWithLimit(response, DISCORD_REST_RESPONSE_BODY_MAX_BYTES, {
100+
chunkTimeoutMs: idleTimeoutMs,
101+
onOverflow: ({ size }) =>
102+
new Error(
103+
`Discord REST response body exceeds ${DISCORD_REST_RESPONSE_BODY_MAX_BYTES} bytes (received ${size})`,
104+
),
105+
onIdleTimeout: ({ chunkTimeoutMs }) =>
106+
new Error(`Discord REST response stalled: no data received for ${chunkTimeoutMs}ms`),
107+
});
108+
return buffer.toString("utf8");
109+
}
110+
92111
function coerceResponseBody(raw: string): unknown {
93112
if (!raw) {
94113
return undefined;
@@ -249,7 +268,7 @@ export class RequestClient {
249268
body: await normalizeFetchBody(body, headers),
250269
signal,
251270
});
252-
const text = await response.text();
271+
const text = await readResponseBodyText(response, this.options.timeout ?? 15_000);
253272
const parsed = coerceResponseBody(text);
254273
this.scheduler.recordResponse(routeKey, path, response, parsed);
255274
if (response.status === 204) {

‎extensions/mattermost/src/mattermost/client.test.ts‎

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -201,6 +201,68 @@ describe("createMattermostClient", () => {
201201
expect(release).toHaveBeenCalledTimes(1);
202202
});
203203

204+
it("bounds and cancels oversized guarded Mattermost success JSON bodies", async () => {
205+
const release = vi.fn(async () => {});
206+
let canceled = false;
207+
let pulled = 0;
208+
const oversizeChunk = new Uint8Array(2 * 1024 * 1024).fill(0x7b); // 2 MiB of '{'
209+
const stream = new ReadableStream<Uint8Array>({
210+
pull(controller) {
211+
pulled += 1;
212+
// Flood far past the 16 MiB JSON cap; an unbounded reader would buffer
213+
// the whole stream before parsing.
214+
controller.enqueue(oversizeChunk);
215+
},
216+
cancel() {
217+
canceled = true;
218+
},
219+
});
220+
fetchWithSsrFGuardMock.mockResolvedValueOnce({
221+
response: new Response(stream, {
222+
status: 200,
223+
headers: { "content-type": "application/json" },
224+
}),
225+
release,
226+
});
227+
const client = createMattermostClient({
228+
baseUrl: "https://chat.example.com",
229+
botToken: "test-token",
230+
});
231+
232+
let caught: Error | undefined;
233+
try {
234+
await client.request("/users/me");
235+
} catch (error) {
236+
caught = error as Error;
237+
}
238+
239+
expect(caught?.message).toContain("JSON response exceeds 16777216 bytes");
240+
// The reader is cancelled at the cap instead of draining the flood: ~8
241+
// chunks of 2 MiB reach the 16 MiB ceiling, never the unbounded tail.
242+
expect(canceled).toBe(true);
243+
expect(pulled).toBeLessThanOrEqual(12);
244+
expect(release).toHaveBeenCalledTimes(1);
245+
});
246+
247+
it("rejects oversized guarded Mattermost success text bodies instead of truncating", async () => {
248+
const release = vi.fn(async () => {});
249+
const tracked = cancelTrackedResponse(`${"plain success ".repeat(7000)}tail`, {
250+
status: 200,
251+
headers: { "content-type": "text/plain" },
252+
});
253+
fetchWithSsrFGuardMock.mockResolvedValueOnce({ response: tracked.response, release });
254+
const client = createMattermostClient({
255+
baseUrl: "https://chat.example.com",
256+
botToken: "test-token",
257+
});
258+
259+
await expect(client.request("/users/me")).rejects.toThrow(
260+
"Mattermost API /users/me: text response exceeds 65536 bytes",
261+
);
262+
expect(tracked.wasCanceled()).toBe(true);
263+
expect(release).toHaveBeenCalledTimes(1);
264+
});
265+
204266
it("releases guarded Mattermost responses when upstream body reads fail", async () => {
205267
const release = vi.fn(async () => {});
206268
const stream = new ReadableStream<Uint8Array>({

‎extensions/mattermost/src/mattermost/client.ts‎

Lines changed: 26 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,10 @@
11
// Mattermost plugin module implements client behavior.
22
import { resolveTimerTimeoutMs } from "openclaw/plugin-sdk/number-runtime";
3-
import { readResponseTextLimited } from "openclaw/plugin-sdk/provider-http";
3+
import {
4+
readProviderJsonResponse,
5+
readResponseTextLimited,
6+
} from "openclaw/plugin-sdk/provider-http";
7+
import { readResponseWithLimit } from "openclaw/plugin-sdk/response-limit-runtime";
48
import { sleep } from "openclaw/plugin-sdk/runtime-env";
59
import {
610
fetchWithSsrFGuard,
@@ -13,6 +17,13 @@ import {
1317
import { z } from "zod";
1418

1519
const MATTERMOST_ERROR_BODY_LIMIT_BYTES = 8 * 1024;
20+
// Mattermost REST control-plane JSON (posts, users, channels, file-upload
21+
// results) stays well under a megabyte; cap successful JSON the same way the
22+
// shared provider path is capped so an untrusted/self-hosted homeserver cannot
23+
// stream an unbounded body into the runtime before parsing.
24+
// Non-JSON success bodies are a rare fallback (the API is JSON-first); keep a
25+
// generous text budget but still bound it instead of buffering the whole stream.
26+
const MATTERMOST_TEXT_RESPONSE_LIMIT_BYTES = 64 * 1024;
1627
const NULL_BODY_STATUSES = new Set([101, 204, 205, 304]);
1728

1829
export type MattermostFetch = (input: RequestInfo | URL, init?: RequestInit) => Promise<Response>;
@@ -84,6 +95,14 @@ function buildMattermostApiUrl(baseUrl: string, path: string): string {
8495
return `${normalized}/api/v4${suffix}`;
8596
}
8697

98+
async function readMattermostSuccessText(res: Response, path: string): Promise<string> {
99+
const bytes = await readResponseWithLimit(res, MATTERMOST_TEXT_RESPONSE_LIMIT_BYTES, {
100+
onOverflow: ({ maxBytes }) =>
101+
new Error(`Mattermost API ${path}: text response exceeds ${maxBytes} bytes`),
102+
});
103+
return new TextDecoder().decode(bytes);
104+
}
105+
87106
export async function readMattermostError(res: Response): Promise<string> {
88107
const contentType = res.headers.get("content-type") ?? "";
89108
if (!res.body) {
@@ -214,9 +233,9 @@ export function createMattermostClient(params: {
214233

215234
const contentType = res.headers.get("content-type") ?? "";
216235
if (contentType.includes("application/json")) {
217-
return (await res.json()) as T;
236+
return await readProviderJsonResponse<T>(res, `Mattermost API ${path}`);
218237
}
219-
return (await res.text()) as T;
238+
return (await readMattermostSuccessText(res, path)) as T;
220239
};
221240

222241
return { baseUrl, apiBaseUrl, token, request, fetchImpl };
@@ -679,7 +698,10 @@ export async function uploadMattermostFile(
679698
const detail = await readMattermostError(res);
680699
throw new Error(`Mattermost API ${res.status} ${res.statusText}: ${detail || "unknown error"}`);
681700
}
682-
const data = (await res.json()) as { file_infos?: MattermostFileInfo[] };
701+
const data = await readProviderJsonResponse<{ file_infos?: MattermostFileInfo[] }>(
702+
res,
703+
"Mattermost API /files",
704+
);
683705
const info = data.file_infos?.[0];
684706
if (!info?.id) {
685707
throw new Error("Mattermost file upload failed");

‎extensions/mattermost/src/mattermost/probe.test.ts‎

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -74,6 +74,37 @@ describe("probeMattermost", () => {
7474
expect(mockRelease).toHaveBeenCalledTimes(1);
7575
});
7676

77+
it("bounds and cancels oversized probe success JSON bodies", async () => {
78+
let canceled = false;
79+
let pulled = 0;
80+
const oversizeChunk = new Uint8Array(2 * 1024 * 1024).fill(0x7b); // 2 MiB of "{"
81+
const stream = new ReadableStream<Uint8Array>({
82+
pull(controller) {
83+
pulled += 1;
84+
controller.enqueue(oversizeChunk);
85+
},
86+
cancel() {
87+
canceled = true;
88+
},
89+
});
90+
mockFetchGuard.mockResolvedValueOnce({
91+
response: new Response(stream, {
92+
status: 200,
93+
headers: { "content-type": "application/json" },
94+
}),
95+
release: mockRelease,
96+
});
97+
98+
const result = await probeMattermost("https://mm.example.com", "bot-token");
99+
100+
expect(result.ok).toBe(false);
101+
expect(result.status).toBeNull();
102+
expect(result.error).toContain("JSON response exceeds 16777216 bytes");
103+
expect(canceled).toBe(true);
104+
expect(pulled).toBeLessThanOrEqual(12);
105+
expect(mockRelease).toHaveBeenCalledTimes(1);
106+
});
107+
77108
it("forwards allowPrivateNetwork to the SSRF guard policy", async () => {
78109
mockFetchGuard.mockResolvedValueOnce({
79110
response: new Response(JSON.stringify({ id: "bot-1" }), {

0 commit comments

Comments
 (0)