Skip to content

Commit 3e066dd

Browse files
committed
fix(synthos): keep the status list old-client safe and theme the library fully
A re-review and a QA re-check of the DAM library asked for these fixes: - Old clients: the cockpit reaches remote environments, so a client built before the library can talk to a server that has it, and the shared status list must not carry an id it cannot decode. "dam" leaves that list; library.status (orchestration:read) returns the library's state, capabilities and reason, including an unrecognized SYNTHOS_DAM_OWNER, and the Library panel reads it. A test decodes this branch's status result with the 311fa52 schema. - Principle code blocks were highlighted with the app's light theme inside the always-dark cockpit (1.36:1). useRenderedAppearance honours a RenderedAppearanceOverride context, which the cockpit sets to dark, so markdown code renders with the dark theme there (13.91:1). - Select lists opened outside the cockpit, light in the light appearance. SelectPopup takes a portalContainer, as AlertDialogPopup does, and the library opens its lists inside its own section. Styles and patterns no visible item carries are no longer offered. - New tests pin the owner gate for an unrecognized SYNTHOS_DAM_OWNER and the licence and framework statuses that files and previews carry. - The stderr budget slides over 100 ms buckets, so a burst that straddles a window reset is caught: any 10-second span over 256 KiB stops the process. The straddling pattern is a TestClock test. - An aborted library request now cancels its work: command callers can pass a signal, synthosRequest passes the panel's, and the HTTP request is interrupted (an aborted duplicate preview answers 499 on the server instead of running to completion). - decodeEntries is a named Effect.fn. Search results report how many entries were dropped, and the panel says so instead of suggesting a narrower search. Failures without a SynthOS reason (unreachable host, timeout, missing scope, a server without the request) get plain messages. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01A1D6J2UMPo7YBWQFvbg1br
1 parent 4fe69ca commit 3e066dd

19 files changed

Lines changed: 561 additions & 238 deletions

‎apps/server/src/cli/synthos.test.ts‎

Lines changed: 13 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -407,6 +407,12 @@ describe("SynthOS CLI authenticated transport", () => {
407407
styles: [{ id: "neon-glow", label: "Neon glow", definition: "Glow.", count: 3 }],
408408
patterns: [],
409409
};
410+
const libraryStatus = {
411+
library: "status",
412+
state: "configured",
413+
capabilities: ["library.search"],
414+
reason: "DAM checkout configured.",
415+
};
410416
const client = HttpClient.make((request) => {
411417
assert.equal(new URL(request.url).pathname, "/api/synthos/connections");
412418
assert.equal(request.method, "POST");
@@ -422,7 +428,9 @@ describe("SynthOS CLI authenticated transport", () => {
422428
? libraryFile
423429
: (body as { type: string }).type === "library.facets"
424430
? libraryFacets
425-
: { path: "note.md", content: "saved", version: "hash" },
431+
: (body as { type: string }).type === "library.status"
432+
? libraryStatus
433+
: { path: "note.md", content: "saved", version: "hash" },
426434
),
427435
),
428436
);
@@ -437,6 +445,7 @@ describe("SynthOS CLI authenticated transport", () => {
437445
},
438446
{ type: "library.file", id: "subframe-motion.magnetic-button", format: "prompt" },
439447
{ type: "library.facets" },
448+
{ type: "library.status" },
440449
];
441450
const results: unknown[] = [];
442451
for (const input of commands)
@@ -449,13 +458,15 @@ describe("SynthOS CLI authenticated transport", () => {
449458
assert.deepEqual(calls, [...commands]);
450459
assert.deepEqual(results[2], libraryFile);
451460
assert.deepEqual(results[3], libraryFacets);
461+
assert.deepEqual(results[4], libraryStatus);
452462
assert.deepEqual(auth.issuedScopes, [
453463
["filesystem:read"],
454464
["filesystem:write"],
455465
[AuthOrchestrationReadScope],
456466
[AuthOrchestrationReadScope],
467+
[AuthOrchestrationReadScope],
457468
]);
458-
assert.deepEqual(auth.revoked, [sessionId, sessionId, sessionId, sessionId]);
469+
assert.deepEqual(auth.revoked, [sessionId, sessionId, sessionId, sessionId, sessionId]);
459470
}),
460471
);
461472

‎apps/server/src/synthos-connections/CONTEXT.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,6 @@ Availability, authentication, and observed activity are separate facts. An execu
1010

1111
DAM is the asset library's system of record. SynthOS reads it only through DAM's own read-only MCP server, spawned from the configured checkout without a shell or SynthOS credentials, and keeps no copy. Owner-only items need both the host opt-in and the actor's `filesystem:read`; every other request reaches only the public process. DAM output, including file contents and quoted upstream text, is untrusted data: decoded within bounds, never interpreted.
1212

13-
SynthOS ships its web client with its server and mobile has no cockpit, so a client older than its server is unsupported: one built before a connection id existed cannot decode a status list that includes it, while current clients drop ids they do not know.
13+
The cockpit reaches remote environments, so a client built before a connection may talk to a server that has it. The shared `status` list therefore keeps the ids old clients can decode: DAM reports its status through `library.status` instead, and current clients drop list ids they do not know.
1414

1515
Mail delivery uses an immutable reviewed envelope. An unknown send result requires reconciliation before retry; a failed observation does not establish failed delivery. See [knowledge and mail](../../../../docs/user/synthos.md#connect-knowledge-and-mail).

‎apps/server/src/synthos-connections/DamLibrary.test.ts‎

Lines changed: 75 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@ import {
77
} from "@t3tools/contracts";
88
import * as HostProcess from "@t3tools/shared/HostProcess";
99
import type * as Cause from "effect/Cause";
10+
import * as Duration from "effect/Duration";
1011
import * as Effect from "effect/Effect";
1112
import * as Fiber from "effect/Fiber";
1213
import * as FileSystem from "effect/FileSystem";
@@ -162,7 +163,10 @@ const installDependencies = Effect.fn("test.installDependencies")(function* (roo
162163

163164
const libraryLayer = (
164165
spawner: ChildProcessSpawner.ChildProcessSpawner["Service"],
165-
config: { readonly damRoot: string; readonly damOwner?: boolean },
166+
config: {
167+
readonly damRoot: string | undefined;
168+
readonly damOwner?: ConnectionsConfig.DamOwnerSetting;
169+
},
166170
) =>
167171
DamLibrary.layer.pipe(
168172
Layer.provide(Layer.succeed(ChildProcessSpawner.ChildProcessSpawner, spawner)),
@@ -172,7 +176,7 @@ const libraryLayer = (
172176
memKey: undefined,
173177
obsidianVaultRoot: undefined,
174178
damRoot: config.damRoot,
175-
damOwner: config.damOwner ? "on" : "off",
179+
damOwner: config.damOwner ?? "off",
176180
}),
177181
),
178182
);
@@ -338,11 +342,13 @@ it.layer(NodeServices.layer)("DamLibrary", (it) => {
338342
publicArgs,
339343
[...publicArgs, "--owner"],
340344
]);
341-
}).pipe(Effect.provide(libraryLayer(dam.spawner, { damRoot: root, damOwner: true })));
342-
yield* Effect.gen(function* () {
343-
expect(yield* searchIds(fileReader)).toEqual(["public.button:public"]);
344-
expect(dam.spawns.at(-1)?.args).toEqual(publicArgs);
345-
}).pipe(Effect.provide(libraryLayer(dam.spawner, { damRoot: root, damOwner: false })));
345+
}).pipe(Effect.provide(libraryLayer(dam.spawner, { damRoot: root, damOwner: "on" })));
346+
for (const damOwner of ["off", "unrecognized"] as const)
347+
yield* Effect.gen(function* () {
348+
expect(yield* searchIds(fileReader)).toEqual(["public.button:public"]);
349+
expect(dam.spawns.at(-1)?.args).toEqual(publicArgs);
350+
}).pipe(Effect.provide(libraryLayer(dam.spawner, { damRoot: root, damOwner })));
351+
expect(dam.spawns.filter((command) => command.args.includes("--owner"))).toHaveLength(1);
346352
}).pipe(Effect.scoped),
347353
);
348354

@@ -379,7 +385,7 @@ it.layer(NodeServices.layer)("DamLibrary", (it) => {
379385
"public.button",
380386
"leaked.original",
381387
]);
382-
}).pipe(Effect.provide(libraryLayer(dam.spawner, { damRoot: root, damOwner: true })));
388+
}).pipe(Effect.provide(libraryLayer(dam.spawner, { damRoot: root, damOwner: "on" })));
383389
}).pipe(Effect.scoped),
384390
);
385391

@@ -396,6 +402,7 @@ it.layer(NodeServices.layer)("DamLibrary", (it) => {
396402
library: "search",
397403
total: 0,
398404
items: [],
405+
dropped: 0,
399406
});
400407
}).pipe(
401408
Effect.provide(libraryLayer(dam.spawner, { damRoot: root })),
@@ -494,7 +501,12 @@ it.layer(NodeServices.layer)("DamLibrary", (it) => {
494501
})
495502
: args.id === "edge"
496503
? edgeAsset()
497-
: asset(),
504+
: args.id === "derived.unverified"
505+
? asset({
506+
licence: { ...asset().licence, status: "unverified" },
507+
framework: { ...asset().framework, status: "derived" },
508+
})
509+
: asset(),
498510
};
499511
if (tool === "list_facets")
500512
return {
@@ -664,7 +676,16 @@ it.layer(NodeServices.layer)("DamLibrary", (it) => {
664676
styles: [{ id: "neon-glow", count: 3 }],
665677
patterns: [{ id: "skeleton-loaders", group: "component", count: 2 }],
666678
});
667-
}).pipe(Effect.provide(libraryLayer(dam.spawner, { damRoot: root, damOwner: true })));
679+
// Retrieval carries the item's own statuses, not defaults.
680+
const derived = { type: "library.file", id: "derived.unverified", format: "tsx" } as const;
681+
expect(yield* run(fileReader, derived)).toMatchObject({
682+
licence: { status: "unverified" },
683+
framework: { status: "derived" },
684+
});
685+
expect(
686+
yield* run(fileReader, { type: "library.preview", id: "derived.unverified" }),
687+
).toMatchObject({ licence: { status: "unverified" } });
688+
}).pipe(Effect.provide(libraryLayer(dam.spawner, { damRoot: root, damOwner: "on" })));
668689
}).pipe(Effect.scoped),
669690
);
670691

@@ -766,6 +787,7 @@ it.layer(NodeServices.layer)("DamLibrary", (it) => {
766787
library: "search",
767788
total: 3,
768789
items: [{ id: "fine" }],
790+
dropped: 2,
769791
});
770792
reply = replies.facets!;
771793
expect(yield* run(reader, { type: "library.facets" })).toMatchObject({
@@ -856,6 +878,33 @@ it.layer(NodeServices.layer)("DamLibrary", (it) => {
856878
}).pipe(Effect.scoped),
857879
);
858880

881+
it.effect("reports its status from configuration alone, without starting DAM", () =>
882+
Effect.gen(function* () {
883+
const root = yield* checkout();
884+
const dam = yield* fakeDam(() => ({ json: { total: 0, items: [] } }));
885+
const statusFor = (
886+
damRoot: string | undefined,
887+
damOwner: ConnectionsConfig.DamOwnerSetting,
888+
) =>
889+
run(reader, { type: "library.status" }).pipe(
890+
Effect.provide(libraryLayer(dam.spawner, { damRoot, damOwner })),
891+
);
892+
const unconfigured = yield* statusFor(undefined, "on");
893+
expect(unconfigured).toMatchObject({ library: "status", state: "unavailable" });
894+
expect(yield* statusFor(root, "on")).toMatchObject({
895+
state: "configured",
896+
reason: expect.stringContaining("Owner-only items are on"),
897+
});
898+
const unrecognized = yield* statusFor(root, "unrecognized");
899+
expect(unrecognized).toMatchObject({
900+
state: "configured",
901+
reason: expect.stringContaining("SYNTHOS_DAM_OWNER has a value SynthOS does not recognize"),
902+
});
903+
expect([unconfigured, unrecognized].every(isResult)).toBe(true);
904+
expect(dam.spawns).toHaveLength(0);
905+
}).pipe(Effect.scoped),
906+
);
907+
859908
it.effect("stops a DAM process for a stderr flood, not for a long uptime", () =>
860909
Effect.gen(function* () {
861910
const root = yield* checkout();
@@ -864,9 +913,10 @@ it.layer(NodeServices.layer)("DamLibrary", (it) => {
864913
const search = { type: "library.search" } as const;
865914
yield* Effect.gen(function* () {
866915
// Over the budget in total, under it in every window: the process keeps serving.
916+
const apart = Duration.sum(DamLibrary.STDERR_WINDOW, Duration.millis(200));
867917
for (let window = 0; window < 3; window += 1) {
868918
yield* run(reader, search);
869-
yield* TestClock.adjust(DamLibrary.STDERR_WINDOW);
919+
yield* TestClock.adjust(apart);
870920
}
871921
yield* run(reader, search);
872922
expect(dam.spawns).toHaveLength(1);
@@ -879,6 +929,20 @@ it.layer(NodeServices.layer)("DamLibrary", (it) => {
879929
}).pipe(Effect.scoped),
880930
);
881931

932+
it.effect("holds every ten-second span of stderr to the budget, across any boundary", () =>
933+
Effect.gen(function* () {
934+
const kibibytes = (count: number) => count * 1_024;
935+
const overBudget = DamLibrary.makeStderrBudget();
936+
// After 10 s of quiet: 50 KiB, then 200 KiB 9.3 s later and 200 KiB 1 s after that.
937+
yield* TestClock.adjust(DamLibrary.STDERR_WINDOW);
938+
expect(yield* overBudget(kibibytes(50))).toBe(false);
939+
yield* TestClock.adjust(Duration.millis(9_300));
940+
expect(yield* overBudget(kibibytes(200))).toBe(false);
941+
yield* TestClock.adjust(Duration.seconds(1));
942+
expect(yield* overBudget(kibibytes(200))).toBe(true);
943+
}),
944+
);
945+
882946
it.effect("stops a DAM that never finishes starting and starts a fresh one", () =>
883947
Effect.gen(function* () {
884948
const root = yield* checkout();

‎apps/server/src/synthos-connections/DamLibrary.ts‎

Lines changed: 62 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -61,6 +61,7 @@ export const MAX_MESSAGE_BYTES = 2 * 1_048_576;
6161
// A flood stops a process, not a long uptime: DAM logs a stack trace per INTERNAL answer.
6262
export const MAX_STDERR_BYTES = 262_144;
6363
export const STDERR_WINDOW = Duration.seconds(10);
64+
const STDERR_BUCKET_MILLIS = 100;
6465
const MCP_PROTOCOL_VERSION = "2025-06-18";
6566
const INVALID_PARAMS = -32_602;
6667
// PATH locates node, the temp variables tsx's compile cache; node needs SystemRoot on Windows.
@@ -283,13 +284,13 @@ const loggableId = (entry: unknown) => {
283284

284285
// A list entry this version cannot read (a newer kind, format or bound) is dropped rather than
285286
// failing the page, and logged by id and schema path only: never its values.
286-
function decodeEntries<A>(
287+
const decodeEntries = Effect.fn("DamLibrary.decodeEntries")(function* <A>(
287288
decode: (entry: unknown) => Effect.Effect<A, Schema.SchemaError>,
288289
entries: ReadonlyArray<unknown>,
289290
tool: string,
290291
mode: Mode,
291-
): Effect.Effect<Array<A>> {
292-
return Effect.forEach(entries, (entry) =>
292+
) {
293+
const decoded = yield* Effect.forEach(entries, (entry) =>
293294
decode(entry).pipe(
294295
Effect.map((value) => [value]),
295296
Effect.catch((error) =>
@@ -305,7 +306,58 @@ function decodeEntries<A>(
305306
),
306307
),
307308
),
308-
).pipe(Effect.map((decoded) => decoded.flat()));
309+
);
310+
return decoded.flat();
311+
});
312+
313+
// Sliding over 100 ms buckets, so a burst that straddles any boundary still counts in one window:
314+
// every STDERR_WINDOW span (and at most one bucket more) is held to MAX_STDERR_BYTES.
315+
export function makeStderrBudget() {
316+
const windowBuckets = Duration.toMillis(STDERR_WINDOW) / STDERR_BUCKET_MILLIS;
317+
const buckets = new Map<number, number>();
318+
let total = 0;
319+
return (bytes: number) =>
320+
Effect.map(Clock.currentTimeMillis, (now) => {
321+
const bucket = Math.floor(now / STDERR_BUCKET_MILLIS);
322+
for (const [key, held] of buckets) {
323+
if (key >= bucket - windowBuckets) break;
324+
buckets.delete(key);
325+
total -= held;
326+
}
327+
buckets.set(bucket, (buckets.get(bucket) ?? 0) + bytes);
328+
total += bytes;
329+
return total > MAX_STDERR_BYTES;
330+
});
331+
}
332+
333+
const CAPABILITIES = [
334+
"library.search",
335+
"library.facets",
336+
"library.asset",
337+
"library.file",
338+
"library.preview",
339+
"library.ideas",
340+
"library.idea",
341+
"library.principles",
342+
"library.principle",
343+
];
344+
345+
function libraryStatus(config: ConnectionsConfig.ConnectionsConfig["Service"]) {
346+
const checkout = config.damRoot
347+
? "DAM checkout configured; availability is checked when first used."
348+
: "Set SYNTHOS_DAM_ROOT to a DAM checkout with its dependencies installed on this host.";
349+
const owner =
350+
config.damOwner === "on"
351+
? "Owner-only items are on for sessions that can read files on this host."
352+
: config.damOwner === "unrecognized"
353+
? "SYNTHOS_DAM_OWNER has a value SynthOS does not recognize, so owner-only items are off; set it to 1 or true to show them to sessions that can read files."
354+
: "Owner-only items are off; set SYNTHOS_DAM_OWNER=1 to show them to sessions that can read files.";
355+
return {
356+
library: "status",
357+
state: config.damRoot ? "configured" : "unavailable",
358+
capabilities: CAPABILITIES,
359+
reason: `${checkout} ${owner}`,
360+
} as const;
309361
}
310362

311363
const PNG_SIGNATURE = [0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a];
@@ -492,19 +544,11 @@ const make = Effect.gen(function* () {
492544
}),
493545
Effect.forkScoped,
494546
);
495-
let stderrWindowStart = yield* Clock.currentTimeMillis;
496-
let stderrBytes = 0;
547+
const overStderrBudget = makeStderrBudget();
497548
yield* Stream.runForEach(handle.stderr, (chunk) =>
498-
Effect.flatMap(Clock.currentTimeMillis, (now) => {
499-
if (now - stderrWindowStart >= Duration.toMillis(STDERR_WINDOW)) {
500-
stderrWindowStart = now;
501-
stderrBytes = 0;
502-
}
503-
stderrBytes += chunk.byteLength;
504-
return stderrBytes > MAX_STDERR_BYTES
505-
? close(failure("transport"), "DAM_STDERR_LIMIT")
506-
: Effect.void;
507-
}),
549+
Effect.flatMap(overStderrBudget(chunk.byteLength), (over) =>
550+
over ? close(failure("transport"), "DAM_STDERR_LIMIT") : Effect.void,
551+
),
508552
).pipe(Effect.ignore, Effect.forkScoped);
509553
yield* Stream.fromQueue(outgoing).pipe(
510554
Stream.run(handle.stdin),
@@ -646,6 +690,7 @@ const make = Effect.gen(function* () {
646690

647691
const execute: DamLibrary["Service"]["execute"] = Effect.fn("DamLibrary.execute")(
648692
function* (actor, command) {
693+
if (command.type === "library.status") return libraryStatus(config);
649694
if (config.damRoot === undefined) return yield* failure("unavailable");
650695
const mode: Mode =
651696
config.damOwner === "on" && actor.scopes.includes("filesystem:read") ? "owner" : "public";
@@ -662,11 +707,11 @@ const make = Effect.gen(function* () {
662707
return yield* failure("response_invalid");
663708
}
664709
const cards = yield* decodeEntries(decodeCard, value.items, "search_assets", mode);
665-
// total stays DAM's count, so it can include entries dropped above.
666710
return {
667711
library: "search",
668712
total: value.total,
669713
items: cards.map((item) => toCard(item, mode)),
714+
dropped: value.items.length - cards.length,
670715
};
671716
}
672717
case "library.facets": {

0 commit comments

Comments
 (0)