Skip to content

Commit b58d937

Browse files
authored
✨ Slice A of #827: project sequential REPL entries from one Journal (#866)
One REPL execution keeps one physical retained stream, and that stream can now hold more than one entry. Nothing new is written down to say so: the protocol already marks the boundary, because an entry's root document is imported under one durable name on one coroutine and that coroutine closes once. A segment is the range from a root `import_component("__root__")` yield on `root` to that range's `close:root`, and only the final one may be unfinished. `packages/cli/src/repl/entries.ts` owns that grammar and the private view over it. `partitionEntrySegments()` validates the complete physical prefix before any part of it is projected, so a file whose boundaries are not boundaries refuses as one error with no entries beside it — work outside any entry, a nested or repeated root admission, a close with nothing open, a second unfinished range. `EntrySegmentStream` is what one execution is handed: `readAll()` answers with that range alone, so replay cannot correlate a repeated coroutine id across entries, and `append()` reaches the physical stream first and is remembered only once that stream has taken it. A settled or non-final range refuses every append, and no physical position crosses the boundary. `ReplModel` now projects an ordered `entries` collection in admission order. Each entry owns its `entry-N` key, its immutable source, its root scope, its transcript, its checkpoints, its outcome and the root bindings in effect after it — the last value each name was durably published under across the entries before it, with its own publications over the top. A value published before a failure or an interruption survives; one the failing block never retained appears nowhere. Prompt-sequence uniqueness is now checked inside the entry that owns it, because a sequence counts one execution's Prompts and every entry is its own execution. Every retained export name is checked against the parser an authored binding is checked against, while the eval result is read. A name no block could have bound refuses the projection there, rather than travelling into a later entry's environment and failing in a generated preamble as a syntax error about a document that never wrote it. Boundaries alone are not enough to show any prefix. Every segment in the complete file is read semantically before a model escapes, whatever a selection names, so a later entry whose retained payload this version cannot read refuses the earlier position too. The alternative is a reader standing in history, shown a catalog that looks whole, and told the file is broken only on returning to the head. The first entry keeps every marker spelling it already had, so every location anybody has written still resolves to the event it named. Later entries namespace their segment-local markers with their key — `entry-2:yield:root:0`, `entry-2:close:root` — so a repeated `close:root` cannot resolve as a global position. Core gains one trusted-host input, `ExecutionInitialization`, as the third argument of `executeInstalled()` and exported only from `@executablemd/core/host`. Its `initialBindings` record is parsed, name-validated and recursively copied before any installation or document code runs, and installed as initial bare root bindings. The copy is the whole of the isolation and nothing is frozen: a durable eval value is ordinary mutable data, so an entry that inherits a list and appends to it does what any block does to any binding, while the caller's own graph stays unreachable and unfrozen. Each name is *defined* rather than assigned, because `values["__proto__"] = …` reaches `Object.prototype`'s inherited setter and what that does depends on the engine — a retained record can carry that name, since `JSON.parse` makes it an own property, and the same bytes would otherwise become a different environment under Node and Bun than under Deno. `props` is the one name that does not cross an entry boundary. Each execution's props namespace is the one its own root validated, so a value published under that name stays in the scope that published it, is absent from effective cross-entry bindings, and is refused outright as an initial binding — rather than projected as passed and then silently dropped by the execution that already owns its own. An ordinary export still replaces an initial binding through the rules that replace any other, and `execute()` is unchanged. Consumers of the shared model move mechanically to the collection and enable nothing: submission still refuses a second entry, the route still names the one entry, and a session refuses a history holding more than one, because serially running them is the lifecycle work the next slice owns. Evidence: `repl-entries.test.ts` (ES1 segments, ES2 atomic refusals including a selection that must not escape an unreadable later entry, EM1 keys and markers, EP1 projection, inheritance, the reserved `props` namespace and invalid retained names, BC1 one-entry compatibility) over journals written by really executing entries through their own views; EB1 in core's binding suite, over the real host entrypoint with installations that rewrite the caller's record mid-run, with a cross-entry mutable array, with `props` and unbindable names refused at the boundary, and with a retained `__proto__` binding that only a Node run can hold to account.
1 parent 40346d9 commit b58d937

14 files changed

Lines changed: 2097 additions & 174 deletions

‎packages/cli/src/repl/application.ts‎

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -674,7 +674,7 @@ function availabilityOf(state: ReplState, live: ReplLive): ReplLiveAvailability
674674
const EMPTY_MODEL: ReplModel = Object.freeze({
675675
selection: undefined,
676676
head: true,
677-
entry: undefined,
677+
entries: Object.freeze([]),
678678
settled: false,
679679
terminal: undefined,
680680
checkpoints: Object.freeze([]),
@@ -711,7 +711,7 @@ export function reduceRepl(
711711
if (answering) {
712712
return editing(state, live, action.field, (value) => value + action.text);
713713
}
714-
if (model.entry !== undefined) {
714+
if (model.entries.length > 0) {
715715
return refuse(state, "this execution has admitted its entry, and an entry is immutable.");
716716
}
717717
return drafting(state, state.draft + action.text);
@@ -720,13 +720,13 @@ export function reduceRepl(
720720
if (answering) {
721721
return editing(state, live, action.field, shortened);
722722
}
723-
if (model.entry !== undefined) {
723+
if (model.entries.length > 0) {
724724
return refuse(state, "this execution has admitted its entry, and an entry is immutable.");
725725
}
726726
return drafting(state, shortened(state.draft));
727727
}
728728
case "submit": {
729-
if (model.entry !== undefined) {
729+
if (model.entries.length > 0) {
730730
return refuse(state, "this REPL admits one entry, and this execution has admitted it.");
731731
}
732732
if (state.draft.length === 0) {
@@ -1305,7 +1305,7 @@ export function focusSettled(view: ReplView, focused: string | undefined): ReplS
13051305
return Object.freeze({ ...view.state, restore: undefined });
13061306
}
13071307

1308-
/** Every answer this history already holds, wherever in the entry it holds it. */
1308+
/** Every answer this history already holds, wherever in its entries it holds it. */
13091309
function retainedAnswers(model: ReplModel): readonly string[] {
13101310
const markers: string[] = [];
13111311
const walk = (scope: ReplScope): void => {
@@ -1316,8 +1316,8 @@ function retainedAnswers(model: ReplModel): readonly string[] {
13161316
walk(child);
13171317
}
13181318
};
1319-
if (model.entry !== undefined) {
1320-
walk(model.entry);
1319+
for (const entry of model.entries) {
1320+
walk(entry.scope);
13211321
}
13221322
return Object.freeze(markers);
13231323
}
@@ -1500,7 +1500,7 @@ function described(view: ReplView): readonly Described[] {
15001500
}
15011501
// Read whether or not this frame draws the entry list: the footer's draft says
15021502
// whether an entry exists at every size and on either surface.
1503-
const entry = model.entry;
1503+
const entry = model.entries[0]?.scope;
15041504
if (!narrow) {
15051505
items.push(toEntries);
15061506
}
@@ -1863,7 +1863,7 @@ function sessionsCapacity(state: ReplState, model: ReplModel, size: ReplTerminal
18631863

18641864
/** How many rows the entry list takes in a sidebar, its heading included. */
18651865
function entryRows(model: ReplModel): number {
1866-
const entry = model.entry;
1866+
const entry = model.entries[0]?.scope;
18671867
return entry === undefined ? 2 : 2 + nested(entry, [entry.key]).length;
18681868
}
18691869

‎packages/cli/src/repl/entries.ts‎

Lines changed: 300 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,300 @@
1+
/**
2+
* Where one entry's history stops and the next one begins.
3+
*
4+
* One REPL execution keeps one physical append-only stream, and a person may
5+
* submit into it more than once. Nothing in the file says which entry a record
6+
* belongs to, and nothing needs to: the protocol already writes the boundary.
7+
* The root document is imported under one durable name on one coroutine, and
8+
* that coroutine closes once — so an entry is exactly the range from a root
9+
* `import_component("__root__")` yield on `root` to that range's `close:root`.
10+
* Reading the boundary out of the records already there is what keeps a second
11+
* entry from needing a second storage, a record family, a manifest or an index.
12+
*
13+
* This module owns that grammar and nothing else. It says which ranges a file
14+
* holds and refuses a file whose ranges are not ranges; what a range *means* —
15+
* scopes, bindings, transcripts, outcomes — belongs to the projector, which
16+
* reads each range exactly as it has always read a one-entry history. Keeping
17+
* the two apart is what lets the whole file be validated before any part of it
18+
* is projected, so a reader is never shown a catalog assembled from the half of
19+
* a history that happened to parse.
20+
*
21+
* ## Why an execution is handed a view and not the file
22+
*
23+
* Replay correlates a record by coroutine id and durable name, and both restart
24+
* for every entry: two entries both import `__root__` on `root`, and their
25+
* nested work carries the same coroutine ids. An execution given the whole file
26+
* would replay somebody else's entry as its own. So each execution is handed
27+
* `EntrySegmentStream`, which reads exactly its own range and appends to the one
28+
* physical stream behind it. There is one writer and one copy of the truth; what
29+
* differs is how much of it an execution is allowed to see.
30+
*/
31+
32+
import { Err, Ok } from "effection";
33+
import type { Operation, Result } from "effection";
34+
import type { DurableEvent, DurableStream, Yield } from "@executablemd/durable-streams";
35+
36+
/** The durable name of the root document import, on the root coroutine. */
37+
export const ROOT_IMPORT = "__root__";
38+
39+
/** The coroutine an entry's root document runs on. */
40+
const ROOT_COROUTINE = "root";
41+
42+
/** What the REPL could not read or write about an entry boundary. */
43+
export class ReplSegmentError extends Error {
44+
constructor(message: string) {
45+
super(message);
46+
this.name = "ReplSegmentError";
47+
}
48+
}
49+
50+
/**
51+
* The stable key of the entry admitted in this position.
52+
*
53+
* Derived from admission order and from nothing else. It is not a Journal
54+
* marker, not a completion rank and not a name anybody typed, so an entry that
55+
* fails, is cancelled or runs for an hour keeps the key it was admitted under.
56+
*/
57+
export function entryKey(order: number): string {
58+
return `entry-${order}`;
59+
}
60+
61+
/**
62+
* One segment-local marker, spelled the way the whole history spells it.
63+
*
64+
* The first entry keeps every marker exactly as a one-entry history wrote it,
65+
* because those spellings are already in people's locations and there is no
66+
* reading of `yield:root:0` that could mean anything else there. Every later
67+
* entry namespaces its own, so a second `close:root` names the second entry's
68+
* close instead of resolving to the first one's — which is what a repeated
69+
* coroutine name would otherwise do to a selection.
70+
*/
71+
export function entryMarker(order: number, local: string): string {
72+
return order === 1 ? local : `${entryKey(order)}:${local}`;
73+
}
74+
75+
/** One entry's range of the physical stream. */
76+
export interface EntrySegment {
77+
/** The admission-order key of the entry this range belongs to. */
78+
readonly key: string;
79+
/** Its admission order, counting from one. */
80+
readonly order: number;
81+
/** Where the range starts in the physical stream, inclusive. */
82+
readonly start: number;
83+
/** Where it ends, exclusive. */
84+
readonly end: number;
85+
/** Whether the range holds this entry's terminal `close:root`. */
86+
readonly settled: boolean;
87+
/** The events in the range, in append order. */
88+
readonly events: readonly DurableEvent[];
89+
/**
90+
* The root admission the range begins with, which is also `events[0]`.
91+
*
92+
* Carried as the yield it is, because the partition is what established that
93+
* it is one: a reader that took `events[0]` and asserted its type would be
94+
* restating the grammar instead of being handed its result.
95+
*/
96+
readonly admission: Yield;
97+
}
98+
99+
/** This event as a root admission, or none because it is not one. */
100+
function rootAdmission(event: DurableEvent): Yield | undefined {
101+
if (
102+
event.type === "yield" &&
103+
event.description.type === "import_component" &&
104+
event.description.name === ROOT_IMPORT
105+
) {
106+
return event;
107+
}
108+
return undefined;
109+
}
110+
111+
/** Whether this event settles a root coroutine. */
112+
function settlesRoot(event: DurableEvent): boolean {
113+
return event.type === "close" && event.coroutineId === ROOT_COROUTINE;
114+
}
115+
116+
/**
117+
* Partition one complete physical prefix into the entries it holds.
118+
*
119+
* The whole prefix, atomically: a file whose boundaries are not boundaries
120+
* comes back as `Err` with no segments at all, because a partial partition is a
121+
* catalog that omits whichever entry stopped parsing, and a reader cannot tell
122+
* that from a history that never held it.
123+
*
124+
* Every refusal names what the records say rather than quoting them. A history
125+
* holds whatever a person's own documents held.
126+
*/
127+
export function partitionEntrySegments(
128+
events: readonly DurableEvent[],
129+
): Result<readonly EntrySegment[]> {
130+
const segments: EntrySegment[] = [];
131+
let open: { readonly start: number; readonly admission: Yield } | undefined;
132+
133+
for (let index = 0; index < events.length; index++) {
134+
const event = events[index];
135+
const admission = rootAdmission(event);
136+
137+
if (admission !== undefined) {
138+
if (admission.coroutineId !== ROOT_COROUTINE) {
139+
// An entry's root document runs on the root coroutine. A root admitted
140+
// from inside somebody's work is an entry nested in an entry, which is
141+
// not a boundary this history could have been written with.
142+
return Err(
143+
new ReplSegmentError(
144+
"this journal admits an entry from inside work another entry had already started. " +
145+
"An entry's source is the first thing its own history decides.",
146+
),
147+
);
148+
}
149+
if (open !== undefined) {
150+
return Err(
151+
new ReplSegmentError(
152+
"this journal admits a second entry before the first one settled. One entry runs at " +
153+
"a time, and the entry before it ends with its own outcome.",
154+
),
155+
);
156+
}
157+
open = { start: index, admission };
158+
continue;
159+
}
160+
161+
if (settlesRoot(event)) {
162+
if (open === undefined) {
163+
return Err(
164+
new ReplSegmentError(
165+
"this journal records an entry settling that it never admitted. An entry's source is " +
166+
"admitted before anything it does.",
167+
),
168+
);
169+
}
170+
segments.push(segment(events, segments.length + 1, open, index + 1, true));
171+
open = undefined;
172+
continue;
173+
}
174+
175+
if (open === undefined) {
176+
// Two different damaged histories, and a reader repairing one needs to
177+
// know which they have: nothing has ever been admitted here, or something
178+
// was and the record landed past its outcome.
179+
return Err(
180+
new ReplSegmentError(
181+
segments.length === 0
182+
? "this journal records work before it admitted its entry. The entry's source is the " +
183+
"first thing an execution decides."
184+
: "this journal records work after the entry settled and before another was " +
185+
"admitted. A settled entry is the end of its own history.",
186+
),
187+
);
188+
}
189+
}
190+
191+
if (open !== undefined) {
192+
segments.push(segment(events, segments.length + 1, open, events.length, false));
193+
}
194+
return Ok(Object.freeze(segments));
195+
}
196+
197+
function segment(
198+
events: readonly DurableEvent[],
199+
order: number,
200+
open: { readonly start: number; readonly admission: Yield },
201+
end: number,
202+
settled: boolean,
203+
): EntrySegment {
204+
return Object.freeze({
205+
key: entryKey(order),
206+
order,
207+
start: open.start,
208+
end,
209+
settled,
210+
events: Object.freeze(events.slice(open.start, end)),
211+
admission: open.admission,
212+
});
213+
}
214+
215+
/** What range of the physical stream one execution may read and write. */
216+
export interface EntrySegmentView {
217+
/** The events this entry's range already holds, in append order. */
218+
readonly retained?: readonly DurableEvent[];
219+
/** Whether nothing follows this range in the physical stream. */
220+
readonly final?: boolean;
221+
}
222+
223+
/**
224+
* The one entry segment an execution is given, over the one physical stream.
225+
*
226+
* `readAll()` answers with this range and never with the file, so replay
227+
* correlates this entry's records against this entry's work and cannot reach a
228+
* repeated coroutine id from somebody else's. `append()` goes to the physical
229+
* stream first and is remembered only once that stream has acknowledged it —
230+
* the same order the physical stream itself keeps, and for the same reason: a
231+
* local record of a write that was never retained is a view describing a history
232+
* the file does not hold. After a failed append there is nothing to roll back,
233+
* because nothing moved.
234+
*
235+
* No physical position crosses this boundary. The execution learns how much of
236+
* its own range it has, which is all replay needs; where that range sits in the
237+
* file is the command's business, and its observer already knows the whole
238+
* acknowledged prefix.
239+
*
240+
* Writing is refused rather than ordered. A settled range is over — its entry
241+
* produced an outcome — and a range something already follows is not where the
242+
* next record goes. Either append would interleave two entries in one file, so
243+
* neither is answered with a position.
244+
*/
245+
export class EntrySegmentStream implements DurableStream {
246+
private readonly physical: DurableStream;
247+
private readonly snapshot: DurableEvent[];
248+
private readonly last: boolean;
249+
private closed: boolean;
250+
251+
constructor(physical: DurableStream, view: EntrySegmentView = {}) {
252+
this.physical = physical;
253+
this.snapshot = (view.retained ?? []).map(clone);
254+
this.last = view.final ?? true;
255+
// Read off the range rather than taken as a claim: what settles an entry is
256+
// its own terminal close, and the records say whether it holds one.
257+
const end = this.snapshot[this.snapshot.length - 1];
258+
this.closed = end !== undefined && settlesRoot(end);
259+
}
260+
261+
/** Whether this range holds its entry's terminal close. */
262+
get settled(): boolean {
263+
return this.closed;
264+
}
265+
266+
// deno-lint-ignore require-yield
267+
*readAll(): Operation<DurableEvent[]> {
268+
return this.snapshot.map(clone);
269+
}
270+
271+
*append(event: DurableEvent): Operation<void> {
272+
if (!this.last) {
273+
throw new ReplSegmentError(
274+
"this entry is not the last one in its history, so nothing can be added to it. A later " +
275+
"entry already follows it.",
276+
);
277+
}
278+
if (this.closed) {
279+
throw new ReplSegmentError(
280+
"this entry has already settled, so nothing can be added to it. A settled entry is the " +
281+
"end of its own history.",
282+
);
283+
}
284+
const recorded = clone(event);
285+
286+
yield* this.physical.append(recorded);
287+
288+
// A copy of its own, because the physical stream may retain the object it
289+
// was handed: two records of one event that share a graph are one record
290+
// either side could edit.
291+
this.snapshot.push(clone(recorded));
292+
if (settlesRoot(recorded)) {
293+
this.closed = true;
294+
}
295+
}
296+
}
297+
298+
function clone(event: DurableEvent): DurableEvent {
299+
return structuredClone(event);
300+
}

0 commit comments

Comments
 (0)