Skip to content

Add server-side orchestration engine with event sourcing - #89

Merged
juliusmarminge merged 99 commits into
mainfrom
codething/orchestration-engine
Feb 26, 2026
Merged

Add server-side orchestration engine with event sourcing#89
juliusmarminge merged 99 commits into
mainfrom
codething/orchestration-engine

Conversation

@juliusmarminge

@juliusmarminge juliusmarminge commented Feb 22, 2026 •
edited by cursor Bot
Loading

Copy link
Copy Markdown
Member

Summary

  • Introduces event-sourced orchestration engine on the server with SQLite-backed persistence
  • Moves state management from client to server for better reliability and consistency
  • Adds Effect-based PubSub to push read-model updates to connected clients in real-time
  • Defines typed orchestration schemas (commands, events, read model) in contracts package
  • Simplifies client-side store by removing legacy client reducers/persistence paths

Architecture: Core Data Flow

flowchart TD
  client[WebClientUI] -->|dispatchCommand| ws[WebSocketGateway]
  ws --> orchestrator[OrchestrationEngine]
  orchestrator --> queue[EffectQueue]
  queue --> mapper[CommandToDomainEvent]
  mapper --> store[SQLiteAppendOnlyEventStore]
  store --> reducer[DeterministicReducer]
  reducer --> readmodel[AuthoritativeReadModel]
  readmodel --> pubsub[EffectPubSubFanout]
  pubsub --> ws
  ws -->|onReadModel snapshot/update| client

  provider[ProviderManagerAdapter] --> orchestrator
  git[GitManagerAdapter] --> orchestrator
  projects[ProjectRegistryAdapter] --> orchestrator
Loading

Core Flow Notes

  • Client sends intent as commands only; no client-side domain reducer is authoritative.
  • Server validates/handles command, emits domain events, and appends to SQLite event log.
  • Deterministic reducer projects current read model from events.
  • Read-model snapshots/updates are fanned out to all connected clients over orchestration channels.
  • On restart/reconnect, replay from event log reconstructs the same authoritative state.

Service Modules and Responsibilities

Module Responsibility
packages/contracts/src/orchestration.ts Shared schema contracts for commands, domain events, read-model payloads, orchestration WS methods/channels.
apps/server/src/orchestration/engine.ts Command orchestration runtime (queueing, dispatch, event publication, subscriptions, replay/snapshot APIs).
apps/server/src/orchestration/eventStore.ts SQLite append-only event persistence with sequence ordering and replay reads.
apps/server/src/orchestration/reducer.ts Deterministic projection logic from domain events to authoritative read model.
apps/server/src/orchestration/uiContractInventory.ts Enumerates UI data contract expectations to keep projections aligned with rendered UX.
apps/server/src/wsServer.ts WS transport gateway: routes orchestration RPCs, pushes read-model updates, bridges adapter events into orchestration commands.
apps/server/src/providerManager.ts Provider adapter lifecycle (session/turn execution), feeding orchestration as an external integration source.
apps/server/src/gitManager.ts Git/worktree adapter operations, with status/actions reflected into orchestration read models.
apps/server/src/projectRegistry.ts Project registry adapter for project CRUD backing orchestration project domain events.
apps/web/src/wsNativeApi.ts Thin client transport wrapper exposing orchestration RPC/subscribe surface to UI.
apps/web/src/store.ts UI-state-focused store (layout/preferences) and server read-model hydration (SYNC_SERVER_READ_MODEL).
apps/web/src/routes/__root.tsx Root subscription point for orchestration snapshot + live read-model updates.

Test plan

  • Verify server starts and orchestration engine initializes
  • Test project/thread CRUD operations via WebSocket command dispatch
  • Confirm read-model updates are pushed to clients and reflected in UI
  • Check event replay produces deterministic state on server restart
  • Validate reconnect behavior with snapshot + subsequent updates

🤖 Generated with Claude Code

Open with Devin

Note

Medium Risk
Adds new Effect-native Git checkpointing, diff computation, and end-to-end orchestration/provider integration tests; changes touch persistence/runtime wiring and WebSocket robustness, which can affect core server behavior if misconfigured.

Overview
Adds an Effect-native checkpointing stack: a new CheckpointStoreLive that captures/restores/diffs checkpoints via Git refs, plus a CheckpointDiffQueryLive service that validates projection state + filesystem refs before serving orchestration.getTurnDiff results.

Introduces diff parsing utilities (parseTurnDiffFilesFromUnifiedDiff) and expands server/integration coverage with a deterministic provider adapter harness and full end-to-end orchestration engine tests that exercise turns, approvals, checkpoint capture/diffing, and revert behavior against real git workspaces + sqlite.

Updates tooling/docs and build wiring: CI now reads Bun/Node versions from package.json, server build switches bundling script to TS, adds Effect/SQLite/diff dependencies and Vitest setup, tightens desktop preload typing via DesktopBridge, and tweaks docs to reflect orchestration push channel usage and remove deprecated provider RPC mentions.

Written by Cursor Bugbot for commit 742e127. This will update automatically on new commits. Configure here.

Summary by CodeRabbit

Release Notes

  • New Features

    • Server orchestration engine with persistent event storage and asynchronous command processing
    • New orchestration API methods: getSnapshot, dispatchCommand, replayEvents, onReadModel, onDomainEvent
    • Server-driven read-model synchronization replacing local event handling

  • Removals

    • Removed provider.listSessions and provider.onEvent methods
    • Removed local browser persistence schema

  • Dependencies

    • Added Effect library dependencies for server runtime

@juliusmarminge @claude
Add server-side orchestration engine with event sourcing
2001d39 
Move state management from client to server using an event-sourced architecture:
- Add SQLite-backed event store with append-only event log
- Add orchestration engine with command queue, reducer, and read model
- Add Effect-based PubSub for real-time read model updates to clients
- Define orchestration schemas in contracts (commands, events, read model)
- Simplify client-side store by removing persistence layer
- Remove obsolete client-side persistence tests

Co-Authored-By: Claude Opus 4.5 <[email protected]>
@coderabbitai

coderabbitai Bot commented Feb 22, 2026 •
edited
Loading

Copy link