Skip to content

RFC: Optional Turso backend for branching, real-time sync, and vector search #2

@gunta

Description

@gunta

Proposal
Keep Beads exactly as-is for the default path (JSONL in Git as source of truth + local SQLite cache). Add an optional Turso/libSQL backend that unlocks:

  • Database Branching for safe experiments and big refactors
  • Near-real-time sync between agents and humans via embedded replicas
  • Native Vector Search for semantic issue lookup
  • Browser-ready persistence (WASM/OPFS) for future web UIs/agent workbenches
  • Client-side encryption at rest (embedded replicas)
  • Concurrent writes beyond SQLite’s single-writer limit

This preserves Beads’ zero-setup workflow and Git-first philosophy while offering a scalable, collaboration-friendly path when teams (or lots of agents) need it.


Motivation

Beads’ current dual-storage design (JSONL in Git + SQLite cache) is elegant and low-friction. Where it may start to creak is multi-agent, multi-machine collaboration and AI-assisted retrieval:

  • Multiple writers can churn Git and create merge friction.
  • Some tasks need shared state fast (without a Git merge queue, we live in a fast world now).
  • AI/agent workflows benefit from vector similarity search over issues/notes/context.
  • A future Beads browser UI would be much simpler if the DB can run locally in the browser and sync when online.

Turso gives us an opt-in path to these capabilities while retaining SQLite compatibility and the local-first ethos.


Proposal (opt-in; default remains unchanged)

  1. Introduce a storage driver interface

    • Keep the existing sqlite+git path as the default driver.
    • Add a turso driver that can be enabled via config/env.
  2. Two connection modes

    • Local-first (embedded replica): keep .beads/local.db for fast offline reads/writes; periodically sync with a remote Turso primary (read-your-writes semantics).
    • Remote-only: connect directly to the Turso database (useful for CI, headless agents, shared runners).
  3. Database Branching as a workflow primitive

    • Map Beads branch commands to DB branches for safe, isolated change sets:
      • bd branch create <name> → create branch from main
      • bd branch switch <name> → point the driver at that branch
      • bd branch merge <name> → controlled merge back to main
    • Use branches for big schema/issue reshapes; use normal sync for routine edits.
  4. Native Vector Search for issues

    • Add an embeddings table (or a vector column on issues) behind a feature flag.
    • Provide bd search --similar "text" when the turso driver is active.
  5. Browser-ready

    • Allow a WASM/OPFS replica for a web UI / agent workbench that runs entirely in the browser and syncs when connectivity returns.
  6. GUI client

    • Document a one-click connection via Outerbase Studio for folks who want a visual client right now.

Migration (optional, one-time)

Projects that want Turso can do:

# 1) (Optional) Import existing SQLite data into a new Turso DB
turso db import .beads/beads.db

# 2) Configure credentials (local dev or CI)
export TURSO_DATABASE_URL="https://<db>.turso.io"
export TURSO_AUTH_TOKEN="<token>"

# 3) Switch Beads to the turso driver
bd config set storage.driver=turso
# or for new repos:
bd init --driver=turso

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions