Skip to content

feat(db): introduce zeph-db crate with database abstraction layer (Phase 1, SQLite)#2371

Merged
bug-ops merged 4 commits intomainfrom
feat/m28/db-abstraction-layer
Mar 28, 2026
Merged

feat(db): introduce zeph-db crate with database abstraction layer (Phase 1, SQLite)#2371
bug-ops merged 4 commits intomainfrom
feat/m28/db-abstraction-layer

Conversation

@bug-ops
Copy link
Copy Markdown
Owner

@bug-ops bug-ops commented Mar 28, 2026

Summary

  • Introduces a new zeph-db Layer 0 crate providing compile-time database abstraction over sqlx
  • Migrates all persistence crates to use zeph_db::DbPool and sql!() instead of sqlx::SqlitePool directly
  • Renames backend-specific types (SqliteVectorStoreDbVectorStore, SqliteGraphStoreDbGraphStore, sqlite/ module → store/)
  • Consolidates all 52 migrations into zeph-db/migrations/sqlite/

Architecture

  • Dialect trait with Sqlite/Postgres zero-sized marker types
  • DatabaseDriver trait: SqliteDriver impl in Phase 1, PostgresDriver skeleton for Phase 2
  • ActiveDriver type alias gates backend at compile time with compile_error! mutual exclusivity guard
  • sql!() macro: no-op on SQLite (&'static str), rewrites ? to $N on PostgreSQL
  • DbConfig enforces write_pool_size = 1 for SQLite to prevent SQLITE_BUSY
  • redact_url() strips credentials from connection URLs in error paths

Spec

Full architectural spec (reviewed by critic, perf, and security agents): .local/specs/database-abstraction/spec.md

Test plan

  • cargo check --features full — clean
  • cargo +nightly fmt --check — clean
  • cargo clippy --workspace --features full -- -D warnings — clean
  • cargo nextest run --workspace --features full --lib --bins — 7117/7117 passed
  • Phase 2 (PostgreSQL backend) — separate PR
  • Phase 3 (config tooling, --migrate-config, vault integration) — separate PR

@github-actions github-actions bot added memory zeph-memory crate (SQLite) rust Rust code changes core zeph-core crate tests Test-related changes dependencies Dependency updates enhancement New feature or request size/XL Extra large PR (500+ lines) labels Mar 28, 2026
…ase 1, SQLite)

Adds a new `zeph-db` Layer 0 crate that provides a compile-time database
abstraction over sqlx, currently supporting SQLite with PostgreSQL groundwork.

Architecture:
- `Dialect` trait with `Sqlite` and `Postgres` zero-sized marker types
- `DatabaseDriver` trait unifying pool construction, migrations, and
  transaction semantics; `SqliteDriver` impl in Phase 1
- `ActiveDriver` type alias gates the active backend at compile time
- `DbPool`/`DbRow`/`DbTransaction`/`ActiveDialect` derived from `ActiveDriver`
- `sql!()` macro for portable placeholder syntax (no-op on SQLite, rewrites
  `?` to `$N` on PostgreSQL)
- `DbConfig` with `write_pool_size = 1` default for SQLite single-writer safety
- `redact_url()` strips credentials from connection URLs in error messages
- `compile_error!` guard enforces mutual exclusivity of `sqlite`/`postgres`

Migrations:
- All 50 migrations moved from `zeph-memory/migrations/` to
  `zeph-db/migrations/sqlite/`
- Added `051_scheduler_jobs.sql` and `052_mcp_trust_scores.sql`, replacing
  inline DDL in `zeph-scheduler` and `zeph-mcp`

Consumer crate changes:
- `zeph-memory`: `sqlite/` module renamed to `store/`, `sqlite_vector_store.rs`
  renamed to `db_vector_store.rs`; `SqliteVectorStore` → `DbVectorStore`,
  `SqliteGraphStore` → `DbGraphStore`; 340+ query strings wrapped in `sql!()`
- `zeph-scheduler`: `JobStore::open()` now uses `DbConfig::connect()` for
  WAL/busy_timeout/foreign_keys; inline DDL removed
- `zeph-mcp`, `zeph-orchestration`, `zeph-index`: migrated to `zeph_db::DbPool`
  and `sql!()` macro
- `MemoryError::Sqlite` → `MemoryError::Sqlx` (backend-neutral name)
- `SchedulerError::Db` variant added

All existing tests pass (7117/7117).
@bug-ops bug-ops force-pushed the feat/m28/db-abstraction-layer branch from 8af9c81 to 8442697 Compare March 28, 2026 20:40
…mer crates

- Replace datetime('now') with CURRENT_TIMESTAMP (standard SQL, both backends)
- Add Dialect::COLLATE_NOCASE ("COLLATE NOCASE" / "" for SQLite/Postgres)
- Remove Dialect::NOW (superseded by CURRENT_TIMESTAMP)
- Replace all INSERT OR IGNORE with ActiveDialect::INSERT_IGNORE + CONFLICT_NOTHING
  across db_vector_store, snapshot, mem_scenes, messages, acp_sessions, graph/store
- Replace COLLATE NOCASE literals with ActiveDialect::COLLATE_NOCASE in graph/store
- Migrate sanitize_fts_query callers to zeph_db::fts; remove local duplicate
- Fix activate_skill_version to use begin_write() instead of pool.begin()
- Remove dead MemoryError::Migration variant
- Add default = ["sqlite"] to zeph-db/Cargo.toml for standalone cargo check
- Fix DatabaseDriver trait: remove run_migrations/begin/begin_write (free fns only)
- Run rustfmt across all modified files

Tests: 7148/7148 passed
@github-actions github-actions bot added the documentation Improvements or additions to documentation label Mar 28, 2026
@bug-ops bug-ops enabled auto-merge (squash) March 28, 2026 21:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

core zeph-core crate dependencies Dependency updates documentation Improvements or additions to documentation enhancement New feature or request memory zeph-memory crate (SQLite) rust Rust code changes size/XL Extra large PR (500+ lines) tests Test-related changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant