Skip to content

feat(db): complete database abstraction layer — eliminate sqlx leaks, fix dynamic queries, security hardening#2392

Merged
bug-ops merged 4 commits intomainfrom
db-abstraction-layer
Mar 29, 2026
Merged

feat(db): complete database abstraction layer — eliminate sqlx leaks, fix dynamic queries, security hardening#2392
bug-ops merged 4 commits intomainfrom
db-abstraction-layer

Conversation

@bug-ops
Copy link
Copy Markdown
Owner

@bug-ops bug-ops commented Mar 29, 2026

Summary

Completes the zeph-db database abstraction layer introduced in PRs #2371#2379. Eliminates all direct sqlx leaks from consumer crates and adds security/performance hardening.

  • Remove direct sqlx dependencies from all 6 consumer crates — all SQL access now routes through zeph-db re-exports
  • Fix 8 dynamic format!()-built queries in graph/store/mod.rs using SQLite-only ? placeholders — now dialect-portable via numbered_placeholder()/placeholder_list() helpers
  • Add acquire_timeout(30s) and min_connections(1) to SQLite pool; fix dead pool_size parameter in postgres pool config
  • Fix get_vectors silently swallowing DB errors; wrap insert_edge_typed in atomic transaction; switch Qdrant hot-path to wait(false) (saves 3–15ms/call)
  • Add importance_score index migration (053) to prevent full table scans
  • PostgreSQL TLS warning when sslmode absent; SQLite file permissions 0o600 on unix
  • Fix all pre-existing clippy warnings exposed by broader workspace compilation
  • Move root binary sqlx dep to [dev-dependencies]

Issues closed

Closes #2386, #2387, #2388, #2389
Part of epic #2385

Architecture decisions

Per pre-implementation review (.local/handoff/db-arch-review.md):

  • #[cfg] type aliases retained — generics migration withdrawn; sqlx compile-time macros (query!, migrate!) are fundamentally monomorphic
  • Qdrant stays in zeph-memory — different concern from relational SQL
  • postgres feature remains opt-in — excluded from full

Test plan

  • cargo +nightly fmt --check — clean
  • cargo clippy --workspace --features full --all-targets -- -D warnings — 0 warnings
  • cargo nextest run --workspace --features full --lib --bins — 7233 passed, 22 skipped
  • cargo check --workspace --features full — clean
  • Live agent session test with testing.toml (post-merge CI)

bug-ops added 4 commits March 29, 2026 21:34
… dynamic queries (#2386)

Remove hardcoded sqlx/sqlite dependencies from zeph-memory, zeph-core,
zeph-index, zeph-mcp, zeph-orchestration, and zeph-scheduler. All
consumer crates now route through zeph-db re-exports.

Add numbered_placeholder() and placeholder_list() helpers to zeph-db
for dialect-agnostic dynamic IN clause construction. Fix 8 dynamic
format!()-built queries in graph/store/mod.rs that used SQLite-only ?
placeholders, making them portable across both backends.

Add MAX_BATCH=490 chunking to mark_messages_graph_processed to prevent
SQLite "too many SQL variables" errors on large batches.

Closes #2386
…2387)

Fix SQLite pool missing acquire_timeout and min_connections. Fix dead
pool_size parameter in postgres connect (was using max_connections).
Replace N+1 SELECT+INSERT in insert_edge_typed with atomic transaction.
Fix get_vectors: use placeholder_list(), propagate errors instead of
swallowing with unwrap_or_default(). Switch Qdrant hot-path upsert to
wait(false) to reduce per-call latency by 3-15ms. Add migration 053
for importance_score index to prevent full table scans. Fix sql! macro
doc comment to accurately describe Box::leak behavior.

Fix all pre-existing clippy warnings exposed by broader workspace
compilation: many_single_char_names, similar_names, float_cmp,
format_collect, items_after_statements, needless_pass_by_value, and
30+ other lint categories across zeph-memory, zeph-tools, zeph-core,
zeph-tui, and root binary.

Closes #2387
…#2388)

Replace direct sqlx::SqlitePool::connect usage in test code (root
binary src/scheduler_executor.rs, src/agent_setup.rs) and crate tests
(zeph-index/watcher.rs, zeph-scheduler/scheduler.rs) with zeph_db::sqlx
re-exports. Move sqlx from [dependencies] to [dev-dependencies] in root
Cargo.toml — no production code in the binary uses sqlx directly.

Closes #2388
@bug-ops bug-ops added enhancement New feature or request P1 High ROI, low complexity — do next sprint architecture Architecture improvements labels Mar 29, 2026
@github-actions github-actions bot added size/XL Extra large PR (500+ lines) documentation Improvements or additions to documentation memory zeph-memory crate (SQLite) rust Rust code changes core zeph-core crate dependencies Dependency updates and removed size/XL Extra large PR (500+ lines) labels Mar 29, 2026
@bug-ops bug-ops merged commit fbf98e8 into main Mar 29, 2026
27 checks passed
@bug-ops bug-ops deleted the db-abstraction-layer branch March 29, 2026 20:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

architecture Architecture improvements core zeph-core crate dependencies Dependency updates documentation Improvements or additions to documentation enhancement New feature or request memory zeph-memory crate (SQLite) P1 High ROI, low complexity — do next sprint rust Rust code changes

Projects

None yet

1 participant