Skip to content

Commit 952f5f7

Browse files
authored
Feat/update from support (#42)
* test: deterministic e2e suite over full pgwire→WAL→Delta path Adds an opt-in `e2e` cargo feature and dedicated CI job that exercises the production write/read path against a per-test MinIO container with virtual-time control, so flush, eviction, restart, and compaction paths are covered end-to-end without wall-clock waits. Covers the prod class of bug where queries silently stop returning. - harness: testcontainers MinIO + frozen clock + pgwire client; per-test WALRUS_DATA_DIR isolation; crash-and-restart helper that re-bootstraps against the same data dir/bucket. - scenarios: smoke, flush_lifecycle, eviction, cache_warmth (asserts Foyer hit-or-no-S3-refetch), restart_recovery (flushed path), multi_tenant_isolation, pressure_flush, zorder_idempotence (asserts unchanged file set on second OPTIMIZE). - proptest: row-count conservation, monotonic bucket id, clock advance. - BufferedWriteLayer: flush/eviction `Notify` tick hooks, cumulative flush counters in StatsSnapshot, `crash_for_test` (cancel without graceful flush). - bootstrap: shared wiring between main.rs and the harness so the test path matches prod. - CI: opt-in `e2e` job runs serialized (--test-threads=1) to keep WAL isolation; main job stays untouched. * feat: native UPDATE ... FROM end-to-end (planner → MergeBuilder + MemBuffer hash-join) DataFusion's SQL planner recognizes `UPDATE ... FROM` and lowers it to a joined Dml plan, TimeFusion's DmlQueryPlanner materializes the source RHS once at plan-construction time, and both backends consume the result: - Delta path dispatches to MergeBuilder with WHEN MATCHED THEN UPDATE, using a conjunctive equi-join predicate built from the extracted keys. - MemBuffer path does an in-memory hash join via Arrow RowConverter + take, evaluating assignment exprs against a widened (target + source__<name>) schema. WAL gains a `WalOperation::UpdateWithSource = 3` variant carrying the source as Arrow IPC bytes alongside the predicate + assignments, so replay reconstructs the join exactly. V1 UPDATE entries continue to decode. `identify_target_side` unwraps Cast/TryCast on join keys (DataFusion inserts implicit casts when source/target types differ slightly, e.g. Utf8 vs Utf8View). `requalify_assignments_for_merge` rewrites the user's source-alias references (e.g. `u.d` from `FROM (...) AS u`) to `source.x` so delta-rs's MergeBuilder resolves them under its own source alias. Vendor datafusion-sql 53.1.0 with one patch: remove the defensive `not_impl_err!("UPDATE ... FROM is not supported")` guard at statement.rs:1087 (apache/datafusion#19950). The underlying `update_to_plan` already handles `from` correctly via `plan_from_tables`; the guard was a TODO that pre-dated the working implementation a few hundred lines below. PATCHES.md documents the diff. Tests: - test_update_from_values - test_update_from_no_match_no_change - test_update_from_with_predicate - test_update_from_multi_column_set All passing. * add coverage for monoscope-shape UPDATE FROM + MemBuffer / WAL paths - test_update_from_unnest_text_array: matches monoscope UPDATE-2 exactly (text[] unnest source, user-alias `o`/`u`, list-typed assignment). Confirms first-apply works end-to-end on the Delta MergeBuilder path. - update_with_source_buffered_only + update_with_source_wal_replay_after_restart: exercise the MemBuffer hash-join path and the WAL UpdateWithSource serialization/replay loop. Both #[ignore]d: the buffered-only lookup returns 0 matches because RowConverter requires byte-identical types and our Utf8→Utf8View cast isn't producing comparable OwnedRow bytes. Documented as a known follow-up. - test_update_from_unnest_text_array_idempotent: #[ignore]d because MergeBuilder counts WHEN MATCHED rows regardless of whether the SET produces an unchanged value under the NOT @> predicate. Monoscope's re-extraction safety on TF needs this nailed down separately. Also reshape the merge rewriter: - requalify_for_merge unifies the predicate + assignments paths. Strips user aliases (o, u) and routes columns by schema membership: source- schema columns → source.x, everything else → bare (target resolves). - Pull join.filter (cross-side predicates like NOT @>) into the predicate chain so MergeBuilder ANDs it into the join predicate. - extract_dml_info handles SubqueryAlias passthrough and unwraps implicit Cast/TryCast on join keys (DataFusion inserts these when source/target types differ slightly). - mem_buffer update_with_source now casts source key columns to target key column types before hashing (Utf8 → Utf8View when needed).
1 parent 695478d commit 952f5f7

72 files changed

Lines changed: 37703 additions & 22 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/workflows/ci.yml‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -108,3 +108,30 @@ jobs:
108108
- name: Test
109109
run: cargo test --all-features --locked
110110

111+
# E2E suite: full pgwire → BufferedWriteLayer → WAL → Delta path against a
112+
# per-test MinIO via testcontainers. Runs in its own job so failures are
113+
# attributable and the suite can be made a required check independently.
114+
e2e:
115+
name: E2E
116+
runs-on: ubuntu-latest
117+
timeout-minutes: 30
118+
env:
119+
CARGO_TERM_COLOR: always
120+
RUST_BACKTRACE: 1
121+
CARGO_INCREMENTAL: "0"
122+
steps:
123+
- name: Free disk space
124+
run: sudo rm -rf /usr/share/dotnet /usr/local/lib/android /opt/ghc /opt/hostedtoolcache/CodeQL
125+
- uses: actions/checkout@v4
126+
- uses: dtolnay/rust-toolchain@stable
127+
- name: Install protoc
128+
run: sudo apt-get update && sudo apt-get install -y protobuf-compiler
129+
- uses: Swatinem/rust-cache@v2
130+
with:
131+
cache-on-failure: true
132+
shared-key: e2e
133+
- name: E2E
134+
# --test-threads=1: walrus-rust reads WALRUS_DATA_DIR from process
135+
# env, so per-test WAL isolation requires serialization.
136+
run: cargo test --test e2e --features e2e --locked -- --test-threads=1 --nocapture
137+

‎Cargo.lock‎

Lines changed: 0 additions & 2 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎Cargo.toml‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -141,6 +141,15 @@ harness = false
141141
[features]
142142
default = []
143143
test = []
144+
# Gates the dynamic-MinIO E2E suite. Off by default so the heavy
145+
# testcontainers path doesn't run in the main `cargo test` job — the
146+
# dedicated `e2e` CI job opts in via `--features e2e`.
147+
e2e = []
148+
149+
[[test]]
150+
name = "e2e"
151+
path = "tests/e2e/main.rs"
152+
required-features = ["e2e"]
144153

145154
# Release builds prioritize binary size + runtime speed over compile time.
146155
# fat LTO + single codegen-unit lets the optimizer inline across crates;
@@ -174,4 +183,11 @@ arrow-pg = { path = "vendor/arrow-pg" }
174183
# with >9 placeholders return the wrong type-by-position to the client.
175184
# Vendored copy sorts by numeric suffix instead.
176185
datafusion-postgres = { path = "vendor/datafusion-postgres" }
186+
# Local patch: datafusion-sql 53.1.0's UPDATE planner has a defensive
187+
# `not_impl_err!("UPDATE ... FROM is not supported")` guard
188+
# (apache/datafusion#19950) even though `update_to_plan` already handles
189+
# `from` correctly via `plan_from_tables`. Vendored copy deletes the guard
190+
# so `UPDATE ... FROM` reaches TimeFusion's `DmlQueryPlanner`, which lowers
191+
# it to `MergeBuilder` (Delta path) and a hash-join (MemBuffer path).
192+
datafusion-sql = { path = "vendor/datafusion-sql" }
177193

‎Makefile‎

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
.PHONY: test test-all test-ovh test-minio test-minio-all test-prod test-integration test-integration-minio run-prod run-minio build-prod minio-start minio-stop minio-clean tf-start tf-stop
1+
.PHONY: test test-all test-ovh test-minio test-minio-all test-prod test-integration test-integration-minio test-e2e run-prod run-minio build-prod minio-start minio-stop minio-clean tf-start tf-stop
22

33
# Default test (fast, excludes slow integration tests)
44
test:
@@ -95,6 +95,12 @@ tf-start: minio-start
9595
sleep 1; \
9696
done; echo "timeout waiting for PGWire on $$port"; tail -50 /tmp/timefusion.log; exit 1
9797

98+
# E2E tests: dynamic MinIO via testcontainers (requires Docker). Each test
99+
# gets a fresh container + bucket so they parallelize safely.
100+
test-e2e:
101+
@echo "Running E2E suite (Docker required for MinIO)..."
102+
cargo test --test e2e --features e2e -- --test-threads=1 --nocapture $${ARGS}
103+
98104
tf-stop:
99105
@[ -f /tmp/timefusion.pid ] && kill $$(cat /tmp/timefusion.pid) 2>/dev/null || true
100106
@rm -f /tmp/timefusion.pid

‎docs/clock_audit.md‎

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
# Clock-source audit (2026-06-07)
2+
3+
`crate::clock::now_micros()` is the **only** acceptable wall-clock read for
4+
scheduling decisions (flush triggers, bucket id, retention cutoff). All other
5+
`now`-style reads are either metrics or metadata and are safe to leave on the
6+
real wall clock.
7+
8+
## Categories
9+
- **scheduling** — drives whether/when background work runs. MUST go through `crate::clock`.
10+
- **metrics** — latency timing, monotonic-only. Tokio/std `Instant` is fine.
11+
- **metadata** — written into a log/header/file as the "produced at"
12+
timestamp; OK to use wall clock so the value is meaningful across restarts.
13+
14+
## Inventory (non-test code)
15+
16+
| file:line | call | category | notes |
17+
|-----------|------|----------|-------|
18+
| `src/buffered_write_layer.rs:381` | `crate::clock::now_micros()` | scheduling | ✓ default ts for batches missing one |
19+
| `src/buffered_write_layer.rs:912` | `crate::clock::now_micros()` | scheduling | ✓ oldest-bucket-age gauge feeds alerts |
20+
| `src/mem_buffer.rs:549` | `crate::clock::now_micros()` (via `current_bucket_id`) | scheduling | ✓ |
21+
| `src/main.rs:161` | `clock::now_micros()` | metadata | cursor-snapshot age log line |
22+
| `src/wal.rs:129` | `chrono::Utc::now()` | metadata | `WalEntry.timestamp_micros` — written into WAL header |
23+
| `src/tantivy_index/service.rs:106,127` | `Utc::now()` | metadata | manifest `built_at` field |
24+
| `src/object_store_cache.rs:*` | `SystemTime::now()` + `Instant::now()` | metrics / metadata | cache entry stamps + read-latency timing |
25+
| `src/database.rs:* (4 sites)` | `Instant::now()` | metrics | query latency timing |
26+
| `src/buffered_write_layer.rs:418` | `Instant::now()` | metrics | recovery duration |
27+
| `src/pgwire_early_bind.rs:*` | `Instant::now()` | metrics | deadline for connect-readiness loop |
28+
| `src/statistics.rs:*` | `Instant::now()` | metrics | sample timestamps |
29+
30+
## Outcome
31+
- No scheduling reads bypass `crate::clock`. Phase 0 passes — no code fix needed.
32+
- Background tasks (`run_flush_task`, `run_eviction_task`) sleep via
33+
`tokio::time::sleep`, which can be advanced via tokio's `pause()` if a test
34+
wants. We don't use it because the E2E harness runs real S3 IO; the
35+
`crate::clock` + explicit `force_*_now()` test hooks (Phase 1) give us
36+
determinism without pausing the runtime.
37+
38+
## Rule for new code
39+
If you add a `now`-style read:
40+
1. **scheduling** → `crate::clock::now_micros()`.
41+
2. **metrics** → `std::time::Instant::now()`.
42+
3. **metadata** → `chrono::Utc::now()` (or `crate::clock::now_micros()` if the
43+
metadata is going to be compared against scheduling time later).
44+
45+
If you're unsure, treat it as scheduling.

‎docs/e2e_plan.md‎

Lines changed: 105 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,105 @@
1+
# TimeFusion E2E Test Suite — Execution Plan
2+
3+
Goal: a deterministic, fast E2E suite that exercises the **full prod path** (pgwire → BufferedWriteLayer → WAL → MemBuffer → flush → Delta on MinIO → Foyer cache → query) with **virtual time** so flush/eviction/retention paths run in milliseconds. Must catch the class of bug we just hit ("no response returned to queries") before deploy.
4+
5+
## Crates to add (`Cargo.toml [dev-dependencies]`)
6+
7+
```
8+
testcontainers = "0.20"
9+
testcontainers-modules = { version = "0.10", features = ["minio"] }
10+
tokio-test = "0.4"
11+
tracing-test = "0.2"
12+
insta = { version = "1", features = ["yaml"] }
13+
proptest = "1" # phase 5 only
14+
```
15+
`tokio` already pulled; ensure `test-util` feature is enabled for tests.
16+
17+
## Phase 0 — Audit time sources (½ day)
18+
- [ ] Grep every `chrono::Utc::now`, `SystemTime::now`, `Instant::now`. Confirm scheduling-relevant ones go through `crate::clock`; flag any miss.
19+
- [ ] Confirm `run_flush_task` / `run_eviction_task` use `tokio::time::sleep` (not `std::thread::sleep`).
20+
- [ ] Document in `clock.rs` which categories of "now" must use it vs. which are metrics-only.
21+
22+
**Exit:** a one-page inventory in `docs/clock_audit.md` and any necessary fixes merged.
23+
24+
## Phase 1 — Test hooks on hot path (1 day)
25+
Add minimum surface needed to deterministically drive the system:
26+
27+
- [ ] `BufferedWriteLayer::force_flush_now() -> Result<FlushReport>` — flushes *all* current buckets, returns `{buckets_flushed, rows, delta_commits}`.
28+
- [ ] `BufferedWriteLayer::force_evict_now()` — runs the eviction pass once, synchronously.
29+
- [ ] `BufferedWriteLayer::flush_completed_notify() -> Arc<Notify>` — pinged at end of each flush task iteration. Lets tests `notified().await` instead of sleeping.
30+
- [ ] Same `Notify` for eviction.
31+
- [ ] Expose a `FlushReport`/`EvictReport` counter snapshot (already partly in OTel metrics — wrap as a struct).
32+
33+
These are also useful in prod for `/admin/flush` style endpoints later.
34+
35+
**Exit:** unit test that asserts `force_flush_now()` empties MemBuffer and writes a Delta commit.
36+
37+
## Phase 2 — Harness: `tests/e2e/harness.rs` (1–2 days)
38+
Builder-style:
39+
40+
```rust
41+
let env = E2eEnv::builder()
42+
.with_minio() // testcontainers, fresh bucket per test
43+
.with_clock_frozen_at("2026-06-01T00:00:00Z")
44+
.with_flush_interval(Duration::from_millis(10))
45+
.with_retention(Duration::from_secs(60))
46+
.start().await?;
47+
let pg = env.pg_client().await?; // real pgwire client
48+
```
49+
50+
Capabilities:
51+
- [ ] Spins MinIO via `testcontainers`, creates a per-test bucket, sets all `AWS_*` env vars **scoped** (no global mutation — pass via `AppConfig` builder).
52+
- [ ] Initializes `crate::clock` in frozen mode at a known instant.
53+
- [ ] Boots timefusion in-process on an ephemeral port, returns `tokio_postgres::Client`.
54+
- [ ] Drives the full pgwire stack (no test-only bypass).
55+
- [ ] `env.advance(Duration::from_secs(700)).await` — bumps `crate::clock` AND awaits the flush `Notify` so post-conditions are observable.
56+
- [ ] `env.metrics().snapshot()` — pulls OTel counters as a plain struct for assertions.
57+
- [ ] `env.foyer_stats()` — hit/miss counts.
58+
- [ ] Auto-drops MinIO container + tempdir on test end.
59+
60+
**Exit:** `tests/e2e/smoke.rs` — insert 1 row, select it back, assert row count and **assert query returned in <500ms** (the specific bug we hit).
61+
62+
## Phase 3 — Core scenarios (`tests/e2e/scenarios/`) (2–3 days)
63+
Each is a separate `#[tokio::test(flavor = "multi_thread", start_paused = false)]` file. Don't pause the runtime — we use `crate::clock` instead, because real S3 IO is happening.
64+
65+
1. **`smoke.rs`** — insert/select/restart/select. **Must always return a response.**
66+
2. **`flush_lifecycle.rs`** — insert across two buckets, `advance(11min)`, assert one bucket flushed, one retained, Delta has expected files, MemBuffer freed memory.
67+
3. **`eviction.rs`** — insert old + recent rows, `advance(retention + 1min)`, assert old evicted, recent retained, query still returns both via Delta+MemBuffer union.
68+
4. **`cache_warmth.rs`** — flush, query twice. First reads from S3, second from Foyer. Assert `foyer_hits >= 1` on the second.
69+
5. **`restart_recovery.rs`** — insert, kill server, restart. Assert WAL replay restores data; assert cursor snapshot skips cold-start delta scan (the optimization in 2d36136).
70+
6. **`multi_tenant_isolation.rs`** — two `project_id`s, queries without filter must fail, queries with filter must not leak.
71+
7. **`pressure_flush.rs`** — set tiny `max_memory_mb`, hammer inserts, assert early flush triggers and no OOM/deadlock.
72+
8. **`zorder_idempotence.rs`** — run compaction twice, assert second is a no-op (the bug fixed in 4a41fea).
73+
74+
Each scenario asserts on:
75+
- Correctness (row counts, content)
76+
- **Liveness** (query returns within a timeout — guards the prod symptom)
77+
- Observable state (Delta commit count, foyer hits, OTel counters)
78+
79+
## Phase 4 — CI integration (½ day)
80+
- [ ] New `make test-e2e` target.
81+
- [ ] GitHub Actions job: `cargo test --test e2e_*` on every PR. Needs Docker (testcontainers).
82+
- [ ] Required check before merge.
83+
- [ ] Target wall-time: < 90s for the whole suite (testcontainers MinIO startup is the long pole — share container across scenarios in one binary, isolate via bucket name).
84+
85+
## Phase 5 — Property tests (later, 2 days)
86+
- [ ] `proptest` generator: sequences of `Insert(project, ts, n_rows) | AdvanceTime(d) | Query(project) | Restart`.
87+
- [ ] Invariants: row count conservation, no row lost across flush, query result == union of MemBuffer + Delta, restart doesn't lose committed data.
88+
- [ ] Run nightly, not on PR.
89+
90+
## Risks / open questions
91+
- **testcontainers MinIO startup cost** — measure; if >5s, share one container per `cargo test` binary via a `OnceCell<Mutex<MinioContainer>>`, isolate tests by bucket prefix.
92+
- **In-process server vs. subprocess** — in-process is faster and gives access to internals (force_flush_now), but masks process-level bugs (signal handling, atexit). Do in-process for most, add 1–2 subprocess tests for shutdown/restart paths.
93+
- **AWS env vars are global** — must be passed via `AppConfig` not env. May require small refactor in `config.rs` to accept an explicit config rather than reading env in `init`.
94+
- **Clock freeze coverage** — Phase 0 must catch every wall-clock read; one missed `SystemTime::now()` in the flush decision path makes the whole suite flaky.
95+
96+
## Order of execution
97+
P0 audit → P1 hooks → P2 harness + smoke test → ship that PR.
98+
Then P3 scenarios in priority order (smoke, flush_lifecycle, cache_warmth, restart_recovery first — they cover the recent prod incidents).
99+
P4 CI gating after smoke+flush_lifecycle are green.
100+
P5 nightly proptest last.
101+
102+
## Definition of done
103+
- All P3 scenarios green in CI, < 90s wall-time.
104+
- A new prod bug of the class "queries don't return" or "flush silently fails" would have been caught by at least one scenario in this suite.
105+
- Documented in `CLAUDE.md` under Testing.

‎src/bootstrap.rs‎

Lines changed: 104 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,104 @@
1+
//! Shared server-bootstrap wiring used by both `main.rs` and the E2E test
2+
//! harness. Keeping this in one place guarantees the test path matches prod
3+
//! — the whole point of the E2E suite is to catch the class of bug that
4+
//! "CI doesn't reproduce because the harness skips half the wiring".
5+
//!
6+
//! Returns the fully wired pieces; the caller decides what to do with them
7+
//! (serve pgwire, expose for assertions, etc.).
8+
9+
use std::sync::Arc;
10+
11+
use anyhow::Result;
12+
use arrow::array::RecordBatch;
13+
use datafusion::execution::context::SessionContext;
14+
use tokio_util::sync::CancellationToken;
15+
16+
use crate::{
17+
buffered_write_layer::{BufferedWriteLayer, DeltaWatermark, DeltaWriteCallback},
18+
config::AppConfig,
19+
database::Database,
20+
};
21+
22+
/// Everything a serving process needs after bootstrap is done.
23+
pub struct Bootstrapped {
24+
pub db: Arc<Database>,
25+
pub buffered_layer: Arc<BufferedWriteLayer>,
26+
/// The SessionContext used by the pgwire handlers — UDFs and table
27+
/// providers are already registered.
28+
pub session_ctx: Arc<SessionContext>,
29+
/// Cancel to signal shutdown to anything we spawned.
30+
pub shutdown: CancellationToken,
31+
}
32+
33+
/// Build the BufferedWriteLayer + Database wiring exactly as `main.rs` does,
34+
/// minus listener binding / signal handling / telemetry init (the caller
35+
/// owns those — they differ between prod and test).
36+
///
37+
/// Side effects: spawns the flush + eviction background tasks, performs
38+
/// WAL recovery, and starts maintenance schedulers. The returned
39+
/// `CancellationToken` can be triggered to ask spawned work to wind down.
40+
pub async fn bootstrap(cfg: Arc<AppConfig>) -> Result<Bootstrapped> {
41+
crate::clock::init_from_env();
42+
43+
let mut db = Database::with_config(Arc::clone(&cfg)).await?;
44+
45+
let db_for_callback = db.clone();
46+
let delta_write_callback: DeltaWriteCallback = Arc::new(
47+
move |project_id: String, table_name: String, batches: Vec<RecordBatch>, wal_watermark: DeltaWatermark| {
48+
let db = db_for_callback.clone();
49+
Box::pin(async move {
50+
let pre = db.list_file_uris(&project_id, &table_name).await.unwrap_or_default();
51+
db.insert_records_batch(&project_id, &table_name, batches, true, Some(&wal_watermark)).await?;
52+
let post = db.list_file_uris(&project_id, &table_name).await.unwrap_or_default();
53+
let pre_set: std::collections::HashSet<String> = pre.into_iter().collect();
54+
let added: Vec<String> = post.into_iter().filter(|u| !pre_set.contains(u)).collect();
55+
db.warm_cache_for_table(&project_id, &table_name, added.clone());
56+
Ok(added)
57+
})
58+
},
59+
);
60+
61+
let mut session_context = Arc::new(db.clone()).create_session_context();
62+
db.setup_session_udfs(&mut session_context)?;
63+
let registry: Arc<crate::functions::FnRegistry> = Arc::new(session_context.state());
64+
65+
let mut layer = BufferedWriteLayer::with_config(Arc::clone(&cfg), registry)?.with_delta_writer(delta_write_callback);
66+
67+
// Optional tantivy sidecar (mirrors main.rs). Disabled when no indexed
68+
// tables OR the bucket is unset (tests with foyer-only setups).
69+
let indexed_tables = cfg.tantivy.indexed_tables();
70+
if !indexed_tables.is_empty() {
71+
let bucket = cfg.aws.aws_s3_bucket.clone().unwrap_or_default();
72+
if !bucket.is_empty() {
73+
let storage_uri = format!("s3://{}/{}/tantivy", bucket, cfg.core.timefusion_table_prefix);
74+
let storage_opts = cfg.aws.build_storage_options(None);
75+
let obj_store = db.create_object_store(&storage_uri, &storage_opts).await?;
76+
let svc = Arc::new(crate::tantivy_index::service::TantivyIndexService::new(
77+
obj_store.clone(),
78+
Arc::new(cfg.tantivy.clone()),
79+
));
80+
layer = layer.with_tantivy_indexer(svc.clone().callback());
81+
let cache_root = cfg.core.timefusion_data_dir.clone();
82+
let search = Arc::new(crate::tantivy_index::search::TantivySearchService::new(obj_store, cache_root));
83+
db = db.with_tantivy_search(search).with_tantivy_indexer(svc);
84+
}
85+
}
86+
87+
let buffered_layer = Arc::new(layer);
88+
89+
// WAL replay before background tasks so we don't race spawned flushes.
90+
buffered_layer.recover_from_wal().await?;
91+
buffered_layer.start_background_tasks().await;
92+
93+
db = db.with_buffered_layer(Arc::clone(&buffered_layer));
94+
db = db.start_maintenance_schedulers().await?;
95+
let db = Arc::new(db);
96+
db.setup_session_tables(&mut session_context)?;
97+
98+
Ok(Bootstrapped {
99+
db,
100+
buffered_layer,
101+
session_ctx: Arc::new(session_context),
102+
shutdown: CancellationToken::new(),
103+
})
104+
}

0 commit comments

Comments
 (0)