TL;DR
Produce a reviewed design for serving queries across every data type's hot + cold stores while ingestion and lifecycle routines concurrently mutate the store set. The deliverable is a design, not code — once it's approved, we carve implementation issues from it. Query-side counterpart to #765 (ingestion).
The problem the design must solve
Three kinds of thread touch the store set at the same time:
- 1 ingestion thread — appends new ledgers to the hot store.
- 1 lifecycle thread — freeze (hot → cold), prune cold stores past the retention window, delete a hot store once its freeze completes.
- Many query threads.
A query in flight must never read a store the lifecycle thread is removing, and must see a consistent view of which stores cover its range.
Deliverable
A design doc covering:
- How a query routes across the live set of hot + cold stores per data type — lookup by ledger range, run the per-chunk engine against each, merge to the requested order/limit.
- The reader-safety contract — how queries stay correct while the ingestion and lifecycle operations above mutate the store set concurrently.
- Design review and sign-off before any implementation.
Out of scope
- Implementation — broken into separate issues once the design is approved.
TL;DR
Produce a reviewed design for serving queries across every data type's hot + cold stores while ingestion and lifecycle routines concurrently mutate the store set. The deliverable is a design, not code — once it's approved, we carve implementation issues from it. Query-side counterpart to #765 (ingestion).
The problem the design must solve
Three kinds of thread touch the store set at the same time:
A query in flight must never read a store the lifecycle thread is removing, and must see a consistent view of which stores cover its range.
Deliverable
A design doc covering:
Out of scope