You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
ShardWriter::close() is the graceful shutdown API that flushes pending data
before stopping background tasks. However, it currently waits for several
close-time completion notifications while discarding their results:
the final WAL flush in MemTable mode;
frozen MemTable/L0 flushes;
the final WAL flush in WAL-only mode.
Failures to send the final WAL flush request and completion channels closing
without a result are also treated as success.
Failure sequence
sequenceDiagram
participant Caller
participant Close as ShardWriter::close()
participant Flusher as WAL / MemTable flusher
participant Store as Object store
participant Tasks as TaskExecutor
Caller->>Close: close()
Close->>Flusher: request final flush
Flusher->>Store: persist pending data
Store-->>Flusher: persistence failure
Flusher-->>Close: Err / DurabilityResult::Failed
Note over Close: Completion is awaited,<br/>but its value is discarded
Close->>Tasks: shutdown_all()
Tasks-->>Close: Ok(())
Close-->>Caller: Ok(()) (incorrect)
Note over Caller,Store: Caller assumes durability<br/>although data was not persisted
Loading
Impact
In WAL-only mode, a failed final WAL append leaves the batch in the in-memory
pending queue. Because close() consumes the writer, that queue is then dropped
while the caller receives Ok(()).
In MemTable mode, callers can receive a successful close result even though the
final WAL or L0 generation was not persisted.
This is a false durability acknowledgement and may cause data loss that is
invisible to the caller.
Expected behavior
ShardWriter::close() should:
return final WAL and MemTable flush failures;
preserve typed FenceReason values carried by WalFlushFailure;
return the first causal failure;
continue draining already queued work after an error;
always shut down and join background tasks before returning;
keep successful close behavior unchanged.
Acceptance criteria
WAL-only final persistence failures are returned to the caller.
MemTable final WAL persistence failures are returned to the caller.
Frozen MemTable/L0 failures are returned to the caller.
Problem
ShardWriter::close()is the graceful shutdown API that flushes pending databefore stopping background tasks. However, it currently waits for several
close-time completion notifications while discarding their results:
Failures to send the final WAL flush request and completion channels closing
without a result are also treated as success.
Failure sequence
sequenceDiagram participant Caller participant Close as ShardWriter::close() participant Flusher as WAL / MemTable flusher participant Store as Object store participant Tasks as TaskExecutor Caller->>Close: close() Close->>Flusher: request final flush Flusher->>Store: persist pending data Store-->>Flusher: persistence failure Flusher-->>Close: Err / DurabilityResult::Failed Note over Close: Completion is awaited,<br/>but its value is discarded Close->>Tasks: shutdown_all() Tasks-->>Close: Ok(()) Close-->>Caller: Ok(()) (incorrect) Note over Caller,Store: Caller assumes durability<br/>although data was not persistedImpact
In WAL-only mode, a failed final WAL append leaves the batch in the in-memory
pending queue. Because
close()consumes the writer, that queue is then droppedwhile the caller receives
Ok(()).In MemTable mode, callers can receive a successful close result even though the
final WAL or L0 generation was not persisted.
This is a false durability acknowledgement and may cause data loss that is
invisible to the caller.
Expected behavior
ShardWriter::close()should:FenceReasonvalues carried byWalFlushFailure;Acceptance criteria
errors.