Backfill 13: Cold ledger store (packfile-backed reader + writer) #695

Summary

• Cold-storage ledger store at cmd/stellar-rpc/internal/fullhistory/pkg/stores/ledger/, sibling to the existing hot_store.go (RocksDB-backed). Reader + writer ship together so round-trip tests are authoritative.
• Backed by the in-repo packfile library at cmd/stellar-rpc/internal/packfile/ with the native CGo zstd codec at cmd/stellar-rpc/internal/zstd/. No github.com/klauspost/compress/zstd anywhere.
• Reuses the Entry type already exported from hot_store.go (type Entry struct { Seq uint32; Bytes []byte }).
• No LedgerStore interface spanning hot + cold — sibling concrete types in one package; federation deferred to a separate slice.

Files to create

• cmd/stellar-rpc/internal/fullhistory/pkg/stores/ledger/cold_writer.go
• cmd/stellar-rpc/internal/fullhistory/pkg/stores/ledger/cold_writer_test.go
• cmd/stellar-rpc/internal/fullhistory/pkg/stores/ledger/cold_store.go
• cmd/stellar-rpc/internal/fullhistory/pkg/stores/ledger/cold_store_test.go

Public API (pinned)

// cold_writer.go
type ColdWriter struct{ /* unexported */ }

func NewColdWriter(
    path        string,
    firstSeq    uint32,
    newEncoder  func() packfile.RecordEncoder,
    logger      *supportlog.Entry,
) (*ColdWriter, error)

func (w *ColdWriter) AppendLedger(seq uint32, bytes []byte) error
func (w *ColdWriter) Finalize() error
func (w *ColdWriter) Close() error

// cold_store.go
type ColdStore struct{ /* unexported */ }

func OpenColdStore(
    path     string,
    decoder  packfile.RecordDecoder,
    logger   *supportlog.Entry,
) (*ColdStore, error)

func (c *ColdStore) FirstSeq() uint32
func (c *ColdStore) LastSeq()  uint32
func (c *ColdStore) GetLedgerRaw(seq uint32) ([]byte, error)
func (c *ColdStore) IterateLedgers(start, end uint32) iter.Seq2[Entry, error]
func (c *ColdStore) Close() error

Behavior contracts (pinned)

• firstSeq encoding. NewColdWriter takes firstSeq at construction. The writer encodes firstSeq as a 4-byte big-endian uint32 into the packfile's AppData trailer field (WriterOptions.AppData). OpenColdStore reads + decodes AppData to recover firstSeq. lastSeq is derived as firstSeq + TotalItems() - 1 from the packfile reader's metadata accessors.
• Contiguous-append invariant. AppendLedger(seq, bytes) accepts only seq == firstSeq + n where n is the count of prior successful appends. A gap or out-of-order seq returns an error and does NOT advance internal state.
• Bytes-verbatim seam. Caller passes uncompressed XDR-encoded LedgerCloseMeta bytes to AppendLedger; GetLedgerRaw returns the same uncompressed bytes. Zstd encode/decode is internal — handled by the packfile primitive's RecordEncoder / RecordDecoder plumbing. Caller never sees zstd-framed bytes.
• Encoder factory. newEncoder returns a fresh packfile.RecordEncoder per call; the packfile writer calls it once per worker. Production wiring: newEncoder = func() packfile.RecordEncoder { return zstd.NewCompressor(...) }.
• Decoder ownership. OpenColdStore takes a single concurrent-safe packfile.RecordDecoder (production: shared *zstd.Decompressor per process). ColdStore.Close closes the packfile reader but does NOT close the decoder — the decoder's lifecycle is the caller's.
• One .pack per ColdStore. Cross-chunk federation (a reader that spans multiple .pack files keyed on seq) is deferred to a separate slice.
• Close-before-Finalize. ColdWriter.Close called before Finalize removes the partial .pack file (matches packfile.Writer.Close behavior).
• Close-after-Finalize. ColdWriter.Close called after Finalize is a no-op (or close-only on the underlying packfile writer if not already closed).
• Closed-fence. Every public method on ColdWriter and ColdStore checks a closed atomic flag at entry and returns stores.ErrStoreClosed (or yields it and returns, for iteration) on closed.
• Error translation at the L2 boundary. packfile.ErrPositionOutOfRange (and any other miss-sentinel from the packfile primitive) translates to stores.ErrNotFound. Callers depend only on pkg/stores sentinels; they never see packfile.* errors directly.
• Logger. *supportlog.Entry (not a custom interface). Tests use bytes.Buffer + (*supportlog.Entry).SetOutput — no fixture types.

Implementation notes — IterateLedgers wrapper

ColdStore.IterateLedgers(start, end) is a thin wrapper over the packfile reader's iterator-shaped API. Signature from PR #712:

func (r *Reader) ReadRange(start, count int) iter.Seq2[[]byte, error]

ReadRange is designed for sequential scans — it coalesces consecutive records into batched ReadAt calls via a pooled 1 MiB buffer.

Sketch of the wrapper:

func (c *ColdStore) IterateLedgers(start, end uint32) iter.Seq2[Entry, error] {
    return func(yield func(Entry, error) bool) {
        if c.closed.Load() {
            yield(Entry{}, stores.ErrStoreClosed)
            return
        }
        if start > end || end < c.firstSeq || start > c.lastSeq {
            return // no-op — matches HotStore.IterateLedgers for out-of-window
        }
        if start < c.firstSeq { start = c.firstSeq }
        if end   > c.lastSeq  { end   = c.lastSeq  }

        startPos := int(start - c.firstSeq)
        count    := int(end - start) + 1

        seq := start
        for item, err := range c.reader.ReadRange(startPos, count) {
            if err != nil {
                yield(Entry{}, err)
                return
            }
            // ReadRange invalidates the yielded slice after the iterator exits
            // (per PR #737's lifetime contract). Copy before handing to caller.
            bytesCopy := append([]byte(nil), item...)
            if !yield(Entry{Seq: seq, Bytes: bytesCopy}, nil) {
                return
            }
            seq++
        }
    }
}

Three things to get right:

• Clamp before subtracting. seq - firstSeq underflows if seq < firstSeq since both are uint32. Clamp start and end to [firstSeq, lastSeq] before the subtraction.
• Copy each yielded slice. ReadRange's []byte becomes invalid after the iterator exits. HotStore.IterateLedgers does the same copy at hot_store.go:114 (append([]byte(nil), e.Value...)) — match that pattern.
• Out-of-window is a no-op, not an error. Iteration semantics: empty result + nil error. packfile.ErrPositionOutOfRange matters only for GetLedgerRaw point lookups (where we translate it to stores.ErrNotFound); iteration has already clamped to bounds before calling ReadRange.

Test fixtures + sizing (pinned)

• Reuse the in-package helpers from hot_store_test.go — same ledger package, unexported helpers are accessible from cold tests:
  • silentLogger() (hot_store_test.go:24) for the logger argument.
  • makeRandomLedgerCloseMeta(ledgerSeq, txCount, networkPassphrase) (hot_store_test.go:302) to build realistic XDR-encoded ledgers. Call .MarshalBinary() on the returned xdr.LedgerCloseMeta to get the raw bytes that ColdWriter.AppendLedger consumes.
• Default test pack size: 10 ledgers. Use 10 (not 100, not 1000) in every round-trip / iteration test — keeps the suite fast and the assertions readable. Use txCount = 2 or 3 in makeRandomLedgerCloseMeta for the same reason — these are storage tests, not transaction-volume tests.
• Build a small openTestColdWriter(t) / openTestColdStore(t) pair (≤5 lines each, mirroring openTestHotStore) for cleanup-on-failure ergonomics. Keep them in cold_*_test.go.

Test plan (this is the spec — implement in this order)

1. NewColdWriter validation: path / newEncoder / logger required; missing any returns rocksdb.ErrInvalidConfig (or the package's own sentinel — match hot_store.go's style).
2. AppendLedger happy path: append one ledger (built via makeRandomLedgerCloseMeta, MarshalBinary()), round-trip via OpenColdStore + GetLedgerRaw. Assert byte-equality.
3. AppendLedger contiguous-append: a gap (seq != expected next) returns an error and the writer's internal counter does NOT advance.
4. AppendLedger out-of-order: seq < expected next returns an error.
5. Finalize emits a complete .pack file: verify the trailer + AppData round-trip via packfile.Open.
6. Close before Finalize removes the partial .pack file.
7. Close after Finalize is a no-op; calling Close twice after Finalize is also a no-op.
8. AppendLedger after Close returns stores.ErrStoreClosed.
9. OpenColdStore reads firstSeq from AppData and derives lastSeq from packfile.Reader.TotalItems().
10. GetLedgerRaw(seq) out-of-range (seq < firstSeq or seq > lastSeq) returns stores.ErrNotFound.
11. GetLedgerRaw on closed store returns stores.ErrStoreClosed.
12. IterateLedgers(start, end) happy path: covers [start, end] inclusive in ascending order, over a 10-ledger fixture.
13. IterateLedgers(start, end) with start > end is a no-op (no yields, no error).
14. IterateLedgers(start, end) clamps to store bounds when start < firstSeq or end > lastSeq.
15. IterateLedgers on closed store yields stores.ErrStoreClosed once and returns.
16. End-to-end round-trip: build 10 ledgers via makeRandomLedgerCloseMeta, marshal each to bytes, write via ColdWriter, read each back via ColdStore.GetLedgerRaw, assert byte-equality.
17. packfile.Reader.Verify(ctx) content-hash check passes on a finalized file (with content-hash enabled in writer options).

Prior art (commit 19a9179)

Read via git show 19a9179:<path>; do NOT check out, do NOT cherry-pick. For EACH file, make an explicit port / adapt / rewrite / discard call before writing:

• full-history/backfill/lfs_writer.go
• full-history/backfill/lfs_writer_test.go
• full-history/pkg/lfs/chunk.go
• full-history/pkg/lfs/discovery.go
• full-history/pkg/lfs/iterator.go

Expect rewrite on the codec + file-format wiring (the reference predated the packfile library and used klauspost zstd). Sequence-bound discovery patterns may adapt.

Out of scope

• Cross-chunk federation (a reader that spans multiple .pack files based on seq).
• A LedgerStore interface unifying hot + cold — siblings only, no spanning interface.
• Retention pruning at the cold tier — deletion of old .pack files is the orchestrator's concern.
• The events writer (owned by Full-History Event Store #665 and its sub-issues).
• Path conventions / chunk-directory layout — path is passed in by the caller; this slice does not derive paths.

Dependencies

• Generic packfile writer — PR Add packfile writer #702 (merged). Caller-supplied NewRecordEncoder factory.
• Generic packfile reader — PR Add packfile reader #712 (merged). Caller-supplied RecordDecoder single instance; ReadRange(start, count int) iter.Seq2[[]byte, error] for iteration.
• Native zstd CGo wrapper — PR Add thin CGo zstd wrapper for packfile compression #650 (merged). *zstd.Compressor and *zstd.Decompressor satisfy the packfile codec interfaces directly.
• Packfile design-doc drift fix — PR Update packfile design doc to match merged writer and reader implementations #737 (open). The PR body is the authoritative spec until Update packfile design doc to match merged writer and reader implementations #737 merges; the on-disk doc at full-history/design-docs/packfile-library.md is stale.
• Existing pkg/stores/ledger/hot_store.go + hot_store_test.go — for Entry type, closed-fence convention, iter.Seq2[Entry, error] shape, bytes-verbatim contract, and the silentLogger() / makeRandomLedgerCloseMeta(...) test helpers.

Supersedes #720 (closed); see the close comment on #720 for context.

Acceptance

• All 17 tests in the test plan pass under go test -race.
• $(go env GOPATH)/bin/golangci-lint run ./cmd/stellar-rpc/internal/fullhistory/... clean.
• gofmt -l cmd/stellar-rpc/internal/fullhistory/ prints nothing.
• No packfile.* or zstd.* types in any public method signature except codec interfaces in constructors.
• No github.com/klauspost/compress/zstd import anywhere in the new code (grep before submitting).
• No LedgerStore interface spanning hot + cold.
• Tests reuse silentLogger() + makeRandomLedgerCloseMeta() from hot_store_test.go; do not duplicate fixture code.