Skip to content

[Design]: A subject's receipts need a monotonic sequence inside the signed body, so a missing one is visible #452

Open

Description

@fm1960hamburg

Feature Area

Backend API / Server

Problem Statement

A receipt is addressed by a ULID (receipt_id, String(26) primary key). ULIDs sort by time, but they are not dense: between two of them, nothing says whether a third ever existed. There is no per-subject numbering — parent_receipt_id chains individual receipts, it does not form a sequence.

So "these are all the receipts for this subject" is currently an assertion, not something an auditor can check. A receipt that was never written, one that was removed from the table, and one that simply did not happen all look identical from the outside. The table is append-only by service-layer convention ("nothing in the server codebase issues UPDATE or DELETE against this table"), which is a statement about our code, not a property an auditor can verify.

This matters most where receipts are the evidence:

Proposed Solution

  1. Number receipts per subject. Add a sequence number, scoped to (tenant_id, subject_id), starting at 1 and increasing by exactly 1 per emitted receipt.
  2. Put it inside the signed body. _UNSIGNED_BODY_FIELDS excludes only receipt_signature, so a new body field is signed automatically. That is the point of the design: removing a receipt then forces renumbering the ones after it, and every renumbered body fails verification. Without the signature, whoever removes a row also renumbers it.
  3. Make the write path actually gapless. A Postgres sequence is the wrong tool: nextval does not roll back, so a failed emission burns a number and produces a gap that looks like tampering. Use a counter row updated in the same transaction as the insert, or an advisory lock on (tenant_id, subject_id), and document the choice. [Bug]: Concurrent compiles of one subject can create duplicate memories #417 is prior art for the locking trade-off.
  4. Keep the number when the body is erased. The sequence is not personal data. Erasure ([Bug]: Receipts keep the subject's task text after subject deletion and after retention expiry #423) and retention tombstoning must both leave it readable, otherwise erasing a subject destroys exactly the evidence that the set was complete.
  5. Expose it. Return the number with the receipt, and give GET /v1/receipts?subject_id=… a way to report the highest number and any missing ones, so "is this set complete?" is one call rather than a manual diff.
  6. Say what happens to existing receipts. Pre-sequence receipts have no number. Treat that as "unnumbered", never as sequence 0, and have verification report it as such rather than as a gap.

API Design (if applicable)

Sketch, not a proposal to copy:

- Body gains `subject_seq: 42` (signed).
- `GET /v1/receipts/{id}` returns it alongside `receipt_id`.
- `GET /v1/receipts?subject_id=…&check_sequence=true` returns `{highest: 42, missing: [17], unnumbered: 3}`.
- `GET /v1/receipts/{id}/verify` reports an unnumbered receipt distinctly from a failed signature, the same way it already reports `no_signature`.

Alternatives Considered

  • A list plus a count inside the destruction receipt (the current plan in [Feature]: Record key destruction as its own receipt, and restrict who can read a subject key #425): proves that our own verification pass covered everything it knew about. It cannot show that the store itself is missing a receipt.
  • A hash chain, each receipt committing to its predecessor: stronger, and it also detects reordering, but it turns emission into a strictly serial operation per subject and makes a single lost receipt break every later verification. The counter degrades more gracefully: one gap, one visible hole.
  • Relying on created_at ordering: timestamps do not prove that nothing sat between two rows.

Priority to You

Medium - Would be helpful

Additional Context

Raised by @TurjoAt on X: "a gap only reads as a gap if the ids are dense. with random ids you can't tell a missing receipt from one that never existed, so the subject needs its own monotonic counter, not just a list."

Open questions for the design:

  • Scope. Per (tenant_id, subject_id) is the obvious unit. Confirm it holds under tenant region pinning (v0.9), where a subject's writes happen in one region anyway, and state explicitly what happens if that ever stops being true.
  • Canonical form. Adding a signed field changes the canonical body. [Bug]: Receipts keep the subject's task text after subject deletion and after retention expiry #423 already plans canonicalize_receipt_body_v2; this field should ride along with that version rather than introduce a third one.
  • Write cost. Receipt emission currently has no per-subject serialisation point. Measure the contention this adds before choosing between a counter row and an advisory lock.

Contribution

  • I would be interested in contributing this feature
  • I would be interested in testing this feature

Activity

  1. fm1960hamburg commented on Sep 25, 2026

    @fm1960hamburg
    Author

    Two additions from @TurjoAt on X.

    1. A signed counter does not survive a truncated tail. It proves nothing is missing below the highest number an auditor can see. It cannot prove that number is the last one written: deleting the final n receipts leaves a complete, fully valid sequence. The check therefore needs an anchor outside the store, a last-seen number per subject.

    The cheapest anchor is the read path: return the sequence number on the response that produced the receipt, next to receipt_id and receipt_emitted. Every caller then keeps its own watermark as a side effect of normal use, and a truncated tail contradicts somebody's log. This connects to #450, which is about receipt_id reaching the caller at all.

    2. Start with the lock, shard only the hot subjects. Per-subject contention only appears when one subject writes hot, which most do not. Worth adding as a constraint: with per-shard counters, gaplessness becomes a per-shard property. The audit check then has to run per shard — over the union, a hole in one shard is invisible.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions