Skip to content

read_transaction_by_version should not populate Dataset Session caches #7801

Description

@Xuanwo

Problem

Dataset::read_transaction_by_version currently delegates to checkout_version and then calls read_transaction:

pub async fn read_transaction_by_version(
    &self,
    version: u64,
) -> Result<Option<Transaction>> {
    let dataset_version = self.checkout_version(version).await?;
    dataset_version.read_transaction().await
}

A caller requesting one transaction does not need a historical Dataset. However, the checkout path has broader Session-cache side effects:

  • The historical Manifest may be inserted into the metadata cache.
  • Manifest loading may opportunistically decode and cache the IndexSection.
  • The Transaction may be inserted into the metadata cache.

A long-lived process scanning many historical transactions can therefore fill shared Dataset and index caches with historical state it never uses.

This became more visible after #7661 made version checkout reuse and populate the Session manifest cache. The optimization remains useful for actual Dataset checkout; the transaction API should use a narrower read path.

Expected behavior

Reading a transaction by version should:

  1. Resolve the requested version through the Dataset's current branch and CommitHandler.
  2. Read the Manifest metadata needed to locate the transaction.
  3. Read the inline or external transaction.
  4. Avoid constructing a historical Dataset.
  5. Avoid decoding the IndexSection.
  6. Avoid modifying the Dataset Session's manifest, index, or transaction caches.

The existing read_transaction_by_version return type and error semantics should remain compatible.

It may also be useful to expose a compact record containing the version, Manifest timestamp, and optional Transaction so callers that need the commit timestamp do not have to check out the Dataset.

Suggested validation

  • Create an indexed dataset with many versions.
  • Record Session index- and metadata-cache statistics.
  • Read transactions from multiple historical versions.
  • Verify that the returned transactions are unchanged and the Session cache entries and bytes do not increase.
  • Cover inline transactions, external transaction files, branches, missing versions, and V1/V2 manifest naming.

checkout_version should retain its current caching behavior.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions