[Bug] Unbounded DE filters materialize excluded managed BLOB payloads #1402

Search before asking

This is a follow-up to #901, which added managed BLOB preselection for Data Evolution reads with LIMIT. The same path is still disabled when the query has no limit. It is separate from the primary-key BLOB issue #1052.

What happened

On a Data Evolution append table, a predicate on ordinary columns is applied after the format reader materializes managed BLOB payloads when the query has no LIMIT. A selective query therefore decodes payloads for rows that will be discarded.

A deterministic regression can use row IDs 100–103, an updated scalar column whose current values are [4, 2, 3, 1], and four managed payloads. Corrupt the fourth payload's entry header while keeping its descriptor index readable, then project the scalar column and payload with the filter id > 1 and no limit. The excluded fourth payload is decoded and the query fails, although only the first three payloads are needed.

Expected behavior

For predicates that depend only on ordinary columns, use the current column values to select row-ID ranges before materializing managed BLOB payloads, with or without a limit. Keep the existing behavior for BLOB-dependent predicates, descriptor reads, BLOB views and legacy files without row IDs.

Affected version

The self.limit.is_none() guard is present on main at 8a432a6398ce9e19a57e595b79dcde4d64304bc4. The regression fixture uses the local filesystem and needs no object storage credentials.