You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit a92b68c
Browse filesBrowse the repository at this point in the historyBrowse files
[opt](fluss) Retain each log-tail key once, not once per change-log record
A tail is read as one row per change-log record, so it names each key once
for every record it wrote about it -- an update alone writes two. That
block is what the scan then holds for its whole life, once per bucket per
partition, and what the suppression predicate builds its hash map from.
Suppression asks only whether a key is in the set, so every repeat is
retained and indexed for nothing.
`EqualityDeletePredicate::distinct_rows` lives on the predicate rather
than beside its caller because that class already owns the definition of
when two key rows are the same. A caller that decided for itself could
call two keys equal where the matching would not, and the row it then
dropped would be a suppression that silently stops happening. The test
pins the invariant directly: the predicate built from the deduplicated
block answers exactly what the one built from the raw block answers,
across composite keys, NULLs and hash-colliding neighbours.
Deduplicated only once the tail has been read in full, never during
accumulation: the limit counts change-log records, which is the yardstick
the Java side applies to this very range, and a limit that counted
distinct keys instead would let the two halves of one table disagree about
whether the table can be read at all. The record count is kept beside the
keys so the profile can report both -- `FlussUnionTailKeysRead` against
the new `FlussUnionTailKeysRetained` is what says whether a scan's
retention is bounded.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
0 commit comments