Add support for shared storage snapshots across subqueries to ensure query-level consistency#79471
Conversation
fee46e3 to
9b19f76
Compare
azat
left a comment
There was a problem hiding this comment.
Looks OK to me
@KochetovNicolai can you also PTAL?
c829d84 to
577278d
Compare
azat
left a comment
There was a problem hiding this comment.
I've discussed this with @KochetovNicolai and he is OK with it
BTW @amosbird I guess we can cache parts after partition pruning for this case?
P.S. can you please resolve the conflicts (and sorry for the delay)
We can't, because each subquery may have different pruning predicates.
Sure! |
577278d to
4cd0f8e
Compare
03519_storage_url #80591 (comment) |
@amosbird I've fixed the conflicts with private, so please don't rebase from now on |
If it will fail again we can use failpoints instead. |
…ared_storage_snapshots
* [GLUTEN-1632][CH]Daily Update Clickhouse Version (20250611) * Fix Build due to ClickHouse/ClickHouse#81191 * Fix UT due to ClickHouse/ClickHouse#79471 --------- Co-authored-by: kyligence-git <[email protected]> Co-authored-by: Chang chen <[email protected]>
Changelog category (leave one):
Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):
Added a setting
enable_shared_storage_snapshot_in_queryto enable sharing the same storage snapshot across all subqueries in a single query. This ensures consistent reads from the same table, even when the table is referenced multiple times within a query.For example, the following query now produces correct and stable results when this setting is enabled:
This helps #78429.
Additionally, introduced a test-only setting
merge_tree_storage_snapshot_sleep_msthat allows injecting a delay during storage snapshot acquisition, to simulate concurrent modifications and verify snapshot consistency.Documentation entry for user-facing changes