Maintain Data Status of Materialized Views for Partitioned Tables. #786
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
Author
|
This commit represents the first step in supporting partitioned tables for answering queries using materialized views of discussion #780. In a subsequent pull request, I will enable the full functionality, while this commit focuses solely on maintaining the data status. |
Contributor
Author
|
Push a temp commit to debug CI failure which I could not reproduce locally. Will remove it after case fixed. |
This commit improves the handling of data status changes for
materialized views associated with partitioned tables.
Writable operations (INSERT, UPDATE, DELETE, COPY, TRUNCATE, CLUSTER) on
a base table lead to changes in the data status of linked materialized views.
For partitioned tables, changes in child partitions can impact parent and
ancestor tables, necessitating updates to the relevant materialized views.
Example:
P0
/ \
P1 P2
/ \
P2_1 P2_2
For a root partitioned table P0 with child partitions P1 and P2
(each having sub-partitions), inserting rows into P1_1 updates the
data status of mv0 (based on P0) and mv1 (based on P1). Conversely,
inserting into P2_1 only affects mv2_1 (based on P2_1) and mv0, leaving
mv1 unchanged.
Propagation Direction: Data status changes propagate both UP and DOWN the
partition tree, except for specific DDL operations:
Create Table XXX Partition Of:
Indicates an insert operation on the parent table.
Drop Table:
Indicates a delete operation on the parent table.
Alter Table ATTACH/DETACH :
Attaching implies an insert, while detaching implies a delete.
Handling TRUNCATE and CLUSTER: A TRUNCATE operation on a parent table is
treated as a DELETE on its child tables, affecting both the parent and
its descendants. Similarly, a CLUSTER operation will transform the
data status in both UP and DOWN directions. However, CLUSTER has a
recognized status to indicate that the changes have been applied to
the pages of the tables.
The COPY command indicates that data has been inserted into the table
and its ancestor tables.
The VACUUM command indicates that data has been recognized on the table
and its ancestor tables.
Authored-by: Zhang Mingli [email protected]
Contributor
Author
Didn't do anything and CI passed after retry. |
yjhjstz
approved these changes
Dec 25, 2024
my-ship-it
approved these changes
Dec 25, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Handle of data status changes for materialized views associated with partitioned tables.
Writable operations (INSERT, UPDATE, DELETE, COPY, TRUNCATE, CLUSTER and etc) on a base table lead to changes in the data status of linked materialized views. For partitioned tables, changes in child partitions can impact parent and ancestor tables, necessitating updates to the relevant materialized views.
For example, consider a root partitioned table P0 with two child partitioned tables: P1 and P2. Each child table further has
two sub-partitions: P1 has P1_1 and P1_2, while P2 has P2_1 and P2_2.
If we insert rows into P1_1, the data status of both mv0 (based on P0) and mv1 (based on P1) will change, as P1_1's parent table P1 and the root table P0 now contain more rows.
Conversely, if we insert rows into P2_1, only the data status of mv2_1 (based on P2_1) and mv0 will be affected, while mv1 remains unchanged because the data in P1 has not changed.
Propagation Direction:
Data status changes propagate both UP and DOWN the partition tree, except for specific DDL operations:
Indicates an insert operation on the parent table.
Indicates a delete operation on the parent table.
Attaching implies an insert, while detaching implies a delete.
Handling TRUNCATE and CLUSTER: A TRUNCATE operation on a parent table is treated as a DELETE on its child tables, affecting both the parent and its descendants. Similarly, a CLUSTER operation will transform the data status in both UP and DOWN directions. However, CLUSTER has a recognized status to indicate that the changes have been applied to the pages of the tables.
The COPY command indicates that data has been inserted into the table and its ancestor tables.
The VACUUM command indicates that data has been recognized on the table and its ancestor tables.
Authored-by: Zhang Mingli [email protected]
Fixes #ISSUE_Number
What does this PR do?
Type of Change
Breaking Changes
Test Plan
make installcheckmake -C src/test installcheck-cbdb-parallelImpact
Performance:
User-facing changes:
Dependencies:
Checklist
Additional Context
CI Skip Instructions