Fix: Group transaction changes by table#3360
Conversation
2380846 to
e0f05c0
Compare
|
nit: |
dimas-b
left a comment
There was a problem hiding this comment.
Nice fix. Thanks, @singhpk234 !
Just a couple of minor comments :)
.../service/src/main/java/org/apache/polaris/service/catalog/iceberg/IcebergCatalogHandler.java
Show resolved
Hide resolved
.../service/src/main/java/org/apache/polaris/service/catalog/iceberg/IcebergCatalogHandler.java
Show resolved
Hide resolved
.../service/src/main/java/org/apache/polaris/service/catalog/iceberg/IcebergCatalogHandler.java
Show resolved
Hide resolved
.../service/src/main/java/org/apache/polaris/service/catalog/iceberg/IcebergCatalogHandler.java
Show resolved
Hide resolved
|
This PR is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days. |
8f1fe58
e0f05c0 to
8f1fe58
Compare
|
Apologies it took me a while to get back to it, was super swamped with internal stuff. Please have another pass when you all get some time, really apprecaite your feedbacks here ! |
|
@singhpk234 : thanks for pushing this forward 👍 Please fix fresh |
8f1fe58 to
90296e0
Compare
Problem
. Groups ALL changes by table- even if they appear randomly in the input like [A1, B1, A2, C1, A3] → groups to {A:
[A1, A2, A3], B: [B1], C: [C1]}
2. For each table, processes changes sequentially :
- Validate R1 against base metadata → Apply U1 → update currentMetadata
- Validate R2 against updated metadata → Apply U2 → update currentMetadata
- Validate R3 against updated metadata → Apply U3 → update currentMetadata
3. Single commit per table prevents duplicate entity IDs in pendingUpdates
This ensures:
related discussion: #3352 (comment)
Checklist
CHANGELOG.md(if needed)site/content/in-dev/unreleased(if needed)