You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit dfe6bac
Browse filesBrowse the repository at this point in the historyBrowse files
### What problem does this PR solve?
Issue Number: closeapache#66816
Related PR: apache#67207
Problem Summary: Iceberg and Paimon row-location metadata columns must remain correct across native FileScannerV2 scans, streaming split planning, schema evolution, and externally-created tables. The previous paths could silently replace or misclassify physical columns with metadata-like names, or route metadata projections through schema-evolution and legacy readers that cannot materialize file and row positions. This change rejects unsafe physical-name collisions at connector metadata assembly, keeps synthetic Paimon handles out of physical projections and schema evolution, validates Iceberg metadata columns on streaming and eager paths, fences scans to the current backend semantics, and makes BE slot classification depend on the explicit synthesized category. Targeted connector and BE tests cover these paths.
### Release note
None
### Check List (For Author)
- Test: Iceberg connector tests (253 passed), Paimon connector tests (118 passed), BE FileScannerV2 classification tests (2 passed), format check, build hygiene check
- Behavior changed: Yes (correctly rejects unsafe external metadata-name collisions and prevents unsupported reader fallback)
- Does this need documentation: No
Copy file name to clipboardExpand all lines: fe/fe-connector/fe-connector-iceberg/src/main/java/org/apache/doris/connector/iceberg/IcebergConnectorMetadata.java
Copy file name to clipboardExpand all lines: fe/fe-connector/fe-connector-iceberg/src/main/java/org/apache/doris/connector/iceberg/IcebergScanPlanProvider.java
Copy file name to clipboardExpand all lines: fe/fe-connector/fe-connector-iceberg/src/test/java/org/apache/doris/connector/iceberg/IcebergConnectorMetadataDdlTest.java
+29Lines changed: 29 additions & 0 deletions
Original file line number
Diff line number
Diff line change
@@ -323,6 +323,35 @@ public void testCreateTableRejectsReservedFileMetadataColumnsAtAllVersions() {
Copy file name to clipboardExpand all lines: fe/fe-connector/fe-connector-iceberg/src/test/java/org/apache/doris/connector/iceberg/IcebergConnectorMetadataTest.java
+31Lines changed: 31 additions & 0 deletions
Original file line number
Diff line number
Diff line change
@@ -698,6 +698,37 @@ public void getTableSchemaParsesColumnsFromLoadedTable() {
Copy file name to clipboardExpand all lines: fe/fe-connector/fe-connector-iceberg/src/test/java/org/apache/doris/connector/iceberg/IcebergScanPlanProviderTest.java
+15Lines changed: 15 additions & 0 deletions
Original file line number
Diff line number
Diff line change
@@ -936,6 +936,21 @@ public void metadataColumnsRejectDisabledFileScannerV2DuringPlanning() {
Copy file name to clipboardExpand all lines: fe/fe-connector/fe-connector-paimon/src/main/java/org/apache/doris/connector/paimon/PaimonColumnHandle.java
0 commit comments