You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 61a8b38
Browse filesBrowse the repository at this point in the historyBrowse files
External Regression on this branch (build 1000131) surfaced two real regressions
that pass on master but fail here; both are fixed with FE regression tests.
1. Hive INSERT into a JuiceFS (jfs://) backed table failed on the BE with
"No FileSystem for scheme jfs". HiveWritePlanProvider.buildHadoopConfig()
sourced the BE writer hadoop config only from the typed getStorageProperties()
pipeline, which is EMPTY for schemes whose fe-filesystem plugin has no typed
bind() (jfs, oss-hdfs), so fs.<scheme>.impl / juicefs.* never reached the BE.
It now also merges getBackendStorageProperties() -- the same source the hive
READ path (HiveScanPlanProvider) already uses, which carries the fs./juicefs.*
passthrough. The typed overlay is applied last so object-store / HDFS writes
stay byte-identical; the default getBackendStorageProperties() is emptyMap so
nothing else changes. This also covers the oss-hdfs (JindoFS) sibling.
2. Selecting an iceberg $files / $data_files system table threw
BufferUnderflowException. IcebergScanPlanProvider planned the metadata-table
scan without a column projection, so the SDK eagerly materialised
readable_metrics for every row (MetricsUtil.readableMetricsStruct ->
Conversions.fromByteBuffer underflows on complex/boolean bound columns) even
when the query selected only a scalar such as file_size_in_bytes.
doPlanSystemTableScan now projects the scan to the requested columns
(scan.select(...)), applied only in the system-table path; $position_deletes
and the shared buildScan (data scans) are unaffected.
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_014ZngdXGBAXNDRe7jA42mAS
Copy file name to clipboardExpand all lines: fe/fe-connector/fe-connector-hive/src/main/java/org/apache/doris/connector/hive/HiveWritePlanProvider.java
Copy file name to clipboardExpand all lines: fe/fe-connector/fe-connector-hive/src/test/java/org/apache/doris/connector/hive/HiveWritePlanProviderTest.java
+21Lines changed: 21 additions & 0 deletions
Original file line number
Diff line number
Diff line change
@@ -488,6 +488,27 @@ public void planWriteHadoopConfigFromStorageProperties() {
Copy file name to clipboardExpand all lines: fe/fe-connector/fe-connector-hive/src/test/java/org/apache/doris/connector/hive/RecordingConnectorContext.java
+10Lines changed: 10 additions & 0 deletions
Original file line number
Diff line number
Diff line change
@@ -48,6 +48,11 @@ final class RecordingConnectorContext implements ConnectorContext {
Copy file name to clipboardExpand all lines: fe/fe-connector/fe-connector-iceberg/src/main/java/org/apache/doris/connector/iceberg/IcebergScanPlanProvider.java
Copy file name to clipboardExpand all lines: fe/fe-connector/fe-connector-iceberg/src/test/java/org/apache/doris/connector/iceberg/IcebergScanPlanProviderTest.java
+31Lines changed: 31 additions & 0 deletions
Original file line number
Diff line number
Diff line change
@@ -746,6 +746,37 @@ public void getScanNodePropertiesForSysHandleStillEmitsLocationCreds() {
0 commit comments