Skip to content

Commit 0bdb7b6

Browse files
authored
[fix](docker)(case) Fix kerberized-HDFS p2 suites: SASL client property, Paimon fixture, iceberg root-cause reporting (apache#66433)
### What problem does this PR solve? Issue Number: close #xxx Related PR: apache#65564, apache#66313, apache#66417 Problem Summary: Master port of the remaining external-regression gaps diagnosed on the branch-4.0 pipeline (apache#66417). apache#66313 already carried the keytab and baseline fixes to master; this PR carries the rest. All three defects were verified to exist on master by inspection — master shares the same code and the same lightweight Kerberos environment (apache#65564, which apache#65939 backported to 4.0). **1. Kerberos HDFS writes fail in the p2 suites (SASL client property missing)** apache#65564 created `kerberos/conf/hdfs-site.xml.tpl` with `dfs.data.transfer.protection=authentication`. A client that does not set the same property sends a plain `writeBlock` op, `SaslDataTransferServer` closes the socket, the client excludes the only DataNode, and the write fails with could only be written to 0 of the 1 minReplication nodes. There are 1 datanode(s) running and 1 node(s) are excluded in this operation. (through iceberg this surfaces as `IllegalArgumentException: Self-suppression not permitted`). Metadata operations only reach the NameNode and the HMS, so `CREATE DATABASE` / `CREATE TABLE` still succeed and only the write fails. apache#65564 set the client property in the four `external_table_p0/kerberos` suites but not in the p2 suites that talk to the same kerberized HDFS. On the branch-4.0 pipeline this turned `hive_on_hms_and_dlf` and `iceberg_on_hms_and_filesystem_and_dlf` red on every run; master carries the identical omission. Fix: set `dfs.data.transfer.protection` in every property block that points at the kerberized HDFS. A scan of `:8520`/`:8620` across the whole suite tree finds seven such blocks still missing it: two each in `hive_on_hms_and_dlf`, `iceberg_on_hms_and_filesystem_and_dlf` and `test_paimon_hms_catalog`, plus one in `hdfs_all_test` (currently returns early because `refactor_params_hdfs_kerberos_test` is unset — it would fail the moment it is enabled). The kerberos block of `test_information_schema_timezone` has the same gap but is disabled by an unrelated TODO, so it is left alone. **2. `test_paimon_hms_catalog`: `Unknown database 'hdfs_db'`** The lightweight Kerberos environment starts an empty metastore — `schematool -initSchema` and nothing else. The environment it replaced uploaded `paimon_data` into HDFS and registered `hdfs_db` through `create_paimon_hive_table.hql` on every start; `paimon_data/hdfs_db.db/` is still in the tree but the compose file never mounted it. This restores the fixture on `kerberos1` only (it owns metastore 9583, the only one the suite talks to), runs it before `DORIS_KERBEROS_READY` so nothing races the suites, and aborts the container on failure instead of handing out an environment silently missing `hdfs_db`. Two deliberate differences from the branch-4.0 sibling commit: - **`hdfs_db` only, no `ali_db`/OSS.** Master's pre-apache#65564 HQL provisioned exactly `hdfs_db`; the `oss_hms_kerberos` case that reads `ali_db` through the kerberized metastore exists only on branch-4.0, master's `ali_db` cases go through the Hive3 metastore, and the golden output here carries only the `hdfs_kerberos` / `hdfs_new_kerberos` blocks. This keeps the jindo/aliyun jars and OSS credentials out of the kerberos stack. - **The fixture is gated on the container role (`HOST`).** Both kerberos containers run the same entrypoint with the same rendered env switch, but only `kerberos1` carries the `sql/`/`paimon_data/`/`auxlib/` mounts — without the gate, `set -e` would abort `kerberos2` the moment `enablePaimonHms` is on. (The branch-4.0 sibling needs the same gate; a follow-up on apache#66417 will add it.) **3. `IcebergConnectorMetadata`: report the root cause of failed metadata commits** `TableMetadataParser.internalWrite` writes the metadata JSON inside try-with-resources, and `DFSOutputStream` throws the same exception instance from `write` and from `close` once its pipeline has failed, so `Throwable.addSuppressed(this)` replaces the failure with `IllegalArgumentException: Self-suppression not permitted`. The FE log keeps the chain, but the message returned over the MySQL protocol is built from `e.getMessage()` and carries only the wrapper. This switches all nineteen metadata-commit paths — `createTable`, the column/schema updates (including the nested-column and comment variants), the branch/tag operations, and the partition-field operations — to `ExceptionUtils.getRootCauseMessage(e)`. The legacy `IcebergMetadataOps` already reported the root cause for the partition-field trio; the connector rewrite lost that, so this also restores it. Read paths and the drop/rename operations cannot hit self-suppression and keep `e.getMessage()`. **Intentionally not ported from apache#66417:** the file-cache de-flake. Master's `test_file_cache_features` (rewritten in apache#66313's lineage) already waits for `disk_resource_limit_mode` / `need_evict_cache_in_advance` to clear before returning, which closes the residual-state window at its source. **Verification (local):** `fe-connector-iceberg` compiles and passes checkstyle (`mvn -pl fe-connector/fe-connector-iceberg package`); the four changed suites compile with the offline groovy compiler; `bash -n` passes on both shell scripts; the compose/env/conf templates render with no leftover placeholders, the rendered XML parses, kerberos1 gets exactly the three new mounts while kerberos2 is unchanged; the settings resolution and the entrypoint role-gate were exercised for all switch/role combinations. The containers themselves need the CI environment and have not been run. ### Release note None ### Check List (For Author) - Test - [x] Regression test - [ ] Unit Test - [ ] Manual test (add detailed scripts or steps below) - [ ] No need to test or manual test. Explain why: - [ ] This is a refactor/code format and no logic has been changed. - [ ] Previous test can cover this change. - [ ] No code files have been changed. - [ ] Other reason - Behavior changed: - [x] No. - [ ] Yes. - Does this need documentation? - [x] No. - [ ] Yes. --------- Signed-off-by: morningman <[email protected]>
1 parent 30a699a commit 0bdb7b6

10 files changed

Lines changed: 141 additions & 20 deletions

File tree

‎docker/thirdparties/docker-compose/kerberos/entrypoint-hive-master.sh‎

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -136,9 +136,35 @@ kdestroy
136136

137137
report_stage "initialize-hive-metastore"
138138
schematool -dbType derby -initSchema
139+
# The Paimon table declares no columns, so the metastore itself resolves its
140+
# schema through the storage handler. The handler class must therefore be on the
141+
# metastore service's own classpath, not merely on the DDL client's.
142+
if [[ -d /opt/doris/auxlib ]]; then
143+
export HIVE_AUX_JARS_PATH=/opt/doris/auxlib
144+
fi
139145
start_service hive --service metastore -p "${HMS_PORT}"
140146
wait_for_port "${HOST}" "${HMS_PORT}" "Hive Metastore"
141147

148+
# Register the Paimon fixture consumed by test_paimon_hms_catalog. This runs
149+
# before the readiness marker below on purpose: run-thirdparties-docker.sh
150+
# releases the pipeline on DORIS_KERBEROS_READY, so anything published later
151+
# would race the suites. A failure here aborts the container (set -e) instead of
152+
# handing out an environment that is silently missing hdfs_db.
153+
#
154+
# Both kerberos containers run this entrypoint with the same env switch, but the
155+
# fixture and its mounts (sql/, paimon_data/, auxlib/) belong to kerberos1 only -
156+
# its metastore (9583) is the one the suite talks to. Gate on the container role,
157+
# not on the mounts, so a broken mount on kerberos1 still fails loudly.
158+
if [[ "${enablePaimonHms:-false}" == "true" && "${HOST:-}" == "hadoop-master" ]]; then
159+
report_stage "load-paimon-hms"
160+
export KRB5CCNAME=FILE:/tmp/hive-admin.ccache
161+
kinit -kt /data/keytabs/hive.keytab "${HIVE_PRINCIPAL}"
162+
hdfs dfs -mkdir -p /user/hive/warehouse
163+
hdfs dfs -put -f /opt/doris/paimon_data/* /user/hive/warehouse/
164+
hive -f /opt/doris/sql/create_paimon_hive_table.hql
165+
kdestroy
166+
fi
167+
142168
touch /tmp/SUCCESS
143169
echo "Minimal Kerberos HDFS and Hive Metastore environment is ready"
144170
echo "DORIS_KERBEROS_READY"

‎docker/thirdparties/docker-compose/kerberos/hadoop-hive.env.tpl‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -29,3 +29,7 @@ PRESTO_CLIENT_KEYTAB=${PRESTO_CLIENT_KEYTAB}
2929
HADOOP_CONF_DIR=/opt/doris/conf
3030
HIVE_CONF_DIR=/opt/doris/conf
3131
KRB5_CONFIG=/etc/krb5.conf
32+
# Paimon-on-kerberized-HMS fixture switch. Read from hive-3x_settings.env by
33+
# start_kerberos() rather than duplicated into kerberos*_settings.env, so that
34+
# the pipeline keeps patching exactly one file and the two stacks cannot drift.
35+
enablePaimonHms=${enablePaimonHms}

‎docker/thirdparties/docker-compose/kerberos/kerberos.yaml.tpl‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -33,11 +33,17 @@ services:
3333
<<: *kerberos-service
3434
container_name: doris-${CONTAINER_UID}-kerberos1
3535
hostname: hadoop-master
36+
# auxlib/sql/paimon_data are mounted on kerberos1 only: test_paimon_hms_catalog
37+
# is the sole consumer and it talks to this metastore (9583). Leaving kerberos2
38+
# untouched keeps the second container as light as it was.
3639
volumes:
3740
- ./conf/kerberos1:/opt/doris/conf:ro
3841
- ./conf/kerberos1/krb5.conf:/etc/krb5.conf:ro
3942
- ./data/kerberos1:/data
4043
- ./two-kerberos-hives:/keytabs
44+
- ./auxlib:/opt/doris/auxlib:ro
45+
- ./sql:/opt/doris/sql:ro
46+
- ./paimon_data:/opt/doris/paimon_data:ro
4147
env_file:
4248
- ./hadoop-hive-1.env
4349

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
-- Paimon table registered in the kerberized Hive Metastore, consumed by the
2+
-- hdfs_kerberos / hdfs_new_kerberos cases of
3+
-- external_table_p2/paimon/test_paimon_hms_catalog.groovy.
4+
--
5+
-- hdfs_db is backed by ../paimon_data, which the entrypoint uploads into HDFS.
6+
-- (branch-4.0 additionally registers ali_db over OSS here; master has no case
7+
-- that reads it through this metastore, so it is deliberately not provisioned.)
8+
--
9+
-- The table declares no columns, so the metastore derives them through the
10+
-- Paimon storage handler (metastore.storage.schema.reader.impl is
11+
-- SerDeStorageSchemaReader) -- the Paimon jar has to be on the metastore's own
12+
-- classpath, not just on the classpath of whoever runs this script.
13+
14+
CREATE DATABASE IF NOT EXISTS hdfs_db;
15+
16+
USE hdfs_db;
17+
18+
DROP TABLE IF EXISTS external_test_table;
19+
20+
CREATE EXTERNAL TABLE external_test_table
21+
STORED BY 'org.apache.paimon.hive.PaimonStorageHandler'
22+
LOCATION 'hdfs:///user/hive/warehouse/hdfs_db.db/external_test_table';

‎docker/thirdparties/run-thirdparties-docker.sh‎

Lines changed: 48 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1595,6 +1595,48 @@ validate_kerberos_container() {
15951595
fi
15961596
}
15971597

1598+
# Aux jar for the Paimon fixture in the kerberized metastore: the Paimon storage
1599+
# handler the metastore loads to derive the fixture table's schema. Same source
1600+
# and same skip-if-present caching the Hive3 stack uses in prepare-hive-data.sh.
1601+
# (branch-4.0 also stages the jindo/aliyun jars for its ali_db-over-OSS case;
1602+
# master has no such case in this metastore.)
1603+
download_kerberos_paimon_jars() {
1604+
local auxlib_dir="$1"
1605+
local url_prefix="https://${s3BucketName}.${s3Endpoint}/regression/docker/hive3"
1606+
local jars=(
1607+
paimon-hive-connector-3.1-1.3-SNAPSHOT.jar
1608+
)
1609+
local jar
1610+
1611+
for jar in "${jars[@]}"; do
1612+
if [[ -f "${auxlib_dir}/${jar}" ]]; then
1613+
echo "Reuse cached kerberos aux jar ${jar}"
1614+
continue
1615+
fi
1616+
echo "Download kerberos aux jar ${jar}"
1617+
curl -sSfL -o "${auxlib_dir}/${jar}" "${url_prefix}/${jar}"
1618+
done
1619+
}
1620+
1621+
# enablePaimonHms lives in hive-3x_settings.env, which the pipeline already
1622+
# patches with the real value. Read it from there instead of duplicating the
1623+
# switch into kerberos*_settings.env, where the two copies would drift and one
1624+
# stack would silently lose the fixture. Sourced in a subshell so that hive3's
1625+
# FS_PORT / HMS_PORT cannot leak into the kerberos settings sourced a few lines
1626+
# below.
1627+
resolve_kerberos_paimon_env() {
1628+
local hive3_settings="${ROOT}/docker-compose/hive/hive-3x_settings.env"
1629+
1630+
enablePaimonHms="false"
1631+
if [[ -f "${hive3_settings}" ]]; then
1632+
read -r enablePaimonHms < <(
1633+
. "${hive3_settings}" >/dev/null 2>&1
1634+
printf '%s\n' "${enablePaimonHms:-false}"
1635+
)
1636+
fi
1637+
export enablePaimonHms
1638+
}
1639+
15981640
start_kerberos() {
15991641
echo "RUN_KERBEROS"
16001642
local KERBEROS_DIR="${ROOT}/docker-compose/kerberos"
@@ -1609,8 +1651,13 @@ start_kerberos() {
16091651

16101652
export CONTAINER_UID=${CONTAINER_UID}
16111653
envsubst <"${KERBEROS_DIR}/kerberos.yaml.tpl" >"${KERBEROS_DIR}/kerberos.yaml"
1654+
# auxlib must exist even when the fixture is off: kerberos1 mounts it read-only.
16121655
mkdir -p "${KERBEROS_DIR}/conf/kerberos1" "${KERBEROS_DIR}/conf/kerberos2" \
1613-
"${KERBEROS_DIR}/two-kerberos-hives"
1656+
"${KERBEROS_DIR}/two-kerberos-hives" "${KERBEROS_DIR}/auxlib"
1657+
resolve_kerberos_paimon_env
1658+
if [[ "${enablePaimonHms}" == "true" && "${STOP}" -ne 1 ]]; then
1659+
download_kerberos_paimon_jars "${KERBEROS_DIR}/auxlib"
1660+
fi
16141661
for i in {1..2}; do
16151662
. "${KERBEROS_DIR}/kerberos${i}_settings.env"
16161663
envsubst <"${KERBEROS_DIR}/hadoop-hive.env.tpl" >"${KERBEROS_DIR}/hadoop-hive-${i}.env"

‎fe/fe-connector/fe-connector-iceberg/src/main/java/org/apache/doris/connector/iceberg/IcebergConnectorMetadata.java‎

Lines changed: 28 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -52,6 +52,7 @@
5252
import org.apache.doris.thrift.TTableDescriptor;
5353
import org.apache.doris.thrift.TTableType;
5454

55+
import org.apache.commons.lang3.exception.ExceptionUtils;
5556
import org.apache.iceberg.BaseTable;
5657
import org.apache.iceberg.MetadataTableType;
5758
import org.apache.iceberg.MetadataTableUtils;
@@ -965,7 +966,8 @@ public void createTable(ConnectorSession session, ConnectorCreateTableRequest re
965966
});
966967
} catch (Exception e) {
967968
throw new DorisConnectorException("Failed to create Iceberg table "
968-
+ request.getDbName() + "." + request.getTableName() + ": " + e.getMessage(), e);
969+
+ request.getDbName() + "." + request.getTableName()
970+
+ ": " + ExceptionUtils.getRootCauseMessage(e), e);
969971
}
970972
}
971973

@@ -1158,7 +1160,8 @@ public void addColumn(ConnectorSession session, ConnectorTableHandle handle,
11581160
});
11591161
} catch (Exception e) {
11601162
throw new DorisConnectorException("Failed to add column " + column.getName() + " to Iceberg table "
1161-
+ iceHandle.getDbName() + "." + iceHandle.getTableName() + ": " + e.getMessage(), e);
1163+
+ iceHandle.getDbName() + "." + iceHandle.getTableName()
1164+
+ ": " + ExceptionUtils.getRootCauseMessage(e), e);
11621165
}
11631166
}
11641167

@@ -1177,7 +1180,8 @@ public void addColumns(ConnectorSession session, ConnectorTableHandle handle, Li
11771180
});
11781181
} catch (Exception e) {
11791182
throw new DorisConnectorException("Failed to add columns to Iceberg table "
1180-
+ iceHandle.getDbName() + "." + iceHandle.getTableName() + ": " + e.getMessage(), e);
1183+
+ iceHandle.getDbName() + "." + iceHandle.getTableName()
1184+
+ ": " + ExceptionUtils.getRootCauseMessage(e), e);
11811185
}
11821186
}
11831187

@@ -1192,7 +1196,8 @@ public void dropColumn(ConnectorSession session, ConnectorTableHandle handle, St
11921196
});
11931197
} catch (Exception e) {
11941198
throw new DorisConnectorException("Failed to drop column " + columnName + " from Iceberg table "
1195-
+ iceHandle.getDbName() + "." + iceHandle.getTableName() + ": " + e.getMessage(), e);
1199+
+ iceHandle.getDbName() + "." + iceHandle.getTableName()
1200+
+ ": " + ExceptionUtils.getRootCauseMessage(e), e);
11961201
}
11971202
}
11981203

@@ -1209,7 +1214,7 @@ public void renameColumn(ConnectorSession session, ConnectorTableHandle handle,
12091214
} catch (Exception e) {
12101215
throw new DorisConnectorException("Failed to rename column " + oldName + " to " + newName
12111216
+ " in Iceberg table " + iceHandle.getDbName() + "." + iceHandle.getTableName()
1212-
+ ": " + e.getMessage(), e);
1217+
+ ": " + ExceptionUtils.getRootCauseMessage(e), e);
12131218
}
12141219
}
12151220

@@ -1255,7 +1260,7 @@ public void modifyColumn(ConnectorSession session, ConnectorTableHandle handle,
12551260
} catch (Exception e) {
12561261
throw new DorisConnectorException("Failed to modify column " + column.getName()
12571262
+ " in Iceberg table " + iceHandle.getDbName() + "." + iceHandle.getTableName()
1258-
+ ": " + e.getMessage(), e);
1263+
+ ": " + ExceptionUtils.getRootCauseMessage(e), e);
12591264
}
12601265
}
12611266

@@ -1299,7 +1304,8 @@ public void reorderColumns(ConnectorSession session, ConnectorTableHandle handle
12991304
});
13001305
} catch (Exception e) {
13011306
throw new DorisConnectorException("Failed to reorder columns in Iceberg table "
1302-
+ iceHandle.getDbName() + "." + iceHandle.getTableName() + ": " + e.getMessage(), e);
1307+
+ iceHandle.getDbName() + "." + iceHandle.getTableName()
1308+
+ ": " + ExceptionUtils.getRootCauseMessage(e), e);
13031309
}
13041310
}
13051311

@@ -1345,7 +1351,7 @@ public void addNestedColumn(ConnectorSession session, ConnectorTableHandle handl
13451351
} catch (Exception e) {
13461352
throw new DorisConnectorException("Failed to add nested column " + path.getFullPath()
13471353
+ " to Iceberg table " + iceHandle.getDbName() + "." + iceHandle.getTableName()
1348-
+ ": " + e.getMessage(), e);
1354+
+ ": " + ExceptionUtils.getRootCauseMessage(e), e);
13491355
}
13501356
}
13511357

@@ -1367,7 +1373,7 @@ public void dropNestedColumn(ConnectorSession session, ConnectorTableHandle hand
13671373
} catch (Exception e) {
13681374
throw new DorisConnectorException("Failed to drop nested column " + path.getFullPath()
13691375
+ " from Iceberg table " + iceHandle.getDbName() + "." + iceHandle.getTableName()
1370-
+ ": " + e.getMessage(), e);
1376+
+ ": " + ExceptionUtils.getRootCauseMessage(e), e);
13711377
}
13721378
}
13731379

@@ -1391,7 +1397,7 @@ public void renameNestedColumn(ConnectorSession session, ConnectorTableHandle ha
13911397
} catch (Exception e) {
13921398
throw new DorisConnectorException("Failed to rename nested column " + path.getFullPath()
13931399
+ " to " + newName + " in Iceberg table " + iceHandle.getDbName() + "."
1394-
+ iceHandle.getTableName() + ": " + e.getMessage(), e);
1400+
+ iceHandle.getTableName() + ": " + ExceptionUtils.getRootCauseMessage(e), e);
13951401
}
13961402
}
13971403

@@ -1440,7 +1446,7 @@ public void modifyNestedColumn(ConnectorSession session, ConnectorTableHandle ha
14401446
} catch (Exception e) {
14411447
throw new DorisConnectorException("Failed to modify nested column " + path.getFullPath()
14421448
+ " in Iceberg table " + iceHandle.getDbName() + "." + iceHandle.getTableName()
1443-
+ ": " + e.getMessage(), e);
1449+
+ ": " + ExceptionUtils.getRootCauseMessage(e), e);
14441450
}
14451451
}
14461452

@@ -1462,7 +1468,7 @@ public void modifyColumnComment(ConnectorSession session, ConnectorTableHandle h
14621468
} catch (Exception e) {
14631469
throw new DorisConnectorException("Failed to modify comment for column " + path.getFullPath()
14641470
+ " in Iceberg table " + iceHandle.getDbName() + "." + iceHandle.getTableName()
1465-
+ ": " + e.getMessage(), e);
1471+
+ ": " + ExceptionUtils.getRootCauseMessage(e), e);
14661472
}
14671473
}
14681474

@@ -1485,7 +1491,7 @@ public void createOrReplaceBranch(ConnectorSession session, ConnectorTableHandle
14851491
} catch (Exception e) {
14861492
throw new DorisConnectorException("Failed to create or replace branch " + branch.getName()
14871493
+ " on Iceberg table " + iceHandle.getDbName() + "." + iceHandle.getTableName()
1488-
+ ": " + e.getMessage(), e);
1494+
+ ": " + ExceptionUtils.getRootCauseMessage(e), e);
14891495
}
14901496
}
14911497

@@ -1501,7 +1507,7 @@ public void createOrReplaceTag(ConnectorSession session, ConnectorTableHandle ha
15011507
} catch (Exception e) {
15021508
throw new DorisConnectorException("Failed to create or replace tag " + tag.getName()
15031509
+ " on Iceberg table " + iceHandle.getDbName() + "." + iceHandle.getTableName()
1504-
+ ": " + e.getMessage(), e);
1510+
+ ": " + ExceptionUtils.getRootCauseMessage(e), e);
15051511
}
15061512
}
15071513

@@ -1517,7 +1523,7 @@ public void dropBranch(ConnectorSession session, ConnectorTableHandle handle, Dr
15171523
} catch (Exception e) {
15181524
throw new DorisConnectorException("Failed to drop branch " + branch.getName()
15191525
+ " from Iceberg table " + iceHandle.getDbName() + "." + iceHandle.getTableName()
1520-
+ ": " + e.getMessage(), e);
1526+
+ ": " + ExceptionUtils.getRootCauseMessage(e), e);
15211527
}
15221528
}
15231529

@@ -1533,7 +1539,7 @@ public void dropTag(ConnectorSession session, ConnectorTableHandle handle, DropR
15331539
} catch (Exception e) {
15341540
throw new DorisConnectorException("Failed to drop tag " + tag.getName()
15351541
+ " from Iceberg table " + iceHandle.getDbName() + "." + iceHandle.getTableName()
1536-
+ ": " + e.getMessage(), e);
1542+
+ ": " + ExceptionUtils.getRootCauseMessage(e), e);
15371543
}
15381544
}
15391545

@@ -1556,7 +1562,8 @@ public void addPartitionField(ConnectorSession session, ConnectorTableHandle han
15561562
});
15571563
} catch (Exception e) {
15581564
throw new DorisConnectorException("Failed to add partition field to Iceberg table "
1559-
+ iceHandle.getDbName() + "." + iceHandle.getTableName() + ": " + e.getMessage(), e);
1565+
+ iceHandle.getDbName() + "." + iceHandle.getTableName()
1566+
+ ": " + ExceptionUtils.getRootCauseMessage(e), e);
15601567
}
15611568
}
15621569

@@ -1572,7 +1579,8 @@ public void dropPartitionField(ConnectorSession session, ConnectorTableHandle ha
15721579
});
15731580
} catch (Exception e) {
15741581
throw new DorisConnectorException("Failed to drop partition field from Iceberg table "
1575-
+ iceHandle.getDbName() + "." + iceHandle.getTableName() + ": " + e.getMessage(), e);
1582+
+ iceHandle.getDbName() + "." + iceHandle.getTableName()
1583+
+ ": " + ExceptionUtils.getRootCauseMessage(e), e);
15761584
}
15771585
}
15781586

@@ -1588,7 +1596,8 @@ public void replacePartitionField(ConnectorSession session, ConnectorTableHandle
15881596
});
15891597
} catch (Exception e) {
15901598
throw new DorisConnectorException("Failed to replace partition field in Iceberg table "
1591-
+ iceHandle.getDbName() + "." + iceHandle.getTableName() + ": " + e.getMessage(), e);
1599+
+ iceHandle.getDbName() + "." + iceHandle.getTableName()
1600+
+ ": " + ExceptionUtils.getRootCauseMessage(e), e);
15921601
}
15931602
}
15941603

‎regression-test/suites/external_table_p0/refactor_storage_param/hdfs_all_test.groovy‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -77,6 +77,7 @@ suite("refactor_params_hdfs_all_test", "p0,external") {
7777
def hdfsNonXmlParams = "\"fs.defaultFS\" = \"hdfs://${externalEnvIp}:8520\",\n" +
7878
"\"dfs.namenode.kerberos.principal\" = \"hdfs/[email protected]\",\n" +
7979
"\"dfs.client.use.datanode.hostname\" = \"true\",\n" +
80+
"\"dfs.data.transfer.protection\" = \"authentication\",\n" +
8081
"\"hadoop.security.token.service.use_ip\" = \"false\",\n" +
8182
"\"hadoop.kerberos.min.seconds.before.relogin\" = \"5\",\n" +
8283
"\"hadoop.security.authentication\" = \"kerberos\",\n" +

‎regression-test/suites/external_table_p2/paimon/test_paimon_hms_catalog.groovy‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -124,6 +124,7 @@ suite("test_paimon_hms_catalog", "p2,external") {
124124
"fs.defaultFS" = "hdfs://${extHiveHmsHost}:8520",
125125
"dfs.namenode.kerberos.principal" = "hdfs/[email protected]",
126126
"dfs.client.use.datanode.hostname" = "true",
127+
"dfs.data.transfer.protection" = "authentication",
127128
"hadoop.security.token.service.use_ip" = "false",
128129
"hadoop.security.authentication" = "kerberos",
129130
"hadoop.kerberos.principal"="hive/[email protected]",
@@ -134,6 +135,7 @@ suite("test_paimon_hms_catalog", "p2,external") {
134135
"fs.defaultFS" = "hdfs://${extHiveHmsHost}:8520",
135136
"dfs.namenode.kerberos.principal" = "hdfs/[email protected]",
136137
"dfs.client.use.datanode.hostname" = "true",
138+
"dfs.data.transfer.protection" = "authentication",
137139
"hadoop.security.token.service.use_ip" = "false",
138140
"hdfs.authentication.type" = "kerberos",
139141
"hdfs.authentication.kerberos.principal"="hive/[email protected]",

‎regression-test/suites/external_table_p2/refactor_catalog_param/hive_on_hms_and_dlf.groovy‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -483,6 +483,7 @@ suite("hive_on_hms_and_dlf", "p2,external") {
483483
"fs.defaultFS" = "hdfs://${externalEnvIp}:8520",
484484
"dfs.namenode.kerberos.principal" = "hdfs/[email protected]",
485485
"dfs.client.use.datanode.hostname" = "true",
486+
"dfs.data.transfer.protection" = "authentication",
486487
"hadoop.security.token.service.use_ip" = "false",
487488
"hadoop.security.authentication" = "kerberos",
488489
@@ -493,6 +494,7 @@ suite("hive_on_hms_and_dlf", "p2,external") {
493494
"fs.defaultFS" = "hdfs://${externalEnvIp}:8520",
494495
"dfs.namenode.kerberos.principal" = "hdfs/[email protected]",
495496
"dfs.client.use.datanode.hostname" = "true",
497+
"dfs.data.transfer.protection" = "authentication",
496498
"hadoop.security.token.service.use_ip" = "false",
497499
"hdfs.authentication.type" = "kerberos",
498500
"hdfs.authentication.kerberos.principal"="hive/[email protected]",

0 commit comments

Comments
 (0)