Skip to content

Commit 2b1442f

Browse files
morningmanclaude
andcommitted
fix: FIX-KERBEROS-DOAS — wire fs/jdbc HDFS authenticator (M-8) + wrap all read RPCs in doAs (M-11)
Both are Kerberos-only (harmless on simple-auth: the no-op authenticator's execute() == task.call()). Root cause - M-8 (fe-core): paimon filesystem/jdbc catalogs over Kerberized HDFS lost UGI doAs on the cutover path. The HDFS HadoopExecutionAuthenticator is built only inside initializeCatalog(), which is dead on the plugin path (only legacy PaimonExternalCatalog calls it), so PluginDrivenExternalCatalog read the base no-op from getExecutionAuthenticator(). HMS was unaffected — it wires the authenticator in initNormalizeAndCheckProps(), which always runs. - M-11 (connector): metadata read RPCs (listDatabases/getDatabase/listTables/ getTable[handle+sys+resolveTable]/listPartitions) ran without executeAuthenticated; only the 4 DDL ops were wrapped (signed D7=B read-vs-DDL asymmetry). On a Kerberos HMS catalog these reads ran outside the catalog principal. Legacy wrapped every read. Fix - M-8 (filesystem+jdbc only; DLF/REST/HMS excluded — DLF uses Aliyun STS not Kerberos, the review's "DLF" clause was overstated): new internal fe-core hook MetastoreProperties.initExecutionAuthenticator(List<StorageProperties>) (default no-op), invoked by PluginDrivenExternalCatalog.initPreExecutionAuthenticator from the already-built storage list; filesystem/jdbc override it to build the HDFS authenticator (shared AbstractPaimonProperties helper), mirroring HMS. No connector change; no connector SPI change. - M-11 (full legacy parity, signed D-052, supersedes the D7=B read clause): wrap all 7 connector read RPCs in context.executeAuthenticated. A single resolveTable wrap covers all resolveTable callers (metadata + scan). Domain exceptions are caught INSIDE the lambda because Kerberos UGI.doAs wraps a thrown checked Catalog.*NotExistException in UndeclaredThrowableException. Tests - M-11: PaimonConnectorMetadataReadAuthTest (12) + 2 scan-path tests assert each read runs inside executeAuthenticated (RecordingConnectorContext failAuth/ authCount). Connector module 248/0/0 (1 CI-gated skip). - M-8: Paimon{FileSystem,Jdbc}MetaStorePropertiesTest assert getExecutionAuthenticator() returns HadoopExecutionAuthenticator after wiring without initializeCatalog; fe-core metastore-props 21/0/0 (DLF/HMS regression-clean). - fail-before verified red for both (M-8: stays base no-op AbstractPaimonProperties$1; M-11: authCount/log-empty). - True end-to-end doAs is live-Kerberos-e2e only (no paimon-kerberos suite); DV-031. Decisions D-052 (M-11) / D-053 (M-8); deviation DV-031; design plan-doc/tasks/designs/P5-fix-KERBEROS-DOAS-design.md. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
1 parent 56bc95f commit 2b1442f

15 files changed

Lines changed: 623 additions & 31 deletions

File tree

‎fe/fe-connector/fe-connector-paimon/src/main/java/org/apache/doris/connector/paimon/PaimonConnectorMetadata.java‎

Lines changed: 80 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -92,8 +92,10 @@ public PaimonConnectorMetadata(PaimonCatalogOps catalogOps, Map<String, String>
9292

9393
@Override
9494
public List<String> listDatabaseNames(ConnectorSession session) {
95+
// M-11: wrap the remote read in executeAuthenticated so the FE-injected Kerberos UGI applies
96+
// (legacy PaimonMetadataOps.listDatabaseNames wrapped it too). Full read-vs-DDL parity (D-052).
9597
try {
96-
return catalogOps.listDatabases();
98+
return context.executeAuthenticated(() -> catalogOps.listDatabases());
9799
} catch (Exception e) {
98100
LOG.warn("Failed to list Paimon databases", e);
99101
return Collections.emptyList();
@@ -102,21 +104,38 @@ public List<String> listDatabaseNames(ConnectorSession session) {
102104

103105
@Override
104106
public boolean databaseExists(ConnectorSession session, String dbName) {
107+
// M-11: wrap the remote read in executeAuthenticated (D-052). DatabaseNotExistException is
108+
// caught INSIDE the lambda: under Kerberos UGI.doAs would otherwise wrap the checked
109+
// exception in UndeclaredThrowableException, so an outer catch would not match.
105110
try {
106-
catalogOps.getDatabase(dbName);
107-
return true;
108-
} catch (Catalog.DatabaseNotExistException e) {
109-
return false;
111+
return context.executeAuthenticated(() -> {
112+
try {
113+
catalogOps.getDatabase(dbName);
114+
return true;
115+
} catch (Catalog.DatabaseNotExistException e) {
116+
return false;
117+
}
118+
});
119+
} catch (Exception e) {
120+
throw new DorisConnectorException(
121+
"Failed to check Paimon database existence " + dbName + ": " + e.getMessage(), e);
110122
}
111123
}
112124

113125
@Override
114126
public List<String> listTableNames(ConnectorSession session, String dbName) {
127+
// M-11: wrap the remote read in executeAuthenticated (D-052). DatabaseNotExistException is
128+
// caught INSIDE the lambda (Kerberos UGI.doAs would wrap it otherwise); other failures fall
129+
// to the outer catch, preserving the original empty-list-on-error behavior.
115130
try {
116-
return catalogOps.listTables(dbName);
117-
} catch (Catalog.DatabaseNotExistException e) {
118-
LOG.warn("Database does not exist: {}", dbName);
119-
return Collections.emptyList();
131+
return context.executeAuthenticated(() -> {
132+
try {
133+
return catalogOps.listTables(dbName);
134+
} catch (Catalog.DatabaseNotExistException e) {
135+
LOG.warn("Database does not exist: {}", dbName);
136+
return Collections.<String>emptyList();
137+
}
138+
});
120139
} catch (Exception e) {
121140
LOG.warn("Failed to list tables in database: {}", dbName, e);
122141
return Collections.emptyList();
@@ -127,18 +146,26 @@ public List<String> listTableNames(ConnectorSession session, String dbName) {
127146
public Optional<ConnectorTableHandle> getTableHandle(
128147
ConnectorSession session, String dbName, String tableName) {
129148
Identifier identifier = Identifier.create(dbName, tableName);
149+
// M-11: wrap the remote getTable in executeAuthenticated (D-052). TableNotExistException is
150+
// caught INSIDE the lambda (Kerberos UGI.doAs would wrap it otherwise) and yields an empty
151+
// handle, exactly as before; the trailing handle build is pure (no remote call).
130152
try {
131-
Table table = catalogOps.getTable(identifier);
132-
List<String> partitionKeys = table.partitionKeys();
133-
List<String> primaryKeys = table.primaryKeys();
134-
PaimonTableHandle handle = new PaimonTableHandle(
135-
dbName, tableName,
136-
partitionKeys != null ? partitionKeys : Collections.emptyList(),
137-
primaryKeys != null ? primaryKeys : Collections.emptyList());
138-
handle.setPaimonTable(table);
139-
return Optional.of(handle);
140-
} catch (Catalog.TableNotExistException e) {
141-
return Optional.empty();
153+
return context.executeAuthenticated(() -> {
154+
Table table;
155+
try {
156+
table = catalogOps.getTable(identifier);
157+
} catch (Catalog.TableNotExistException e) {
158+
return Optional.<ConnectorTableHandle>empty();
159+
}
160+
List<String> partitionKeys = table.partitionKeys();
161+
List<String> primaryKeys = table.primaryKeys();
162+
PaimonTableHandle handle = new PaimonTableHandle(
163+
dbName, tableName,
164+
partitionKeys != null ? partitionKeys : Collections.emptyList(),
165+
primaryKeys != null ? primaryKeys : Collections.emptyList());
166+
handle.setPaimonTable(table);
167+
return Optional.<ConnectorTableHandle>of(handle);
168+
});
142169
} catch (Exception e) {
143170
LOG.warn("Failed to get Paimon table handle: {}.{}", dbName, tableName, e);
144171
return Optional.empty();
@@ -287,10 +314,22 @@ public Optional<ConnectorTableHandle> getSysTableHandle(ConnectorSession session
287314
String sys = sysName.toLowerCase(java.util.Locale.ROOT);
288315
Identifier sysId = new Identifier(
289316
base.getDatabaseName(), base.getTableName(), "main", sys);
317+
// M-11: wrap the remote getTable in executeAuthenticated (D-052). TableNotExistException is
318+
// caught INSIDE the lambda (Kerberos UGI.doAs would wrap it otherwise) and signalled out as a
319+
// null Table so this method can still short-circuit to Optional.empty().
290320
Table sysTable;
291321
try {
292-
sysTable = catalogOps.getTable(sysId);
293-
} catch (Catalog.TableNotExistException e) {
322+
sysTable = context.executeAuthenticated(() -> {
323+
try {
324+
return catalogOps.getTable(sysId);
325+
} catch (Catalog.TableNotExistException e) {
326+
return null;
327+
}
328+
});
329+
} catch (Exception e) {
330+
throw new RuntimeException("Failed to load Paimon system table: " + sysId, e);
331+
}
332+
if (sysTable == null) {
294333
return Optional.empty();
295334
}
296335
boolean forceJni = "binlog".equals(sys) || "audit_log".equals(sys);
@@ -889,13 +928,22 @@ private List<ConnectorPartitionInfo> collectPartitions(PaimonTableHandle paimonH
889928
Table table = resolveTable(paimonHandle);
890929
Identifier identifier = Identifier.create(
891930
paimonHandle.getDatabaseName(), paimonHandle.getTableName());
931+
// M-11: wrap the remote listPartitions in executeAuthenticated (D-052), mirroring legacy
932+
// PaimonExternalCatalog.getPaimonPartitions which ran it inside executionAuthenticator.execute
933+
// and swallowed TableNotExistException INSIDE the wrap (Kerberos UGI.doAs would otherwise wrap
934+
// the checked exception, so it must be caught inside).
892935
List<Partition> paimonPartitions;
893936
try {
894-
paimonPartitions = catalogOps.listPartitions(identifier);
895-
} catch (Catalog.TableNotExistException e) {
896-
// Legacy getPaimonPartitions swallows TableNotExistException and returns empty.
897-
LOG.warn("Paimon table not found while listing partitions: {}", identifier, e);
898-
return Collections.emptyList();
937+
paimonPartitions = context.executeAuthenticated(() -> {
938+
try {
939+
return catalogOps.listPartitions(identifier);
940+
} catch (Catalog.TableNotExistException e) {
941+
LOG.warn("Paimon table not found while listing partitions: {}", identifier, e);
942+
return Collections.<Partition>emptyList();
943+
}
944+
});
945+
} catch (Exception e) {
946+
throw new RuntimeException("Failed to list Paimon partitions: " + identifier, e);
899947
}
900948

901949
boolean legacyName = Boolean.parseBoolean(
@@ -983,8 +1031,12 @@ public Optional<ConnectorTableStatistics> getTableStatistics(
9831031
* <p>Preserves this site's original wrapping of a reload failure as a {@link RuntimeException}.
9841032
*/
9851033
private Table resolveTable(PaimonTableHandle paimonHandle) {
1034+
// M-11: wrap the (possibly remote) reload in executeAuthenticated (D-052) so every metadata
1035+
// read path that resolves a table runs under the FE-injected Kerberos UGI. The transient-table
1036+
// fast path inside resolve issues no RPC, so the wrap is a no-op there. The existing catch-all
1037+
// absorbs the (under Kerberos, UGI.doAs-wrapped) reload failure exactly as before.
9861038
try {
987-
return PaimonTableResolver.resolve(catalogOps, paimonHandle);
1039+
return context.executeAuthenticated(() -> PaimonTableResolver.resolve(catalogOps, paimonHandle));
9881040
} catch (Exception e) {
9891041
throw new RuntimeException("Failed to load Paimon table: " + paimonHandle, e);
9901042
}

‎fe/fe-connector/fe-connector-paimon/src/main/java/org/apache/doris/connector/paimon/PaimonScanPlanProvider.java‎

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -183,8 +183,16 @@ static boolean isCppReaderEnabled(ConnectorSession session) {
183183
* net (it is not snapshot-consistent with the handle's originating catalog).
184184
*/
185185
Table resolveTable(PaimonTableHandle paimonHandle) {
186+
// M-11: wrap the (possibly remote) reload in executeAuthenticated (D-052) so the scan path's
187+
// table resolution runs under the FE-injected Kerberos UGI, matching the metadata twin. The
188+
// transient-table fast path issues no RPC; the FileIO split planning below is intentionally
189+
// NOT wrapped (legacy did not wrap it either). When there is no context (offline unit tests
190+
// via the 2-arg ctor), resolve directly — same convention as getScanNodeProperties above.
186191
try {
187-
return PaimonTableResolver.resolve(catalogOps, paimonHandle);
192+
if (context == null) {
193+
return PaimonTableResolver.resolve(catalogOps, paimonHandle);
194+
}
195+
return context.executeAuthenticated(() -> PaimonTableResolver.resolve(catalogOps, paimonHandle));
188196
} catch (Exception e) {
189197
throw new RuntimeException("Failed to load Paimon table: " + paimonHandle, e);
190198
}

0 commit comments

Comments
 (0)