Skip to content

Commit b417c55

Browse files
morningmanclaude
andcommitted
[refactor](catalog) P6 iceberg: migrate to catalog SPI + cutover + remove legacy fe-core subsystem (#64688)
Migrates the legacy in-tree Iceberg catalog (fe-core `datasource/iceberg`, its `property/metastore/*Iceberg*` clusters, and every reverse `instanceof IcebergExternal*` / `case ICEBERG` coupling scattered across nereids / planner / alter) onto the catalog SPI as a self-contained `fe-connector-iceberg` plugin, following the paimon full-adopter + cutover template. `iceberg` is added to `SPI_READY_TYPES`, so an Iceberg catalog now deserializes to `PluginDrivenExternalCatalog` and every functional area — read, write, row-level DML, `ALTER TABLE EXECUTE` procedures, system tables, DDL and MVCC/time-travel — flows through the generic `PluginDriven*` bridge with no Iceberg-specific branch in fe-core. (Iceberg tables surfaced *through* a `hive`/HMS catalog stay on the hive path until P7; this PR covers the dedicated `type=iceberg` catalog.) Iceberg is the widest lakehouse adopter migrated so far — 7 catalog flavors, merge-on-read row-level DML, 9 stored procedures, v3 row-lineage / deletion vectors — so the cutover is held to behavior-parity with the legacy path. Unlike the paimon migration (a cutover PR + a separate removal PR), this PR does the build-out, the live cutover **and** the full legacy-subsystem removal on one branch: fe-core `datasource/iceberg` and its property clusters are deleted, not just left dead. Part of the catalog-SPI migration tracked in #65185 (phase **P6**). Full read + write + row-level DML + procedures + system tables + DDL + MVCC/time-travel, planning native (Parquet/ORC) vs JNI reads. Key seams: - `IcebergCatalogFactory` (pure flavor switch, classloader-pinned for child-first loading), `IcebergCatalogOps` (metadata / DDL ops), `IcebergConnectorMetadata` (schema latest / at-snapshot, sys tables, MVCC/time-travel, DDL, partitions, statistics, table descriptor), `IcebergConnectorProvider` (ServiceLoader entry). - `IcebergScanPlanProvider` + `IcebergScanRange`: predicate/projection pushdown (`IcebergPredicateConverter` → Iceberg `Expression`), FileScanTask byte-offset sub-splitting, COUNT(\*) collapse, merge-on-read delete attachment (position delete / Puffin DV / equality delete), `IcebergColumnHandle` field-id column pruning, schema-evolution-safe reads (`IcebergSchemaUtils` field-id dictionary), synthetic rowid + v3 row-lineage metadata columns. - Write path: `IcebergWritePlanProvider` / `IcebergWriteContext` / `IcebergWriterHelper` / `IcebergConnectorTransaction` — INSERT / INSERT OVERWRITE (dynamic + static partition), branch-targeted writes, WRITE ORDERED BY, per-spec distribution, optimistic-concurrency conflict detection, one SDK transaction per statement, BE commit-fragment → DataFile/DeleteFile conversion. - Row-level DML: DELETE / UPDATE / MERGE INTO over position-delete (v2) and deletion-vector (v3) merge-on-read, baseSnapshot-anchored read. - `action/`: the 9 `ALTER TABLE EXECUTE` procedures (rollback_to_snapshot, rollback_to_timestamp, set_current_snapshot, cherrypick_snapshot, fast_forward, expire_snapshots, publish_changes, rewrite_manifests, rewrite_data_files) dispatched by `IcebergExecuteActionFactory`; `rewrite/` hosts the distributed bin-pack `rewrite_data_files` engine. - `IcebergSchemaBuilder` (CREATE TABLE), `IcebergTypeMapping` (both directions), `IcebergColumnChange` / `IcebergComplexTypeDiff` (schema + partition evolution), `IcebergLatestSnapshotCache` / `IcebergManifestCache` (per-catalog caches on the long-lived connector), `dlf/` (Aliyun DLF client). (`fe-connector-metastore-iceberg`) An Iceberg catalog resolves its backend by `MetaStoreProvider` ServiceLoader dispatch on `iceberg.catalog.type` (registered via `META-INF/services`), instead of a fe-core switch — **7 flavors**: REST, Hive Metastore, AWS Glue, Hadoop / filesystem, JDBC, Aliyun DLF, AWS S3Tables. Adding a backend = one provider class + one services line. The connector self-builds the HMS-metastore Kerberos authenticator, so metastore auth is owned entirely by the plugin. - `CatalogFactory` adds `iceberg` to `SPI_READY_TYPES` and drops the built-in case. - `GsonUtils` replay-compat: persisted `IcebergExternalCatalog` → `PluginDrivenExternalCatalog`, `IcebergExternalDatabase` → `PluginDrivenExternalDatabase`, `IcebergExternalTable` → `PluginDrivenMvccExternalTable` on deserialization, so existing clusters upgrade without losing catalogs (metadata backward-compat invariant). - Nereids/DDL/alter generalized off the concrete Iceberg types; SHOW CREATE renders LOCATION / PARTITION BY / ORDER BY / PROPERTIES via the generic path (iceberg engine name preserved, no credential leak). Per the 2026-07 architecture decision (fe-core holds **no** property parsing for migrated connectors; storage parsing → `fe-filesystem`, metastore parsing → `fe-connector`, both plugin-side; the plugin assembles → BE thrift → hands back to fe-core), iceberg's property/credential/auth resolution moves entirely into the connector: - Storage credentials bound to `fe-filesystem` on both scan and write paths (one source), retiring the second fe-core `StorageProperties.createAll` parse. - Vended temporary credentials, `warehouse` → `fs.defaultFS` derivation (hadoop flavor only, via the neutral `Connector.deriveStorageProperties` SPI), and Kerberos doAs are all connector-owned; the fe-core pre-execution authenticator is a no-op for plugin catalogs. - fe-core's shared static property/metastore helpers are kept alive only for the connectors still on the legacy path (Hive / Hudi / LakeSoul / HMS-iceberg). Net **−21.7k** lines from fe-core. Removes the 4 dead entity classes (`IcebergExternalTable` / `IcebergExternalDatabase` / `IcebergSysExternalTable` / `IcebergExternalCatalog`), the fe-core Iceberg `MetastoreProperties` cluster + its credential helpers, the 5 fe-core Iceberg connectivity probes and the coordinator branches, and un-registers the Iceberg property factories (fail-loud on any stray call). The now-dead paimon `MetastoreProperties` cluster left behind by P5 is swept in the same pass. `GsonUtils` retains only the three string compat labels for old-image upgrade. Parity fixes surfaced by the SPI cutover (each restores legacy semantics): - Classloader: TCCL pinned to the plugin classloader at all three loci where fe-core crosses into bundled Iceberg (scan call thread, write/DDL/commit engine thread via `TcclPinningConnectorContext`, and Iceberg's own manifest-writer worker pool), preventing a duplicate-copy `ClassCastException`. - Nested schema dictionary field names lowercased so a mixed-case nested struct (e.g. Iceberg `DROP_AND_ADD`) can't crash BE with `std::out_of_range`. - Session time zone: lazy `ZoneId` resolution tolerant of Doris CST/PST-style aliases in `FOR TIME AS OF` and datetime partition predicates. - Storage / URI normalization (`oss/cos/obs/s3a` → `s3`), SHOW CREATE secret redaction, and `ALTER TABLE EXECUTE ... WHERE` rejection kept at legacy wording. - ~68 FE unit test classes: iceberg connector (offline recording fakes for the remote Catalog / Table), metastore-iceberg per-flavor dispatch + properties, and the fe-core `PluginDriven*` MVCC / sys-table / scan-node / GSON-replay surfaces; many carry mutation checks. - checkstyle 0 violations; connector import-gate passes (no `fe-connector-iceberg → fe-core` import); FE main + test compile clean. - End-to-end iceberg behavior is docker-gated (`enableIcebergTest=true`); read, write, row-level DML, procedures, system tables, DDL and time-travel are covered by the `external_table_p0/iceberg` (+ `external_table_p2/iceberg`) regression suites. --------- Co-authored-by: Claude Opus 4.8 (1M context) <[email protected]>
1 parent 556712a commit b417c55

690 files changed

Lines changed: 79806 additions & 25567 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎fe/check/checkstyle/import-control.xml‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -43,6 +43,10 @@ under the License.
4343
<allow pkg="org.junit.jupiter"/>
4444
<disallow pkg="org.junit"/>
4545
<disallow pkg="lombok"/>
46+
<disallow pkg="org.apache.iceberg"/>
47+
</subpackage>
48+
<subpackage name="planner">
49+
<disallow pkg="org.apache.iceberg"/>
4650
</subpackage>
4751
</import-control>
4852

‎fe/fe-connector/fe-connector-api/src/main/java/org/apache/doris/connector/api/Connector.java‎

Lines changed: 74 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,13 +17,17 @@
1717

1818
package org.apache.doris.connector.api;
1919

20+
import org.apache.doris.connector.api.handle.WriteOperation;
21+
import org.apache.doris.connector.api.procedure.ConnectorProcedureOps;
2022
import org.apache.doris.connector.api.scan.ConnectorScanPlanProvider;
2123
import org.apache.doris.connector.api.write.ConnectorWritePlanProvider;
2224

2325
import java.io.Closeable;
2426
import java.io.IOException;
2527
import java.util.Collections;
28+
import java.util.EnumSet;
2629
import java.util.List;
30+
import java.util.Map;
2731
import java.util.OptionalLong;
2832
import java.util.Set;
2933

@@ -51,11 +55,81 @@ default ConnectorWritePlanProvider getWritePlanProvider() {
5155
return null;
5256
}
5357

58+
/**
59+
* The write operations the engine may perform on this connector — the single admission source. Reads the
60+
* write provider's {@link ConnectorWritePlanProvider#supportedOperations()}; no provider ⇒ empty set ⇒ all
61+
* writes rejected. The engine consults this instead of {@code getWritePlanProvider() != null}.
62+
*/
63+
default Set<WriteOperation> supportedWriteOperations() {
64+
ConnectorWritePlanProvider p = getWritePlanProvider();
65+
return p == null ? EnumSet.noneOf(WriteOperation.class) : p.supportedOperations();
66+
}
67+
68+
/** Null-safe view of {@link ConnectorWritePlanProvider#supportsWriteBranch()}. No provider ⇒ false. */
69+
default boolean supportsWriteBranch() {
70+
ConnectorWritePlanProvider p = getWritePlanProvider();
71+
return p != null && p.supportsWriteBranch();
72+
}
73+
74+
/** Null-safe view of {@link ConnectorWritePlanProvider#requiresParallelWrite()}. No provider ⇒ false. */
75+
default boolean requiresParallelWrite() {
76+
ConnectorWritePlanProvider p = getWritePlanProvider();
77+
return p != null && p.requiresParallelWrite();
78+
}
79+
80+
/** Null-safe view of {@link ConnectorWritePlanProvider#requiresFullSchemaWriteOrder()}. No provider ⇒ false. */
81+
default boolean requiresFullSchemaWriteOrder() {
82+
ConnectorWritePlanProvider p = getWritePlanProvider();
83+
return p != null && p.requiresFullSchemaWriteOrder();
84+
}
85+
86+
/** Null-safe view of {@link ConnectorWritePlanProvider#requiresPartitionLocalSort()}. No provider ⇒ false. */
87+
default boolean requiresPartitionLocalSort() {
88+
ConnectorWritePlanProvider p = getWritePlanProvider();
89+
return p != null && p.requiresPartitionLocalSort();
90+
}
91+
92+
/**
93+
* Null-safe view of {@link ConnectorWritePlanProvider#requiresMaterializeStaticPartitionValues()}. No
94+
* provider ⇒ false.
95+
*/
96+
default boolean requiresMaterializeStaticPartitionValues() {
97+
ConnectorWritePlanProvider p = getWritePlanProvider();
98+
return p != null && p.requiresMaterializeStaticPartitionValues();
99+
}
100+
101+
/**
102+
* Returns the procedure ops for {@code ALTER TABLE EXECUTE} dispatch, or {@code null} if this
103+
* connector exposes no table procedures. Procedure-side analogue of {@link #getWritePlanProvider()}.
104+
*/
105+
default ConnectorProcedureOps getProcedureOps() {
106+
return null;
107+
}
108+
54109
/** Returns the set of capabilities this connector supports. */
55110
default Set<ConnectorCapability> getCapabilities() {
56111
return Collections.emptySet();
57112
}
58113

114+
/**
115+
* Storage-configuration defaults this connector derives from its own catalog properties, which the raw
116+
* catalog map does not already supply. Design S8: storage-property derivation is owned by the connector —
117+
* fe-core does not parse metastore properties. fe-core folds the returned map into the catalog's storage
118+
* properties as DEFAULTS (an explicit user key always wins via {@code putIfAbsent}), and does so BEFORE
119+
* both the fe-filesystem bind ({@code ConnectorContext.getStorageProperties()}) and the BE storage map
120+
* ({@code getBackendStorageProperties()}), so the FE bind and the BE scan see the same derived storage.
121+
*
122+
* <p>The default is empty (no derivation), so every connector that does not need it is unaffected. The
123+
* iceberg connector overrides this to bridge a hadoop-catalog {@code warehouse=hdfs://<ns>/path} into
124+
* {@code fs.defaultFS=hdfs://<ns>}, which the shared HDFS detection never derives from {@code warehouse}.</p>
125+
*
126+
* @param rawCatalogProps the catalog's current persisted properties
127+
* @return extra storage-property defaults; an empty map when there is nothing to derive
128+
*/
129+
default Map<String, String> deriveStorageProperties(Map<String, String> rawCatalogProps) {
130+
return Collections.emptyMap();
131+
}
132+
59133
/** Returns the table-level property descriptors. */
60134
default List<ConnectorPropertyMetadata<?>> getTableProperties() {
61135
return Collections.emptyList();

‎fe/fe-connector/fe-connector-api/src/main/java/org/apache/doris/connector/api/ConnectorCapability.java‎

Lines changed: 85 additions & 61 deletions
Original file line numberDiff line numberDiff line change
@@ -18,85 +18,109 @@
1818
package org.apache.doris.connector.api;
1919

2020
/**
21-
* Enumerates the optional capabilities a connector may declare.
22-
* The planner and execution engine use these to decide which
23-
* pushdown and write paths are available.
21+
* Enumerates optional, connector-declared capability switches consumed directly by
22+
* static query-planning code (pushdown/DDL/view/statistics gating, etc.).
23+
*
24+
* <p>This is an escape-hatch layer for capability checks that don't warrant a dedicated
25+
* provider abstraction. Write operations and sink traits (parallel write, partition-local
26+
* sort, full-schema write order, static-partition materialization) are NOT declared here —
27+
* they live on the connector's {@link org.apache.doris.connector.api.write.ConnectorWritePlanProvider}
28+
* instead, surfaced via {@link Connector#getWritePlanProvider()}.</p>
2429
*/
2530
public enum ConnectorCapability {
26-
SUPPORTS_FILTER_PUSHDOWN,
27-
SUPPORTS_PROJECTION_PUSHDOWN,
28-
SUPPORTS_LIMIT_PUSHDOWN,
29-
SUPPORTS_PARTITION_PRUNING,
30-
SUPPORTS_INSERT,
31-
SUPPORTS_DELETE,
32-
SUPPORTS_UPDATE,
33-
SUPPORTS_MERGE,
34-
SUPPORTS_CREATE_TABLE,
3531
SUPPORTS_MVCC_SNAPSHOT,
36-
SUPPORTS_METASTORE_EVENTS,
37-
SUPPORTS_STATISTICS,
38-
SUPPORTS_VENDED_CREDENTIALS,
39-
SUPPORTS_ACID_TRANSACTIONS,
40-
SUPPORTS_TIME_TRAVEL,
4132
/**
42-
* Indicates the connector supports multiple concurrent writers (sink instances).
33+
* Indicates the connector supports passthrough query via the {@code query()} TVF.
4334
*
44-
* <p>Connectors that do NOT declare this capability will use GATHER distribution
45-
* (single writer), which is the safe default for transactional sinks like JDBC
46-
* where each writer commits independently.</p>
35+
* <p>Connectors declaring this capability must implement
36+
* {@link ConnectorTableOps#getColumnsFromQuery} to provide column metadata
37+
* for arbitrary SQL queries passed through to the remote data source.</p>
38+
*/
39+
SUPPORTS_PASSTHROUGH_QUERY,
40+
/**
41+
* Indicates the connector exposes per-partition statistics (record count, on-disk size,
42+
* file count) via {@link ConnectorTableOps#listPartitions}.
4743
*
48-
* <p>File-based connectors (Hive, Iceberg, etc.) that can safely handle
49-
* parallel writers should declare this capability.</p>
44+
* <p>{@code SHOW PARTITIONS} renders a rich multi-column result (Partition / PartitionKey /
45+
* RecordCount / FileSizeInBytes / FileCount) for connectors declaring this capability, instead
46+
* of the single partition-name column used by connectors that only implement
47+
* {@code listPartitionNames}.</p>
5048
*/
51-
SUPPORTS_PARALLEL_WRITE,
49+
SUPPORTS_PARTITION_STATS,
5250
/**
53-
* Indicates the connector requires dynamic-partition writes to be hash-distributed by
54-
* partition columns and locally sorted by them before reaching the sink.
51+
* Indicates the connector's tables support background per-column auto-analyze (NDV / min / max /
52+
* null-count collection) through the generic {@code ExternalAnalysisTask} FULL path.
5553
*
56-
* <p>Streaming partition writers (e.g. the MaxCompute Storage API) close the previous
57-
* partition writer as soon as a new partition value appears; un-grouped (unsorted)
58-
* multi-partition rows therefore cause "writer has been closed" errors. The planner uses
59-
* this capability to require a hash-by-partition distribution plus a mandatory local sort
60-
* on the partition columns for dynamic-partition writes.</p>
54+
* <p>The statistics auto-collector admits a plugin-driven table into the background auto-analyze
55+
* framework only when its connector declares this (replacing the legacy {@code instanceof
56+
* IcebergExternalTable} whitelist), and then forces {@code AnalysisMethod.FULL} — sample analyze is
57+
* unimplemented for external SQL-driven tables ({@code ExternalAnalysisTask.doSample} throws).
58+
* Row/passthrough connectors that cannot serve per-column statistics (e.g. JDBC, ES) must NOT
59+
* declare it so they stay excluded.</p>
60+
*/
61+
SUPPORTS_COLUMN_AUTO_ANALYZE,
62+
/**
63+
* Indicates the connector's file-scan tables support Top-N lazy materialization: the scan first
64+
* reads only the ordering/filter columns to locate the Top-N row ids, then materializes the
65+
* remaining columns for just those rows (via the synthesized {@code GLOBAL_ROWID_COL}).
6166
*
62-
* <p>A connector declaring this is expected to also declare
63-
* {@link #SUPPORTS_PARALLEL_WRITE} (hash distribution is inherently parallel) and
64-
* {@link #SINK_REQUIRE_FULL_SCHEMA_ORDER}: the sink distribution locates partition columns by their
65-
* <b>full-schema</b> position in the child output, which only holds when the bind layer projects the
66-
* write to full-schema order (the projection gated by {@code SINK_REQUIRE_FULL_SCHEMA_ORDER}). A
67-
* connector declaring this without {@code SINK_REQUIRE_FULL_SCHEMA_ORDER} would shuffle/sort by the
68-
* wrong column whenever cols order diverges from the full schema.</p>
67+
* <p>The nereids Top-N lazy-materialize probe enables the {@code LazyMaterializeTopN} post-processor
68+
* for a plugin-driven table only when its connector declares this (replacing the legacy exact-class
69+
* {@code SUPPORT_RELATION_TYPES} membership of {@code IcebergExternalTable}). Row/passthrough
70+
* connectors (e.g. JDBC, ES) must NOT declare it.</p>
6971
*/
70-
SINK_REQUIRE_PARTITION_LOCAL_SORT,
72+
SUPPORTS_TOPN_LAZY_MATERIALIZE,
7173
/**
72-
* Indicates the connector's write path maps data columns <b>positionally</b> against the full
73-
* table schema (e.g. MaxCompute's columnar Storage API / JNI writer), rather than by column name.
74+
* Indicates the connector's table/database properties are user-facing and safe to render in
75+
* {@code SHOW CREATE TABLE} / {@code SHOW CREATE DATABASE}.
7476
*
75-
* <p>For such connectors the sink's output rows must be projected to <b>full table schema order</b>
76-
* with any unmentioned columns filled (NULL / default) — exactly like the legacy MaxCompute bind
77-
* path — so that a reordered or partial explicit column list does not land values in the wrong
78-
* remote columns. Name-mapped connectors (e.g. JDBC, which builds an {@code INSERT INTO t (cols)}
79-
* statement) must NOT declare this capability: their data stays in user/cols order to match the
80-
* generated column list.</p>
77+
* <p>The SHOW CREATE TABLE plugin-driven arm renders LOCATION + PROPERTIES (and, when the
78+
* connector pre-renders them under the {@code show.*} reserved keys, the PARTITION BY / ORDER BY
79+
* clauses) only for connectors declaring this (replacing the legacy paimon-only engine-name gate).
80+
* Row/passthrough connectors whose {@code getTableProperties()} returns connection properties
81+
* <b>including credentials</b> (e.g. JDBC, ES) must NOT declare it, or SHOW CREATE TABLE would leak
82+
* the connection password — the security control the legacy engine-name gate provided.</p>
8183
*/
82-
SINK_REQUIRE_FULL_SCHEMA_ORDER,
84+
SUPPORTS_SHOW_CREATE_DDL,
8385
/**
84-
* Indicates the connector supports passthrough query via the {@code query()} TVF.
86+
* Indicates the connector exposes views as queryable objects distinct from tables.
8587
*
86-
* <p>Connectors declaring this capability must implement
87-
* {@link ConnectorTableOps#getColumnsFromQuery} to provide column metadata
88-
* for arbitrary SQL queries passed through to the remote data source.</p>
88+
* <p>When a connector declares this, a plugin-driven table resolves its {@code isView()} from the
89+
* connector ({@link ConnectorTableOps#viewExists}) instead of the {@code false} default, the catalog
90+
* merges the connector's {@link ConnectorTableOps#listViewNames} back into {@code SHOW TABLES} (iceberg
91+
* subtracts views from {@code listTableNames}), and the read/DML/SHOW CREATE arms treat the object as a
92+
* view. Connectors with no view concept (e.g. JDBC, ES) must NOT declare it so every table stays
93+
* {@code isView()==false} and no view round-trips are issued.</p>
8994
*/
90-
SUPPORTS_PASSTHROUGH_QUERY,
95+
SUPPORTS_VIEW,
9196
/**
92-
* Indicates the connector exposes per-partition statistics (record count, on-disk size,
93-
* file count) via {@link ConnectorTableOps#listPartitions}.
97+
* Indicates the connector's file-scan tables support nested-column pruning: a query that reads only some
98+
* sub-fields of a STRUCT/ARRAY/MAP column reads just those leaves from the data file instead of the whole
99+
* complex column (read-amplification avoidance).
94100
*
95-
* <p>{@code SHOW PARTITIONS} renders a rich multi-column result (Partition / PartitionKey /
96-
* RecordCount / FileSizeInBytes / FileCount) for connectors declaring this capability, instead
97-
* of the single partition-name column used by connectors that only implement
98-
* {@code listPartitionNames}. This is distinct from {@link #SUPPORTS_STATISTICS}, which is
99-
* table-level statistics for the optimizer.</p>
101+
* <p>The nereids nested-column-prune probe ({@code LogicalFileScan.supportPruneNestedColumn}) enables it
102+
* for a plugin-driven table only when its connector declares this (replacing the legacy exact-class
103+
* {@code IcebergExternalTable} arm). It is only correct when the connector also carries a stable per-field
104+
* id down its column tree (top-level via {@link ConnectorColumn#withUniqueId} + nested via
105+
* {@link ConnectorType#withChildrenFieldIds}), because the engine rewrites the nested access path from
106+
* field <em>names</em> to those ids ({@code SlotTypeReplacer}) and the BE field-id scan path matches
107+
* nested leaves by id — an un-translated (name / {@code -1}) leaf is skipped and returns NULL. Row/
108+
* passthrough connectors (e.g. JDBC, ES) and connectors that do not carry nested field ids must NOT
109+
* declare it.</p>
110+
*/
111+
SUPPORTS_NESTED_COLUMN_PRUNE,
112+
/**
113+
* Indicates the connector's external metadata (schema / partitions / snapshot) can be pre-warmed
114+
* asynchronously by the planner before it takes the internal read lock, rather than loaded lazily
115+
* during binding.
116+
*
117+
* <p>{@code PluginDrivenExternalTable.supportsExternalMetadataPreload} returns true for a plugin-driven
118+
* table only when its connector declares this (replacing the legacy engine-name {@code "jdbc"} gate), so
119+
* {@code StatementContext.registerExternalTableForPreload} admits the table into the async pre-load pass
120+
* (itself opt-in via the {@code enable_preload_external_metadata} session variable, default off). It is a
121+
* pure planning/lock-latency optimization with no correctness effect: connectors whose metadata reads are
122+
* cheap or not yet validated for concurrent pre-warming (e.g. ES) simply do not declare it and fall back
123+
* to synchronous load at binding time.</p>
100124
*/
101-
SUPPORTS_PARTITION_STATS
125+
SUPPORTS_METADATA_PRELOAD
102126
}

0 commit comments

Comments
 (0)