Repository navigation
[fix](fe) Validate Hive HDFS HA properties during catalog DDL - #68585
Conversation
### What problem does this PR solve?
Issue Number: DORIS-29461
Related PR: None
Problem Summary: Plugin-driven Hive catalogs deferred HDFS HA validation until the first storage access, allowing incomplete dfs.nameservices configurations to be created and persisted. The DDL validation path now binds Hive storage properties for CREATE and ALTER, and rejects nameservice lists without a valid nameservice.
### Release note
Hive Catalog creation and alteration now reject incomplete HDFS HA configuration earlier.
### Check List (For Author)
- Test: Unit Test / FE build
- HdfsPropertiesTest (10 tests)
- ./build.sh --fe
- Behavior changed: Yes (invalid Hive HDFS HA configurations fail during CREATE/ALTER)
- Does this need documentation: No
### What problem does this PR solve? Issue Number: DORIS-29461 Related PR: None Problem Summary: Extend catalog DDL regression coverage to reject incomplete HDFS HA properties and empty nameservice entries. ### Release note None ### Check List (For Author) - Test: FE unit tests and external regression suite - Behavior changed: Yes - Does this need documentation: No
### What problem does this PR solve? Issue Number: DORIS-29461 Related PR: None Problem Summary: Connector/J performs a MySQL compatibility query for auto_increment_increment during connection setup. Resolve the compatibility variable to its default value when the variable registry does not expose it. ### Release note None ### Check List (For Author) - Test: Pending focused FE and external regression tests - Behavior changed: Yes - Does this need documentation: No
### What problem does this PR solve? Issue Number: DORIS-29461 Related PR: None Problem Summary: Update the regression assertion to match the explicit empty nameservice validation error. ### Release note None ### Check List (For Author) - Test: External Hive HA regression suite - Behavior changed: No - Does this need documentation: No
### What problem does this PR solve? Issue Number: DORIS-29461 Related PR: None Problem Summary: Keep Hive catalog DDL validation focused on storage-property validation errors and avoid changing unrelated MySQL compatibility variable behavior. ### Release note None ### Check List (For Author) - Test: Pending focused FE tests - Behavior changed: No - Does this need documentation: No
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
TPC-H: Total hot run time: 28066 ms |
TPC-DS: Total hot run time: 153104 ms |
ClickBench: Total hot run time: 23.9 s |
|
Reviewed head 1. [P2] Relative Hadoop XML resources lose their configured base directory
For a catalog with Please reuse the existing directory-injection behavior or unify the binding entry points so validation and storage access resolve the same files. Add CREATE and ALTER tests using relative resource names under a temporary configured Hadoop directory. 2. [P2] CREATE validation failures leave the already-created connector unclosedThe new check runs from If HA validation throws, This leaves more than a temporary unreachable object: constructing The cleanup gap is in the existing lifecycle, but this PR introduces a new failure path through it. Please either perform storage validation inside the protected creation phase or ensure registration failures close the unregistered catalog outside the catalog lock. Add a lifecycle test that repeatedly rejects CREATE and verifies connector close and no growth in managed-cache registrations. This review was based on code inspection and call-chain analysis; I did not run builds or tests. |
### What problem does this PR solve? Problem Summary: Hive catalog validation of relative Hadoop XML resources could resolve files from the FE working directory instead of the configured Hadoop directory. A rejected CREATE could also leave the constructed connector registered in process resources. Reuse the existing Hadoop directory injection in filesystem binding and creation, and close an unregistered catalog when property validation fails. ### Release note None ### Check List (For Author) - Test: FE CatalogFactoryTest unit test and external_table_p0/test_catalog_ddl regression test; local FE Checkstyle and build - Behavior changed: Yes (relative Hadoop configuration resources resolve against the configured directory and failed CREATE releases catalog resources) - Does this need documentation: No
|
Addressed both P2 issues and pushed the fixes to this PR.
Validation:
Fix commit: |
|
run buildall |
|
/review |
There was a problem hiding this comment.
Review complete for 36555a965a12a9f5dd1c32ad55f17becc29e80b4: request changes for one P1 compatibility regression and two P2 storage/locking issues. I reviewed every changed path, the CREATE/ALTER/replay and runtime binding chains, and both rounds of independent full and risk-focused review. No existing inline threads or additional user focus points were present. All candidate concerns were resolved; the proposed failed-CREATE cleanup concern was dismissed because a distinct blocking effect was not substantiated.
Critical checkpoints:
- Goal and proof: The new HMS DDL bind reaches the HDFS HA checker, so the missing-namenode and missing-address cases in the added FE and regression tests match the intended early error. The tests do not prove compatibility for persisted catalogs or absolute XML resources. This review was static only: I ran no builds or tests, and did not treat the PR's claimed test runs as independent proof.
- Scope and clarity: The DDL validation hook is HMS-specific, but the changed
FileSystemFactorybinding path is shared by connector storage consumers. The change is otherwise localized; the global factory effect is captured by the absolute-resource comment. - Concurrency and locks: CREATE and ALTER hold
CatalogMgr's catalog-wide write lock across HDFS XML file reads and parsing (P2). I found no new shared-state race or lock-order inversion in the detached ALTER candidate or cleanup paths. The possible delay from failed-CREATE HMS pool close was not sufficiently distinct or proven for another inline finding. - Lifecycle and initialization: CREATE validates after connector construction, ALTER validates a detached merged map before publication, and replay intentionally skips DDL checks. No new static-initialization or ownership cycle was found. Runtime binding after replay is precisely where the persisted HA regression occurs.
- Configuration and dynamic behavior: No new configuration item was added. The existing
hadoop_config_dirbridge now changes absolute resource resolution in the factory (P2); no new dynamic-config obligation was found. - Compatibility: A stored
dfs.nameservices=ns1,with otherwise complete HA settings was accepted by the old parser and by Hadoop, but now fails on runtime binding after upgrade (P1). Absolutehadoop.config.resourcespaths now fail in the changed factory path (P2). No FE/BE protocol or storage-format change was introduced. - Parallel paths: I traced CREATE, ALTER, replay, HDFS/JFS/OSS provider selection, manager and ServiceLoader binding, and connector runtime storage access. HMS DDL and runtime provider selection agree for a fixed map.
- Special conditions: The HMS-only guard targets the intended catalog type. The new strict nameservice condition and config-directory injection have the concrete failures noted inline; their rationale does not account for persisted values and absolute paths.
- Test coverage: The new FE cases cover CREATE/ALTER rejection of incomplete HA and the HDFS cases cover empty list tokens; the regression suite checks successive missing keys, one valid catalog, and ALTER rejection. Factory absolute-path and persisted-catalog upgrade coverage are missing.
- Test results: The added assertions and regression error substrings match the checker, and no new
.outresult was added. No test execution occurred in this review. - Observability: The DDL error carries the missing property name; no new metric or log requirement was identified for this validation path.
- Transactions and persistence: Catalog changes are journaled after validation, replay uses stored properties, and there is no new transaction processing or persistent format.
- Data writes: The change validates catalog metadata and does not alter table data writes or crash recovery of writes.
- FE/BE variables: No transmitted variable was added or changed.
- Performance: XML resource reads under the global catalog lock are the confirmed performance issue (P2). The extra map copy in the ServiceLoader fallback is cold-path overhead and was not a separate valuable finding.
- Other issues: No other substantiated changed-line issue remains.
TPC-H: Total hot run time: 27929 ms |
TPC-DS: Total hot run time: 152551 ms |
ClickBench: Total hot run time: 24.03 s |
FE Regression Coverage ReportIncrement line coverage |
|
run vault_p0 |
TPC-H: Total hot run time: 27762 ms |
TPC-DS: Total hot run time: 152253 ms |
ClickBench: Total hot run time: 23.91 s |
FE Regression Coverage ReportIncrement line coverage |
FE UT Coverage ReportIncrement line coverage |
|
/review |
There was a problem hiding this comment.
Static review of PR #68585 at head 734acab1629019c0e811e931749960e1b08ef749. The change moves HMS/HDFS HA property errors to CREATE/ALTER DDL, and the added negative regression cases exercise that path. No distinct new inline findings remain after the full changed-file review, risk scan, independent validation, and one convergence round in which all three review subagents reported NO_NEW_VALUABLE_FINDINGS.
Existing blocking finding: P1 inline comment 4134824627 remains applicable in a narrower replay case. The current guard in HdfsPropertiesUtils.checkHaConfig accepts ns1, but rejects a previously persisted dfs.nameservices=, (or whitespace): before this PR, the split produced no service and binding continued; replay skips DDL validation, while runtime HDFS binding now throws. This is the same stored-catalog compatibility issue already raised in that thread, so I am not reposting it. The existing absolute-resource and global catalog-lock P2 threads are addressed on this head.
Review checkpoints:
- Goal, focus, and tests: CREATE/ALTER now bind HMS storage properties before publication; FE unit tests and the Hive regression suite cover incomplete HA settings. The focus file specifies no additional focus. Validation is static only; no build or test was run in this review.
- Scope and clarity: Changes are confined to DDL validation, filesystem binding/resource resolution, HA compatibility, and focused tests. Existing bindAll and catalog validation paths are reused.
- Concurrency and locks: CREATE validates before the CatalogMgr write lock. ALTER validates a detached property snapshot and rechecks catalog identity and property equality under the write lock before mutation/journaling. Concurrent ALTER, DROP, and RENAME were traced; legacy validators retain in-lock validation. No new lock-order or lifecycle defect was substantiated.
- Lifecycle and cleanup: Failed CREATE validation closes the unpublished catalog; ALTER resets connector/cache state only after validation. No new static-initialization dependency or resource leak was substantiated.
- Configuration and compatibility: No new config item or FE/BE variable is added. Hadoop config directory injection and absolute XML resolution were traced through raw and typed filesystem entry points. The existing P1 replay incompatibility remains as described above; no other mixed-version or storage-format change was found.
- Parallel paths and conditions: HMS raw properties are the effective runtime storage map, and DDL uses the same filesystem binder/provider routing, including explicit support flags and HDFS fallback. The new condition is limited to HMS catalogs; no distinct parallel-path defect was substantiated.
- Test coverage and results: Reviewed all added FE/HDFS unit assertions and the regression suite's CREATE and ALTER errors. The suite does not require a live HMS because
test_connection=false. No result file was added or changed. Test execution was prohibited by the review contract, so runtime success is unverified here. - Observability and errors: Invalid HA settings return specific property names through DDL exceptions; no additional metrics or logs appear necessary for this validation path. No swallowed status or exception was found.
- Persistence, writes, and performance: ALTER journals after validation and publication, replay skips new DDL checks, and no transaction/data-write path changes. Hadoop XML binding is outside the global catalog write lock. No new FE/BE protocol or data-visibility path is involved.
Review status: complete static review; the existing P1 remains blocking.
Existing P0/P1 findings confirmed for this head: #68585 (comment)
|
run buildall |
|
/review |
TPC-H: Total hot run time: 27730 ms |
TPC-DS: Total hot run time: 152228 ms |
There was a problem hiding this comment.
Static review complete for PR #68585 at head 138785c9808a4253f2bf0a8a15bcdb940ee695f8. Two complete review rounds and separate risk scans converged. I found one new P2 issue, attached inline. The three earlier inline concerns were independently rechecked: runtime replay tolerance, absolute Hadoop resource paths, and XML binding under the global catalog lock are addressed at this head. The earlier P1 no longer applies. There were no additional user focus points.
Critical checkpoints:
- Goal and tests: CREATE and ALTER now bind HMS storage properties and reject incomplete HDFS HA settings before persistence. The FE, HDFS, and SQL regression additions exercise invalid CREATE/ALTER cases, but do not cover an unrelated ALTER of a replayed catalog with a legacy empty nameservice list; that is the inline finding. This was a static review; I ran no builds or tests.
- Scope and clarity: The change is focused on DDL validation, lock placement, Hadoop resource resolution, and compatibility. The merged-candidate empty-list guard needs the submitted-key distinction described inline.
- Concurrency and lifecycle: CREATE validates before publication and closes failed or duplicate unregistered catalogs. ALTER validates a detached snapshot outside the global write lock, then rechecks catalog identity and properties under that lock before applying and journaling. Catalog reset and lazy initialization use the catalog monitor. I found no additional lock-order, deadlock, or resource-lifetime issue.
- Configuration and compatibility: No new configuration item, FE-BE variable, function symbol, or storage format is introduced. Runtime HA parsing tolerates persisted empty tokens; the new DDL guard breaks unrelated ALTERs for that legacy value. Relative and absolute XML paths, direct overrides, and explicit filesystem-provider routing were checked.
- Parallel paths and conditions: The DDL and normal HMS table-access paths use the same filesystem binder; replay deliberately skips statement validation. The special empty-list condition is applied to the merged ALTER candidate, which is the identified mismatch.
- Persistence, writes, and failover: Catalog property changes still apply and journal under the write lock; replay uses its nonvalidating path. No table-data write path changes, and no separate failover defect was substantiated.
- Tests, results, and observability: No
.outresult file changed. The new error messages identify the missing HA property; existing logging is adequate for this DDL path. The test suite lacks the legacy unrelated-ALTER case. Author-reported test results were not treated as independent runtime verification. - Performance and other risks: Hadoop XML reads now occur outside the global catalog write lock. No additional concrete performance or correctness defect survived the final nine-file sweep.
Review status: COMPLETE static review with one P2 inline finding and no unresolved candidates.
| if (nameservices != null | ||
| && java.util.Arrays.stream(nameservices.split(",")) | ||
| .map(String::trim).noneMatch(value -> !value.isEmpty())) { | ||
| throw new DdlException("Property dfs.nameservices must contain a nameservice"); |
There was a problem hiding this comment.
[P2] Check empty nameservices only when the ALTER supplies them. A previously stored HMS catalog with dfs.nameservices=, replays and binds successfully because runtime HA parsing now tolerates an empty service list. But validatePropertiesBeforeUpdate passes the merged candidate here, so ALTER CATALOG ... SET PROPERTIES ("test_connection"="false") fails with this error despite not changing HDFS settings. The earlier inline thread addressed runtime readability; this is a separate ALTER regression. Apply this statement-time rejection to an explicitly submitted dfs.nameservices value, while letting untouched legacy values pass the runtime-compatible bind.
ClickBench: Total hot run time: 23.9 s |
|
PR approved by at least one committer and no changes requested. |
### What problem does this PR solve? Issue Number: None Related PR: apache#68585 Problem Summary: A legacy HMS catalog can contain an empty HDFS nameservice list, such as `dfs.nameservices=,`. Runtime binding tolerates this persisted value, but ALTER validation checked the merged catalog properties and rejected even an unrelated statement such as `ALTER CATALOG ... SET PROPERTIES ("test_connection"="false")`. Validate empty nameservice values against the properties submitted by the current statement. CREATE and ALTER still reject explicitly submitted empty values, while unrelated ALTERs preserve legacy values. The complete merged configuration continues through storage binding and HA validation. ### Release note Allow unrelated property changes on legacy Hive catalogs containing an empty HDFS nameservice list. ### Check List (For Author) - Test - [x] Unit Test - Added `PluginDrivenExternalCatalogConcurrencyTest` coverage for legacy comma-only, whitespace, and empty-string nameservice values during unrelated ALTERs, and explicit CREATE/ALTER rejection of these values. - Community FE UT and COMPILE passed on commit `2fcc2ef6e0c93582414a50037394f094012f34f9`. - Local focused FE UT compiled and reached execution, but all 18 tests failed during Mockito initialization because the inline mock maker could not self-attach. This local run is not counted as passing. - [x] Regression test - Community External, P0, NonConcurrent, Cloud P0, and Vault P0 checks passed on the same commit. No new regression suite was added; the legacy persisted-property case is covered by FE unit tests. - Behavior changed: - [ ] No. - [x] Yes. Unrelated ALTERs accept legacy empty nameservice values; explicitly submitted empty values remain rejected. - Does this need documentation? - [x] No. - [ ] Yes.
### What problem does this PR solve? Issue Number: None Related PR: #68585 Problem Summary: A legacy HMS catalog can contain an empty HDFS nameservice list, such as `dfs.nameservices=,`. Runtime binding tolerates this persisted value, but ALTER validation checked the merged catalog properties and rejected even an unrelated statement such as `ALTER CATALOG ... SET PROPERTIES ("test_connection"="false")`. Validate empty nameservice values against the properties submitted by the current statement. CREATE and ALTER still reject explicitly submitted empty values, while unrelated ALTERs preserve legacy values. The complete merged configuration continues through storage binding and HA validation. ### Release note Allow unrelated property changes on legacy Hive catalogs containing an empty HDFS nameservice list. ### Check List (For Author) - Test - [x] Unit Test - Added `PluginDrivenExternalCatalogConcurrencyTest` coverage for legacy comma-only, whitespace, and empty-string nameservice values during unrelated ALTERs, and explicit CREATE/ALTER rejection of these values. - Community FE UT and COMPILE passed on commit `2fcc2ef6e0c93582414a50037394f094012f34f9`. - Local focused FE UT compiled and reached execution, but all 18 tests failed during Mockito initialization because the inline mock maker could not self-attach. This local run is not counted as passing. - [x] Regression test - Community External, P0, NonConcurrent, Cloud P0, and Vault P0 checks passed on the same commit. No new regression suite was added; the legacy persisted-property case is covered by FE unit tests. - Behavior changed: - [ ] No. - [x] Yes. Unrelated ALTERs accept legacy empty nameservice values; explicitly submitted empty values remain rejected. - Does this need documentation? - [x] No. - [ ] Yes. ### Check List (For Reviewer who merge this PR) - [ ] Confirm the release note - [ ] Confirm test cases - [ ] Confirm document - [ ] Add branch pick label
… ALTER CATALOG #69588 (#70091) ### What problem does this PR solve? Issue Number: None Related PR: #69588 (Same backport for branch-4.1: #70092) Problem Summary: Backport of #69588 to branch-4.2. The master change cannot be cherry-picked: it is built on the fe-connector plugin SPI (`ConnectorProvider`, `ConnectorSecuritySurface`, `PluginDrivenExternalCatalog`), and none of those exist on 4.2. This PR ports the security fix itself onto the 4.2 catalog classes, with the same rules and the same error messages. **In short** With the SSRF security SDK configured (`security_checker_class_name`), CREATE / ALTER CATALOG on 4.2 never checked an HMS catalog's metastore endpoint, so a user with CREATE_PRIV on a catalog (no ADMIN) could make the FE's HMS client connect to an intranet address. `HMSExternalCatalog` now runs the same CREATE / ALTER gate master got in #69588 (on ALTER, before `CatalogMgr` takes its write lock), and the SDK binding no longer leaks its net-hook onto pooled FE threads. Without a checker configured (the open-source default), the new gate does nothing. **Background** - `security_checker_class_name` names a third-party SDK (Aliyun `com.aliyun.securitysdk.SecurityUtil`), loaded by reflection through `SecurityChecker` / `UrlSecurityChecker` in fe-common. Its net-hook sits at the JDK socket layer: install it on the current thread, connect, and the connect throws `SSRFUnsafeConnectionException` (a `SecurityException`) if the resolved address is intranet. The SDK has no "is this host intranet" string API, so it can only judge a real connect. Today only HTTP callers use it (HTTP TVF, S3, ES, small-file download, stage creation). - An HMS catalog takes its endpoint from `hive.metastore.uris` or its alias `uri`. `HMSBaseProperties` sets that value, plus every other `hive.`-prefixed catalog property, into the `HiveConf` of the FE's HMS client. `HiveMetaStoreClient` reads the endpoint through `MetastoreConf.getVar`, which looks at `metastore.thrift.uris` first, and through Hadoop's `Configuration.get`, which expands `${...}` placeholders (other properties, `${env.X}`, JVM system properties). **1. The problem, and what it cost** - CREATE CATALOG (CREATE_PRIV) and ALTER CATALOG (ALTER privilege on the catalog) accepted any metastore endpoint, so the FE opened a thrift connection to a host inside its own network on behalf of a non-admin user. - `UrlSecurityChecker` bound the SDK's stop method as `stopSSRFNetHookChecking(String)` but invoked it with no argument, so stop never ran and the net-hook stayed on the pooled FE thread after every bracketed HTTP request. - Only the http form of the SDK's start was bound, so nothing could hook a non-http connection such as HMS thrift. **2. How this PR fixes it** - fe-common, `SecurityChecker`: adds the no-arg `startSSRFChecking()` (the SDK's TCP form), `isEnabled()`, and `checkEndpointForSsrf(host, port)`, which brackets a single 5-second TCP connect with the net-hook. A `SecurityException` from the connect is the rejection; a refused / timed-out / unknown-host connect is ignored, so a public endpoint that is down right now can still be used. `UrlSecurityChecker` binds the SDK's no-arg start and no-arg stop; `DummySecurityChecker.isEnabled()` returns false. The singleton becomes lazily initialized with a test setter. - fe-core, `HMSExternalCatalog`: `checkWhenCreating()` (CREATE, before `test_connection`) and the new `validatePropertiesBeforeUpdateUnlocked()` (ALTER) call the new gate on `hive.metastore.uris`, `uri` and `metastore.thrift.uris`. The rules match master: no-op without a checker; a value containing `${` is rejected before anything else (so an ALTER of only the property a stored placeholder refers to is rejected as well); an ALTER probes only when it touches one of these keys; every entry of a comma-separated HA list is probed; schemed values are parsed with `java.net.URI` host/port semantics, so a query, fragment or userinfo part cannot hide the real destination. - fe-core, `CatalogMgr.alterCatalogProps`: on 4.2 ALTER CATALOG is validated under `CatalogMgr`'s write lock, and a probe to an unreachable endpoint can wait up to 5 seconds, so the probe must not run there. ALTER now takes a snapshot of the catalog's properties under the read lock, runs `ExternalCatalog.validatePropertiesBeforeUpdateUnlocked` (new, a no-op by default) on it without holding the lock, and then applies the change under the write lock only if the same catalog still has the snapshot's properties; if another DDL changed or dropped the catalog in between, it starts over from a new snapshot. The rest of the ALTER validation still runs under the lock, unchanged. CREATE already probes before `CatalogMgr` takes its lock. Where the backport differs from master: | Master (#69588) | This PR | Why | | --- | --- | --- | | Endpoint keys declared by the hive connector through `ConnectorSecuritySurface` | Keys are a constant in `HMSExternalCatalog` | 4.2 has no connector SPI; `HMSExternalCatalog` is already the HMS-specific class | | Gate in `PluginDrivenExternalCatalog` | Gate in `HMSExternalCatalog`, through `checkWhenCreating` (CREATE) and the new `validatePropertiesBeforeUpdateUnlocked` (ALTER) | Neither runs on replay: `CatalogFactory` calls `checkWhenCreating` only for a new catalog, and replay goes through `replayAlterCatalogProps`, not `alterCatalogProps` | | `driver_url` gate moved onto the security surface, `driverUrlKeysOf` deleted | Not ported | 4.2 checks `driver_url` in `JdbcExternalCatalog` / `IcebergExternalCatalog` / `PaimonExternalCatalog.checkProperties`; there is no key table to replace | | ALTER rejected while the connector provider is missing | Not ported | 4.2 has no connector providers | | `checkEndpointForSsrf` throws `DdlException` | Throws `Exception` with the same messages; the HMS gate rethrows it as `DdlException` | On 4.2 `DdlException` is in fe-core, which fe-common cannot reference | | Connector plugin API 22.0 -> 23.0 | No version change | No connector plugin API on 4.2 | | All ALTER validation runs before `CatalogMgr` takes its write lock (#68585) | Only the lock-free checks (the probe) run before the write lock, through `validatePropertiesBeforeUpdateUnlocked`, on a snapshot that is checked again under the lock; the rest of ALTER validation stays under the lock | #68585 is not on 4.2 and also changes HDFS HA validation; this keeps the change to the probe | | `metastore.thrift.uris` gated because the hive connector passes every property to `HiveConf` | Gated as well | `HMSBaseProperties` copies only `hive.`-prefixed properties today, so the key does not reach the client yet; gating it keeps the gate correct if that copy widens, and keeps one rule on both branches | **3. Results** With an SSRF checker configured: | Statement | Before | After | | --- | --- | --- | | `CREATE CATALOG ... ("type"="hms", "hive.metastore.uris"="thrift://10.0.0.1:9083")` | Accepted | Rejected: `Endpoint 10.0.0.1:9083 is rejected by the SSRF policy (intranet address not allowed): ...` | | Same, with `test_connection` = false | Accepted | Rejected (the gate does not depend on `test_connection`) | | `ALTER CATALOG c SET PROPERTIES ("hive.metastore.uris"="thrift://10.0.0.1:9083")` | Accepted | Rejected | | `thrift://[email protected]:9083?x#y`, or an intranet host second in an HA list | Accepted | Rejected | | `"hive.metastore.uris"="thrift://${hive.x}:9083"` with `"hive.x"="127.0.0.1"` | Accepted; the client connects to 127.0.0.1 | Rejected: `... must give its endpoint literally while an SSRF checker is configured ...` | | Public endpoint that is currently unreachable | Accepted | Accepted (a failed connect is not an SSRF signal) | | ALTER of an unrelated property | Accepted | Accepted, no probe | | ALTER that probes an unreachable public endpoint | No probe | Probes for up to 5 seconds without holding `CatalogMgr`'s lock | | Replay of CREATE / ALTER from the edit log | No probe | No probe | Without a checker configured, nothing changes. The only change outside the gate is that the SDK's net-hook is now removed after each bracketed HTTP request, as intended. **4. The classes, and how they call each other** - `SecurityChecker` (fe-common): the SSRF primitive `checkEndpointForSsrf`, plus `isEnabled()` and the TCP start. - `UrlSecurityChecker` / `DummySecurityChecker` (fe-common): SDK binding / no-op. - `HMSExternalCatalog` (fe-core): the endpoint keys, `checkEndpointsAgainstSsrf`, `parseEndpoint`, and the two hooks. - `ExternalCatalog` (fe-core): the new `validatePropertiesBeforeUpdateUnlocked` hook, a no-op by default. - `CatalogMgr` (fe-core): `alterCatalogProps` runs that hook on a snapshot before taking its write lock and checks the snapshot again under it. ``` CREATE CATALOG CatalogFactory.createFromCommand └ HMSExternalCatalog.checkWhenCreating ├ checkEndpointsAgainstSsrf (new) -> SecurityChecker.checkEndpointForSsrf └ super.checkWhenCreating (test_connection) ALTER CATALOG ... SET PROPERTIES CatalogMgr.alterCatalogProps ├ read lock: snapshot the catalog's properties ├ no lock: HMSExternalCatalog.validatePropertiesBeforeUpdateUnlocked (new) │ └ checkEndpointsAgainstSsrf (new) -> SecurityChecker.checkEndpointForSsrf └ write lock: same catalog, same properties? if not, start over └ alterExternalCatalogPropsFenced -> validatePropertiesBeforeUpdate -> checkProperties (unchanged), publish, journal SecurityChecker.checkEndpointForSsrf(host, port) ├ startSSRFChecking() -> SDK startSSRFNetHookChecking() ├ doConnect(host, port, 5s) -> SDK throws SecurityException for an intranet target └ stopSSRFChecking() -> SDK stopSSRFNetHookChecking() Replay: CatalogFactory.createFromLog and replayAlterCatalogProps(isReplay=true) call neither hook. ``` SSRF surface note (per repo policy): the probe is a new FE -> user-named-host outbound TCP connect, made only when the operator installed the SSRF SDK and only at CREATE / ALTER CATALOG (CREATE_PRIV or ADMIN for CREATE, ALTER privilege for ALTER). It exists only so the SDK can reject an intranet target. Known limits, same as master: it is a point-in-time check, so it does not guard a later runtime connection (DNS rebinding) or a client that bypasses the JDK socket layer. ### Release note When an SSRF checker is configured, CREATE / ALTER CATALOG checks HMS endpoint destinations (`hive.metastore.uris`, `uri`, and `metastore.thrift.uris`), including URIs containing query, fragment, or userinfo components, and rejects an HMS endpoint value that contains a `${...}` placeholder. ### Check List (For Author) - Test <!-- At least one of them must be included. --> - [ ] Regression test - [x] Unit Test - [ ] Manual test (add detailed scripts or steps below) - [ ] No need to test or manual test. Explain why: - [ ] This is a refactor/code format and no logic has been changed. - [ ] Previous test can cover this change. - [ ] No code files have been changed. - [ ] Other reason <!-- Add your reason? --> - Unit Test: the new `SecurityCheckerTest` (6 tests) and `HMSExternalCatalogSsrfGateTest` (48), plus the existing `HMSExternalCatalogParquetTimeZoneTest`, `HttpTableValuedFunctionTest`, `CatalogMgrTest`, `CatalogMgrLanceGuardTest`, `ExternalCatalogTest`, `ExternalCatalogRuntimeStateTest`, `CatalogVarbinaryMigrationTest`, `HMSPropertiesTest`, `HiveTableSinkTest`, `JdbcExternalCatalogTest` and `PaimonExternalCatalogDriverUrlSecurityTest`: 134 tests in 13 classes passed with `TZ=Asia/Shanghai mvn test -pl fe-common,fe-core -am`; Checkstyle reports 0 violations in fe-common and fe-core. - `HMSExternalCatalogSsrfGateTest` also drives ALTER through a real `CatalogMgr`. It checks that the probe runs while the thread holds neither the read nor the write lock, that an intranet target leaves the catalog and the journal untouched, and that an ALTER whose catalog is altered or dropped during the probe checks again, or fails with `No catalog found`, instead of applying a stale check. - Mutation check: each of 21 mutants is killed by at least one new test. The mutants drop either hook call, the `isEnabled()` guard of the gate or of `checkEndpointForSsrf`, the touched-key guard, a declared key, the comma split or the URI parsing; run the placeholder rule only on touched keys or ahead of the `isEnabled()` guard; swallow the SDK's rejection or a failed hook install; treat a failed connect as a rejection; skip `stopSSRFChecking()` in `finally`; probe under `CatalogMgr`'s write lock (from `validatePropertiesBeforeUpdate`, or by calling the lock-free hook under the lock); or drop the snapshot comparison or the catalog identity check under the lock. - Checked on this branch that `HMSBaseProperties` hands `hive.metastore.uris` = `thrift://${hive.x}:9083` with `hive.x` = `127.0.0.1` to the HMS client as `thrift://127.0.0.1:9083`, and does not hand it a `metastore.thrift.uris` catalog property. - Behavior changed: - [ ] No. - [x] Yes. With an SSRF checker configured, CREATE / ALTER CATALOG probes HMS metastore endpoints and rejects intranet targets and `${...}` placeholders; the SDK net-hook is now removed after each bracketed request. Without a checker, nothing changes. - Does this need documentation? - [x] No. - [ ] Yes. <!-- Add document PR link here. eg: apache/doris-website#1214 --> ### Check List (For Reviewer who merge this PR) - [ ] Confirm the release note - [ ] Confirm test cases - [ ] Confirm document - [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into --> Co-authored-by: Claude Opus 5.5 <[email protected]>
… ALTER CATALOG #69588 (#70092) ### What problem does this PR solve? Issue Number: None Related PR: #69588 (Same backport for branch-4.2: #70091) Problem Summary: Backport of #69588 to branch-4.1. The master change cannot be cherry-picked: it is built on the fe-connector plugin SPI (`ConnectorProvider`, `ConnectorSecuritySurface`, `PluginDrivenExternalCatalog`), and none of those exist on 4.1. This PR ports the security fix itself onto the 4.1 catalog classes, with the same rules and the same error messages. **In short** With the SSRF security SDK configured (`security_checker_class_name`), CREATE / ALTER CATALOG on 4.1 never checked an HMS catalog's metastore endpoint, so a user with CREATE_PRIV on a catalog (no ADMIN) could make the FE's HMS client connect to an intranet address. `HMSExternalCatalog` now runs the same CREATE / ALTER gate master got in #69588 (on ALTER, before `CatalogMgr` takes its write lock), and the SDK binding no longer leaks its net-hook onto pooled FE threads. Without a checker configured (the open-source default), the new gate does nothing. **Background** - `security_checker_class_name` names a third-party SDK (Aliyun `com.aliyun.securitysdk.SecurityUtil`), loaded by reflection through `SecurityChecker` / `UrlSecurityChecker` in fe-common. Its net-hook sits at the JDK socket layer: install it on the current thread, connect, and the connect throws `SSRFUnsafeConnectionException` (a `SecurityException`) if the resolved address is intranet. The SDK has no "is this host intranet" string API, so it can only judge a real connect. Today only HTTP callers use it (HTTP TVF, S3, ES, small-file download, stage creation). - An HMS catalog takes its endpoint from `hive.metastore.uris` or its alias `uri`. `HMSBaseProperties` sets that value, plus every other `hive.`-prefixed catalog property, into the `HiveConf` of the FE's HMS client. `HiveMetaStoreClient` reads the endpoint through `MetastoreConf.getVar`, which looks at `metastore.thrift.uris` first, and through Hadoop's `Configuration.get`, which expands `${...}` placeholders (other properties, `${env.X}`, JVM system properties). **1. The problem, and what it cost** - CREATE CATALOG (CREATE_PRIV) and ALTER CATALOG (ALTER privilege on the catalog) accepted any metastore endpoint, so the FE opened a thrift connection to a host inside its own network on behalf of a non-admin user. - `UrlSecurityChecker` bound the SDK's stop method as `stopSSRFNetHookChecking(String)` but invoked it with no argument, so stop never ran and the net-hook stayed on the pooled FE thread after every bracketed HTTP request. - Only the http form of the SDK's start was bound, so nothing could hook a non-http connection such as HMS thrift. **2. How this PR fixes it** - fe-common, `SecurityChecker`: adds the no-arg `startSSRFChecking()` (the SDK's TCP form), `isEnabled()`, and `checkEndpointForSsrf(host, port)`, which brackets a single 5-second TCP connect with the net-hook. A `SecurityException` from the connect is the rejection; a refused / timed-out / unknown-host connect is ignored, so a public endpoint that is down right now can still be used. `UrlSecurityChecker` binds the SDK's no-arg start and no-arg stop; `DummySecurityChecker.isEnabled()` returns false. The singleton becomes lazily initialized with a test setter. - fe-core, `HMSExternalCatalog`: `checkWhenCreating()` (CREATE, before `test_connection`) and the new `validatePropertiesBeforeUpdateUnlocked()` (ALTER) call the new gate on `hive.metastore.uris`, `uri` and `metastore.thrift.uris`. The rules match master: no-op without a checker; a value containing `${` is rejected before anything else (so an ALTER of only the property a stored placeholder refers to is rejected as well); an ALTER probes only when it touches one of these keys; every entry of a comma-separated HA list is probed; schemed values are parsed with `java.net.URI` host/port semantics, so a query, fragment or userinfo part cannot hide the real destination. - fe-core, `CatalogMgr.alterCatalogProps`: on 4.1 ALTER CATALOG is validated under `CatalogMgr`'s write lock, and a probe to an unreachable endpoint can wait up to 5 seconds, so the probe must not run there. ALTER now takes a snapshot of the catalog's properties under the read lock, runs `ExternalCatalog.validatePropertiesBeforeUpdateUnlocked` (new, a no-op by default) on it without holding the lock, and then applies the change under the write lock only if the same catalog still has the snapshot's properties; if another DDL changed or dropped the catalog in between, it starts over from a new snapshot. The rest of the ALTER validation still runs under the lock, unchanged. CREATE already probes before `CatalogMgr` takes its lock. Where the backport differs from master: | Master (#69588) | This PR | Why | | --- | --- | --- | | Endpoint keys declared by the hive connector through `ConnectorSecuritySurface` | Keys are a constant in `HMSExternalCatalog` | 4.1 has no connector SPI; `HMSExternalCatalog` is already the HMS-specific class | | Gate in `PluginDrivenExternalCatalog` | Gate in `HMSExternalCatalog`, through `checkWhenCreating` (CREATE) and the new `validatePropertiesBeforeUpdateUnlocked` (ALTER) | Neither runs on replay: `CatalogFactory` calls `checkWhenCreating` only for a new catalog, and replay goes through `replayAlterCatalogProps`, not `alterCatalogProps` | | `driver_url` gate moved onto the security surface, `driverUrlKeysOf` deleted | Not ported | 4.1 checks `driver_url` in `JdbcExternalCatalog` / `IcebergExternalCatalog` / `PaimonExternalCatalog.checkProperties`; there is no key table to replace | | ALTER rejected while the connector provider is missing | Not ported | 4.1 has no connector providers | | `checkEndpointForSsrf` throws `DdlException` | Throws `Exception` with the same messages; the HMS gate rethrows it as `DdlException` | On 4.1 `DdlException` is in fe-core, which fe-common cannot reference | | Connector plugin API 22.0 -> 23.0 | No version change | No connector plugin API on 4.1 | | All ALTER validation runs before `CatalogMgr` takes its write lock (#68585) | Only the lock-free checks (the probe) run before the write lock, through `validatePropertiesBeforeUpdateUnlocked`, on a snapshot that is checked again under the lock; the rest of ALTER validation stays under the lock | #68585 is not on 4.1 and also changes HDFS HA validation; this keeps the change to the probe | | `metastore.thrift.uris` gated because the hive connector passes every property to `HiveConf` | Gated as well | `HMSBaseProperties` copies only `hive.`-prefixed properties today, so the key does not reach the client yet; gating it keeps the gate correct if that copy widens, and keeps one rule on both branches | **3. Results** With an SSRF checker configured: | Statement | Before | After | | --- | --- | --- | | `CREATE CATALOG ... ("type"="hms", "hive.metastore.uris"="thrift://10.0.0.1:9083")` | Accepted | Rejected: `Endpoint 10.0.0.1:9083 is rejected by the SSRF policy (intranet address not allowed): ...` | | Same, with `test_connection` = false | Accepted | Rejected (the gate does not depend on `test_connection`) | | `ALTER CATALOG c SET PROPERTIES ("hive.metastore.uris"="thrift://10.0.0.1:9083")` | Accepted | Rejected | | `thrift://[email protected]:9083?x#y`, or an intranet host second in an HA list | Accepted | Rejected | | `"hive.metastore.uris"="thrift://${hive.x}:9083"` with `"hive.x"="127.0.0.1"` | Accepted; the client connects to 127.0.0.1 | Rejected: `... must give its endpoint literally while an SSRF checker is configured ...` | | Public endpoint that is currently unreachable | Accepted | Accepted (a failed connect is not an SSRF signal) | | ALTER of an unrelated property | Accepted | Accepted, no probe | | ALTER that probes an unreachable public endpoint | No probe | Probes for up to 5 seconds without holding `CatalogMgr`'s lock | | Replay of CREATE / ALTER from the edit log | No probe | No probe | Without a checker configured, nothing changes. The only change outside the gate is that the SDK's net-hook is now removed after each bracketed HTTP request, as intended. **4. The classes, and how they call each other** - `SecurityChecker` (fe-common): the SSRF primitive `checkEndpointForSsrf`, plus `isEnabled()` and the TCP start. - `UrlSecurityChecker` / `DummySecurityChecker` (fe-common): SDK binding / no-op. - `HMSExternalCatalog` (fe-core): the endpoint keys, `checkEndpointsAgainstSsrf`, `parseEndpoint`, and the two hooks. - `ExternalCatalog` (fe-core): the new `validatePropertiesBeforeUpdateUnlocked` hook, a no-op by default. - `CatalogMgr` (fe-core): `alterCatalogProps` runs that hook on a snapshot before taking its write lock and checks the snapshot again under it. ``` CREATE CATALOG CatalogFactory.createFromCommand └ HMSExternalCatalog.checkWhenCreating ├ checkEndpointsAgainstSsrf (new) -> SecurityChecker.checkEndpointForSsrf └ super.checkWhenCreating (test_connection) ALTER CATALOG ... SET PROPERTIES CatalogMgr.alterCatalogProps ├ read lock: snapshot the catalog's properties ├ no lock: HMSExternalCatalog.validatePropertiesBeforeUpdateUnlocked (new) │ └ checkEndpointsAgainstSsrf (new) -> SecurityChecker.checkEndpointForSsrf └ write lock: same catalog, same properties? if not, start over └ alterExternalCatalogPropsFenced -> validatePropertiesBeforeUpdate -> checkProperties (unchanged), publish, journal SecurityChecker.checkEndpointForSsrf(host, port) ├ startSSRFChecking() -> SDK startSSRFNetHookChecking() ├ doConnect(host, port, 5s) -> SDK throws SecurityException for an intranet target └ stopSSRFChecking() -> SDK stopSSRFNetHookChecking() Replay: CatalogFactory.createFromLog and replayAlterCatalogProps(isReplay=true) call neither hook. ``` SSRF surface note (per repo policy): the probe is a new FE -> user-named-host outbound TCP connect, made only when the operator installed the SSRF SDK and only at CREATE / ALTER CATALOG (CREATE_PRIV or ADMIN for CREATE, ALTER privilege for ALTER). It exists only so the SDK can reject an intranet target. Known limits, same as master: it is a point-in-time check, so it does not guard a later runtime connection (DNS rebinding) or a client that bypasses the JDK socket layer. ### Release note When an SSRF checker is configured, CREATE / ALTER CATALOG checks HMS endpoint destinations (`hive.metastore.uris`, `uri`, and `metastore.thrift.uris`), including URIs containing query, fragment, or userinfo components, and rejects an HMS endpoint value that contains a `${...}` placeholder. ### Check List (For Author) - Test <!-- At least one of them must be included. --> - [ ] Regression test - [x] Unit Test - [ ] Manual test (add detailed scripts or steps below) - [ ] No need to test or manual test. Explain why: - [ ] This is a refactor/code format and no logic has been changed. - [ ] Previous test can cover this change. - [ ] No code files have been changed. - [ ] Other reason <!-- Add your reason? --> - Unit Test: the new `SecurityCheckerTest` (6 tests) and `HMSExternalCatalogSsrfGateTest` (48), plus the existing `HMSExternalCatalogParquetTimeZoneTest`, `HttpTableValuedFunctionTest`, `CatalogMgrTest`, `CatalogMgrLanceGuardTest`, `ExternalCatalogTest`, `ExternalCatalogRuntimeStateTest`, `CatalogVarbinaryMigrationTest`, `HMSPropertiesTest`, `HiveTableSinkTest`, `JdbcExternalCatalogTest` and `PaimonExternalCatalogDriverUrlSecurityTest`: 134 tests in 13 classes passed with `TZ=Asia/Shanghai mvn test -pl fe-common,fe-core -am`; Checkstyle reports 0 violations in fe-common and fe-core. - `HMSExternalCatalogSsrfGateTest` also drives ALTER through a real `CatalogMgr`. It checks that the probe runs while the thread holds neither the read nor the write lock, that an intranet target leaves the catalog and the journal untouched, and that an ALTER whose catalog is altered or dropped during the probe checks again, or fails with `No catalog found`, instead of applying a stale check. - Mutation check: each of 21 mutants is killed by at least one new test. The mutants drop either hook call, the `isEnabled()` guard of the gate or of `checkEndpointForSsrf`, the touched-key guard, a declared key, the comma split or the URI parsing; run the placeholder rule only on touched keys or ahead of the `isEnabled()` guard; swallow the SDK's rejection or a failed hook install; treat a failed connect as a rejection; skip `stopSSRFChecking()` in `finally`; probe under `CatalogMgr`'s write lock (from `validatePropertiesBeforeUpdate`, or by calling the lock-free hook under the lock); or drop the snapshot comparison or the catalog identity check under the lock. - Checked on this branch that `HMSBaseProperties` hands `hive.metastore.uris` = `thrift://${hive.x}:9083` with `hive.x` = `127.0.0.1` to the HMS client as `thrift://127.0.0.1:9083`, and does not hand it a `metastore.thrift.uris` catalog property. - Behavior changed: - [ ] No. - [x] Yes. With an SSRF checker configured, CREATE / ALTER CATALOG probes HMS metastore endpoints and rejects intranet targets and `${...}` placeholders; the SDK net-hook is now removed after each bracketed request. Without a checker, nothing changes. - Does this need documentation? - [x] No. - [ ] Yes. <!-- Add document PR link here. eg: apache/doris-website#1214 --> ### Check List (For Reviewer who merge this PR) - [ ] Confirm the release note - [ ] Confirm test cases - [ ] Confirm document - [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into --> Co-authored-by: Claude Opus 5.5 <[email protected]>
What problem does this PR solve?
Issue Number: DORIS-29461
Related PR: None
Problem Summary: Plugin-driven Hive catalogs accepted incomplete HDFS HA properties during CREATE and ALTER. For example, a catalog with
dfs.nameservices=ns1but nodfs.ha.namenodes.ns1, or with an empty nameservice list, could be persisted successfully and only fail when the first external table accessed storage. The DDL validation path now binds Hive storage properties before persisting the catalog, and HDFS HA validation rejects empty nameservice entries and validates the required NameNode, RPC address, and failover provider properties. The behavior is covered by FE unit tests and an external regression suite.Release note
Hive Catalog CREATE and ALTER now reject incomplete HDFS HA configurations during DDL validation.
Check List (For Author)
PluginDrivenExternalCatalogConcurrencyTest,HdfsPropertiesTestexternal_table_p0/hive/test_hive_ha_catalog_validation