Skip to content

Commit 635aee0

Browse files
morningmanclaude
andcommitted
[perf](catalog) fe-connector-iceberg: derive the vended storage config once per scan, not per file (PERF-06)
Root cause: for a REST catalog with vended credentials, iceberg scan planning re-derived the whole StorageProperties map (StorageProperties.createAll + a hadoop Configuration build) for EVERY data file and EVERY delete file, via context.normalizeStorageUri(uri, token). But the vended token is scan-invariant (extracted once, the same Map instance threads through every range builder), and the derivation is a pure function of the token, independent of the per-file URI. So a 50k-file scan burned tens of seconds of FE CPU (doubled on merge-on-read) rebuilding an identical map O(N_files + N_deletes) times. Fix (connector-side "prepare once, apply many"): add a scan-scoped SPI seam ConnectorContext.newStorageUriNormalizer(token) that returns a UnaryOperator<String> which bakes the token-derived config in once and then normalizes each URI cheaply. DefaultConnectorContext overrides it to build the effective map lazily-memoized on the first non-empty URI, byte-identical to normalizeStorageUri(uri, token) (same empty-uri short-circuit, same vended-replaces-static precedence, same fail-loud LocationPath, same exception timing). The SPI default folds back to a per-call normalizeStorageUri, so every other connector is unaffected. The iceberg provider builds the normalizer once at each of the three scan-start sites (synchronous data, streaming, position_deletes) and threads it through buildRange / convertDelete / buildPositionDeleteRange in place of the token; TcclPinningConnectorContext delegates the new method to the raw context so iceberg gets the hoist (this path is pure fe-core, no TCCL pin needed). DefaultConnectorContext is per-catalog and shared across queries, so this scan-scoped hoist deliberately keeps no cross-query state and retains no credentials (the normalizer is a scan-local, single-threaded, GC'd at scan end). Not in scope: the write path getBackendFileType is O(1) per write, not per data file. Measurement / guard: one vended scan now derives the config once instead of O(N_files + N_deletes) times. A new connector-side guard test asserts a 3-file scan enters newStorageUriNormalizer exactly once while normalizing all three paths; new fe-core parity tests assert the normalizer's output is byte-identical to normalizeStorageUri across the vended / static / empty-uri / fail-loud cases. Tests: fe-connector-iceberg full module 974 UT green (0 fail / 1 skip); fe-core DefaultConnectorContextNormalizeUriTest 13 UT green. Checkstyle clean. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01SMtYwYyyubZZiC1odLZTG3
1 parent 4ad49b2 commit 635aee0

7 files changed

Lines changed: 216 additions & 46 deletions

File tree

‎fe/fe-connector/fe-connector-iceberg/src/main/java/org/apache/doris/connector/iceberg/IcebergScanPlanProvider.java‎

Lines changed: 45 additions & 36 deletions
Large diffs are not rendered by default.

‎fe/fe-connector/fe-connector-iceberg/src/main/java/org/apache/doris/connector/iceberg/TcclPinningConnectorContext.java‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,7 @@
3030
import java.util.Objects;
3131
import java.util.concurrent.Callable;
3232
import java.util.function.Supplier;
33+
import java.util.function.UnaryOperator;
3334

3435
/**
3536
* A {@link ConnectorContext} decorator that pins the thread-context classloader (TCCL) to the iceberg plugin
@@ -167,6 +168,15 @@ public String normalizeStorageUri(String rawUri, Map<String, String> rawVendedCr
167168
return delegate.normalizeStorageUri(rawUri, rawVendedCredentials);
168169
}
169170

171+
@Override
172+
public UnaryOperator<String> newStorageUriNormalizer(Map<String, String> rawVendedCredentials) {
173+
// Delegate to the raw engine context so the connector gets DefaultConnectorContext's once-per-scan
174+
// hoist. Without this override the SPI default would fold back to THIS wrapper's per-call
175+
// normalizeStorageUri, silently defeating the optimization. Like normalizeStorageUri, this path runs
176+
// entirely in fe-core (LocationPath/StorageProperties, no plugin reflection), so no TCCL pin is needed.
177+
return delegate.newStorageUriNormalizer(rawVendedCredentials);
178+
}
179+
170180
@Override
171181
public String getBackendFileType(String rawUri, Map<String, String> rawVendedCredentials) {
172182
return delegate.getBackendFileType(rawUri, rawVendedCredentials);

‎fe/fe-connector/fe-connector-iceberg/src/test/java/org/apache/doris/connector/iceberg/IcebergScanPlanProviderTest.java‎

Lines changed: 38 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -83,6 +83,7 @@
8383
import java.util.NoSuchElementException;
8484
import java.util.Optional;
8585
import java.util.Set;
86+
import java.util.function.UnaryOperator;
8687

8788
/**
8889
* Tests for {@link IcebergScanPlanProvider}. T01 pinned the capability constants + that {@code planScan}
@@ -1559,7 +1560,7 @@ public void convertDeletePositionDeleteCarriesBoundsAndFormat() {
15591560
// POSITION_DELETES (non-PUFFIN) -> content 1, parquet/orc format, [lower,upper] bounds decoded from the
15601561
// delete file's DELETE_FILE_POS bounds. MUTATION: wrong content id / dropped bounds / wrong format -> red.
15611562
DeleteFile delete = positionDeleteFile("s3://b/db/t1/pos.parquet", FileFormat.PARQUET, 3L, 17L);
1562-
TIcebergDeleteFileDesc d = provider().convertDelete(delete, Collections.emptyMap()).toThrift();
1563+
TIcebergDeleteFileDesc d = provider().convertDelete(delete, UnaryOperator.identity()).toThrift();
15631564

15641565
Assertions.assertEquals(1, d.getContent());
15651566
Assertions.assertEquals("s3://b/db/t1/pos.parquet", d.getPath());
@@ -1575,7 +1576,7 @@ public void convertDeletePositionDeleteWithoutBoundsLeavesThemUnset() {
15751576
// No DELETE_FILE_POS bounds present -> position_lower/upper_bound stay unset (legacy emits them only
15761577
// when present; it stores a -1 sentinel and skips emission). MUTATION: emitting 0/-1 -> red.
15771578
DeleteFile delete = positionDeleteFile("s3://b/db/t1/pos.orc", FileFormat.ORC, null, null);
1578-
TIcebergDeleteFileDesc d = provider().convertDelete(delete, Collections.emptyMap()).toThrift();
1579+
TIcebergDeleteFileDesc d = provider().convertDelete(delete, UnaryOperator.identity()).toThrift();
15791580

15801581
Assertions.assertEquals(TFileFormatType.FORMAT_ORC, d.getFileFormat());
15811582
Assertions.assertFalse(d.isSetPositionLowerBound());
@@ -1588,7 +1589,7 @@ public void convertDeleteDeletionVectorCarriesBlobRefAndUnsetsFormat() {
15881589
// (legacy setDeleteFileFormat skips PUFFIN). MUTATION: classifying it as content 1 / emitting a format
15891590
// for the puffin blob -> red (BE would mis-read the DV blob).
15901591
DeleteFile delete = deletionVectorFile("s3://b/db/t1/dv.puffin", 16L, 64L);
1591-
TIcebergDeleteFileDesc d = provider().convertDelete(delete, Collections.emptyMap()).toThrift();
1592+
TIcebergDeleteFileDesc d = provider().convertDelete(delete, UnaryOperator.identity()).toThrift();
15921593

15931594
Assertions.assertEquals(3, d.getContent());
15941595
Assertions.assertFalse(d.isSetFileFormat());
@@ -1663,7 +1664,7 @@ public void convertDeleteEqualityDeleteCarriesFieldIds() {
16631664
// the T06 data-schema dictionary). MUTATION: wrong content id / dropped field-ids -> red (BE projects
16641665
// the wrong columns for the equality match).
16651666
DeleteFile delete = equalityDeleteFile("s3://b/db/t1/eq.parquet", FileFormat.PARQUET, 1, 2);
1666-
TIcebergDeleteFileDesc d = provider().convertDelete(delete, Collections.emptyMap()).toThrift();
1667+
TIcebergDeleteFileDesc d = provider().convertDelete(delete, UnaryOperator.identity()).toThrift();
16671668

16681669
Assertions.assertEquals(2, d.getContent());
16691670
Assertions.assertEquals(Arrays.asList(1, 2), d.getFieldIds());
@@ -1683,7 +1684,8 @@ public void convertDeleteNormalizesDeletePathViaContext() {
16831684
new IcebergScanPlanProvider(Collections.emptyMap(), new RecordingIcebergCatalogOps(), context);
16841685
DeleteFile delete = positionDeleteFile("oss://bucket/db/t1/pos.parquet", FileFormat.PARQUET, null, null);
16851686

1686-
TIcebergDeleteFileDesc d = provider.convertDelete(delete, Collections.emptyMap()).toThrift();
1687+
TIcebergDeleteFileDesc d =
1688+
provider.convertDelete(delete, provider.newUriNormalizer(Collections.emptyMap())).toThrift();
16871689

16881690
Assertions.assertEquals("s3://bucket/db/t1/pos.parquet", d.getPath());
16891691
Assertions.assertTrue(context.normalizedUris.contains("oss://bucket/db/t1/pos.parquet"));
@@ -2410,6 +2412,32 @@ public void planScanThreadsVendedTokenIntoDataAndDeletePathNormalize() {
24102412
Assertions.assertEquals(IcebergScanPlanProvider.extractVendedToken(table, false), context.lastVendedToken);
24112413
}
24122414

2415+
@Test
2416+
public void planScanDerivesUriNormalizerOncePerScanNotPerFile() {
2417+
// C3 PERF GUARD: the vended token is scan-invariant, so the expensive token->storage-config
2418+
// derivation must be built ONCE per scan and reused for every file path — not rebuilt per data file.
2419+
// Drive a scan over three data files and assert the connector entered newStorageUriNormalizer exactly
2420+
// once while still normalizing all three paths. MUTATION: reverting to a per-file
2421+
// context.normalizeStorageUri (re-deriving the config per file) leaves newNormalizerCount == 0 (the
2422+
// once-per-scan seam is never used) -> red; dropping a path's normalize -> normalizeCount != 3 -> red.
2423+
Table table = createTable("t1", SCHEMA, PartitionSpec.unpartitioned());
2424+
table.newAppend()
2425+
.appendFile(dataFile(table.spec(), "oss://b/db/t1/f1.parquet", 1024, null, null))
2426+
.appendFile(dataFile(table.spec(), "oss://b/db/t1/f2.parquet", 1024, null, null))
2427+
.appendFile(dataFile(table.spec(), "oss://b/db/t1/f3.parquet", 1024, null, null))
2428+
.commit();
2429+
RecordingConnectorContext context = new RecordingConnectorContext();
2430+
IcebergScanPlanProvider provider =
2431+
new IcebergScanPlanProvider(Collections.emptyMap(), opsReturning(table), context);
2432+
2433+
List<ConnectorScanRange> ranges = provider.planScan(
2434+
null, new IcebergTableHandle("db1", "t1"), Collections.emptyList(), Optional.empty());
2435+
2436+
Assertions.assertEquals(3, ranges.size());
2437+
Assertions.assertEquals(1, context.newNormalizerCount);
2438+
Assertions.assertEquals(3, context.normalizeCount);
2439+
}
2440+
24132441
@Test
24142442
public void convertDeleteNormalizesDeletePathViaVendedToken() {
24152443
RecordingConnectorContext context = new RecordingConnectorContext();
@@ -2418,10 +2446,10 @@ public void convertDeleteNormalizesDeletePathViaVendedToken() {
24182446
DeleteFile delete = positionDeleteFile("oss://bucket/db/t1/pos.parquet", FileFormat.PARQUET, null, null);
24192447
Map<String, String> token = Collections.singletonMap("s3.access-key-id", "ak");
24202448

2421-
TIcebergDeleteFileDesc d = provider.convertDelete(delete, token).toThrift();
2449+
TIcebergDeleteFileDesc d = provider.convertDelete(delete, provider.newUriNormalizer(token)).toThrift();
24222450

2423-
// WHY: convertDelete must thread the vended token into the 2-arg normalize (T09). MUTATION: passing no
2424-
// token / the 1-arg normalize -> lastVendedToken != token -> red.
2451+
// WHY: the scan-scoped normalizer must bake in the vended token so the delete path normalizes via the
2452+
// 2-arg seam (T09). MUTATION: dropping the token / the 1-arg normalize -> lastVendedToken != token -> red.
24252453
Assertions.assertEquals("s3://bucket/db/t1/pos.parquet", d.getPath());
24262454
Assertions.assertEquals(token, context.lastVendedToken);
24272455
}
@@ -2509,14 +2537,14 @@ public void convertDeletePositionDeleteTreatsStoredMinusOneBoundAsUnset() {
25092537
// sentinel. The existing no-bounds test passes a null map (early return), never reaching the value==-1L
25102538
// arm. MUTATION: dropping the `|| value == -1L` arm (emitting -1 as a real bound) -> red.
25112539
DeleteFile bothMinusOne = positionDeleteFile("s3://b/db/t1/pos.parquet", FileFormat.PARQUET, -1L, -1L);
2512-
TIcebergDeleteFileDesc d = provider().convertDelete(bothMinusOne, Collections.emptyMap()).toThrift();
2540+
TIcebergDeleteFileDesc d = provider().convertDelete(bothMinusOne, UnaryOperator.identity()).toThrift();
25132541
Assertions.assertEquals(1, d.getContent());
25142542
Assertions.assertFalse(d.isSetPositionLowerBound());
25152543
Assertions.assertFalse(d.isSetPositionUpperBound());
25162544

25172545
// Mixed: only the -1L bound is dropped; a real lower bound still emits.
25182546
DeleteFile mixed = positionDeleteFile("s3://b/db/t1/pos2.parquet", FileFormat.PARQUET, 3L, -1L);
2519-
TIcebergDeleteFileDesc m = provider().convertDelete(mixed, Collections.emptyMap()).toThrift();
2547+
TIcebergDeleteFileDesc m = provider().convertDelete(mixed, UnaryOperator.identity()).toThrift();
25202548
Assertions.assertEquals(3L, m.getPositionLowerBound());
25212549
Assertions.assertFalse(m.isSetPositionUpperBound());
25222550
}

‎fe/fe-connector/fe-connector-iceberg/src/test/java/org/apache/doris/connector/iceberg/RecordingConnectorContext.java‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,7 @@
2929
import java.util.List;
3030
import java.util.Map;
3131
import java.util.concurrent.Callable;
32+
import java.util.function.UnaryOperator;
3233

3334
/**
3435
* Hand-written {@link ConnectorContext} test double (no Mockito), adapted verbatim from the paimon
@@ -60,6 +61,9 @@ final class RecordingConnectorContext implements ConnectorContext {
6061
final List<String> normalizedUris = new ArrayList<>();
6162
/** Number of times the connector invoked {@link #normalizeStorageUri} (1- or 2-arg). */
6263
int normalizeCount;
64+
/** Number of times the connector built a scan-scoped normalizer via {@link #newStorageUriNormalizer}
65+
* (should be once per scan — the perf hoist guard). */
66+
int newNormalizerCount;
6367
/** The vended token the connector passed to the most recent 2-arg {@link #normalizeStorageUri} (T09). */
6468
Map<String, String> lastVendedToken;
6569

@@ -119,6 +123,16 @@ public String normalizeStorageUri(String rawUri, Map<String, String> vendedToken
119123
return rawUri == null ? null : rawUri.replaceFirst("^(oss|cos|obs|s3a)://", "s3://");
120124
}
121125

126+
@Override
127+
public UnaryOperator<String> newStorageUriNormalizer(Map<String, String> vendedToken) {
128+
// Count the once-per-scan derivation (the perf hoist) but still record each per-URI normalize by
129+
// delegating every apply back to the recording normalizeStorageUri — so existing recording assertions
130+
// (normalizedUris / normalizeCount / lastVendedToken) keep firing, while newNormalizerCount proves the
131+
// token->config derivation is entered once per scan, not once per file.
132+
newNormalizerCount++;
133+
return rawUri -> normalizeStorageUri(rawUri, vendedToken);
134+
}
135+
122136
@Override
123137
public List<StorageProperties> getStorageProperties() {
124138
return storageProperties;

‎fe/fe-connector/fe-connector-spi/src/main/java/org/apache/doris/connector/spi/ConnectorContext.java‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,7 @@
2727
import java.util.List;
2828
import java.util.Map;
2929
import java.util.concurrent.Callable;
30+
import java.util.function.UnaryOperator;
3031

3132
/**
3233
* Runtime context provided by fe-core to connector implementations.
@@ -208,6 +209,28 @@ default String normalizeStorageUri(String rawUri, Map<String, String> rawVendedC
208209
return normalizeStorageUri(rawUri);
209210
}
210211

212+
/**
213+
* Scan-scoped batch form of {@link #normalizeStorageUri(String, Map)}: derives the vended storage
214+
* configuration from the (scan-invariant) per-table token ONCE and returns a normalizer that applies
215+
* it to many raw URIs cheaply. A vended-credentials scan normalizes O(N_files + N_deletes) paths but
216+
* the token→storage-config derivation ({@code StorageProperties.createAll} + a hadoop config build) is
217+
* a pure function of the token, so hoisting it out of the per-file loop turns O(N) heavy derivations
218+
* into one. The connector builds the normalizer once (where it extracts the token) and reuses it for
219+
* every data/delete/position-delete path in the scan.
220+
*
221+
* <p>The default returns a normalizer that delegates per call to {@link #normalizeStorageUri(String,
222+
* Map)} — behavior-identical, no hoist — so a connector with no engine context (offline unit tests)
223+
* and any connector that does not override the engine side are unaffected. The engine
224+
* ({@code DefaultConnectorContext}) overrides this to perform the actual once-per-scan derivation.
225+
*
226+
* @param rawVendedCredentials the raw per-table vended token map (may be null/empty → static path)
227+
* @return a URI normalizer for this scan; each application is byte-identical to
228+
* {@link #normalizeStorageUri(String, Map)} with the same token
229+
*/
230+
default UnaryOperator<String> newStorageUriNormalizer(Map<String, String> rawVendedCredentials) {
231+
return rawUri -> normalizeStorageUri(rawUri, rawVendedCredentials);
232+
}
233+
211234
/**
212235
* Resolves the BE-facing file type (a {@code TFileType} enum name, e.g. {@code "FILE_S3"}) for a raw
213236
* storage URI a connector emits (e.g. an iceberg write output path). A write-side analogue of

‎fe/fe-core/src/main/java/org/apache/doris/connector/DefaultConnectorContext.java‎

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -66,6 +66,7 @@
6666
import java.util.concurrent.Callable;
6767
import java.util.function.Function;
6868
import java.util.function.Supplier;
69+
import java.util.function.UnaryOperator;
6970
import java.util.stream.Collectors;
7071

7172
/**
@@ -408,6 +409,37 @@ public String normalizeStorageUri(String rawUri, Map<String, String> rawVendedCr
408409
return LocationPath.of(rawUri, effective).toStorageLocation().toString();
409410
}
410411

412+
@Override
413+
public UnaryOperator<String> newStorageUriNormalizer(Map<String, String> rawVendedCredentials) {
414+
// PERF: the vended token is scan-invariant, so derive the effective storage map (the expensive
415+
// buildVendedStorageMap = StorageProperties.createAll + hadoop config build) ONCE per scan and reuse
416+
// it for every per-file normalize, instead of rebuilding it per data/delete file. Each application is
417+
// byte-identical to normalizeStorageUri(rawUri, token): the SAME empty-uri short-circuit, the SAME
418+
// vended-replaces-static precedence, the SAME fail-loud LocationPath. The derivation is done LAZILY on
419+
// the first non-empty URI (not eagerly at construction) so a scan that normalizes zero non-empty URIs
420+
// triggers no derivation — preserving the exact exception timing of the per-call method. The returned
421+
// normalizer is single-threaded per scan (the streaming pump drives one thread; the synchronous and
422+
// position-delete loops are single-threaded), so the memo needs no lock.
423+
return new UnaryOperator<String>() {
424+
private Map<StorageProperties.Type, StorageProperties> effective;
425+
private boolean built;
426+
427+
@Override
428+
public String apply(String rawUri) {
429+
if (Strings.isNullOrEmpty(rawUri)) {
430+
return rawUri;
431+
}
432+
if (!built) {
433+
Map<StorageProperties.Type, StorageProperties> vended =
434+
buildVendedStorageMap(rawVendedCredentials);
435+
effective = vended != null ? vended : storagePropertiesSupplier.get();
436+
built = true;
437+
}
438+
return LocationPath.of(rawUri, effective).toStorageLocation().toString();
439+
}
440+
};
441+
}
442+
411443
@Override
412444
public String getBackendFileType(String rawUri, Map<String, String> rawVendedCredentials) {
413445
// Same LocationPath build as normalizeStorageUri (vended-aware), then read the BE file type from

‎fe/fe-core/src/test/java/org/apache/doris/connector/DefaultConnectorContextNormalizeUriTest.java‎

Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,7 @@
3030
import java.util.Map;
3131
import java.util.function.Function;
3232
import java.util.function.Supplier;
33+
import java.util.function.UnaryOperator;
3334
import java.util.stream.Collectors;
3435

3536
/**
@@ -169,4 +170,57 @@ public void backendFileTypeVendedRestResolvesUnderEmptyStaticMap() {
169170
Assertions.assertEquals(TFileType.FILE_S3.name(),
170171
restCtx.getBackendFileType("oss://bkt/warehouse/db/t/data", ossVendedToken()));
171172
}
173+
174+
// ---- FIX-PERF-06: newStorageUriNormalizer hoists the (scan-invariant) token->storage-config
175+
// derivation to ONCE per scan; every application must stay byte-identical to a per-call
176+
// normalizeStorageUri(uri, token), across all four cases the per-call form covers. ----
177+
178+
@Test
179+
public void newNormalizerVendedMatchesPerCallAndServesManyUris() {
180+
// WHY: the scan-scoped normalizer bakes the vended token in once, then normalizes many paths;
181+
// each application must equal normalizeStorageUri(uri, token) (REST empty-static -> vended
182+
// replaces static), and ONE normalizer must serve multiple files (the whole point of the hoist).
183+
// MUTATION: dropping the token (static-only) throws; a stale/rebuilt map yielding a different path
184+
// -> red.
185+
DefaultConnectorContext restCtx = new DefaultConnectorContext("c", 1L);
186+
UnaryOperator<String> n = restCtx.newStorageUriNormalizer(ossVendedToken());
187+
Assertions.assertEquals(restCtx.normalizeStorageUri("oss://bkt/a/f1.parquet", ossVendedToken()),
188+
n.apply("oss://bkt/a/f1.parquet"));
189+
Assertions.assertEquals("s3://bkt/a/f1.parquet", n.apply("oss://bkt/a/f1.parquet"));
190+
// Reuse the SAME normalizer for a second, different path — one derivation, many applications.
191+
Assertions.assertEquals("s3://bkt/b/f2.parquet", n.apply("oss://bkt/b/f2.parquet"));
192+
}
193+
194+
@Test
195+
public void newNormalizerStaticMapMatchesPerCallUnderEmptyToken() throws Exception {
196+
// WHY: with a static OSS map and an empty token, the normalizer folds to the static-map path,
197+
// byte-identical to the per-call form. MUTATION: an empty token suppressing the static map -> red.
198+
DefaultConnectorContext ctx = ossContext();
199+
UnaryOperator<String> n = ctx.newStorageUriNormalizer(Collections.emptyMap());
200+
Assertions.assertEquals(
201+
ctx.normalizeStorageUri("oss://bkt/warehouse/db/t/part-0.parquet", Collections.emptyMap()),
202+
n.apply("oss://bkt/warehouse/db/t/part-0.parquet"));
203+
}
204+
205+
@Test
206+
public void newNormalizerShortCircuitsNullAndBlankWithoutForcingDerivation() throws Exception {
207+
// WHY: same empty-uri short-circuit as normalizeStorageUri — a null/blank path returns unchanged
208+
// and never reaches the fail-loud LocationPath, even on an empty static map + empty token (so a
209+
// scan that only ever sees blank uris triggers no derivation/throw). MUTATION: NPE / fabricated
210+
// output / forcing the derivation to throw -> red.
211+
DefaultConnectorContext restCtx = new DefaultConnectorContext("c", 1L);
212+
UnaryOperator<String> n = restCtx.newStorageUriNormalizer(Collections.emptyMap());
213+
Assertions.assertNull(n.apply(null));
214+
Assertions.assertEquals("", n.apply(""));
215+
}
216+
217+
@Test
218+
public void newNormalizerFailsLoudOnBadPathLikePerCall() {
219+
// WHY: fail-loud parity — an empty static map + empty token has no credential, so applying to a
220+
// real oss:// path must throw (not ship the raw path to BE), exactly like normalizeStorageUri.
221+
// MUTATION: swallowing to the raw path -> red.
222+
DefaultConnectorContext restCtx = new DefaultConnectorContext("c", 1L);
223+
UnaryOperator<String> n = restCtx.newStorageUriNormalizer(Collections.emptyMap());
224+
Assertions.assertThrows(RuntimeException.class, () -> n.apply("oss://bkt/a/part-0.parquet"));
225+
}
172226
}

0 commit comments

Comments
 (0)