Skip to content

Commit ea7d994

Browse files
committed
Async refresh for Iceberg Cache - fixed tag in the doc & added check for global context
1 parent 856c771 commit ea7d994

File tree

2 files changed

+9
-2
lines changed

2 files changed

+9
-2
lines changed

docs/en/engines/table-engines/integrations/iceberg.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -304,7 +304,7 @@ In order to enable this feature, a non-zero value of milliseconds should be give
304304
If enabled, the server will run a recurring background operation to list the remote catalog and to detect new metadata version. It will then parse it and recursively walk the snapshot, fetching active manifest list files and manifest files.
305305
The files already available at the metadata cache, won't be downloaded again. At the end of each prefetching cycle, the latest metadata snapshot is available at the metadata cache.
306306

307-
```sql
307+
```sql
308308
CREATE TABLE example_table ENGINE = Iceberg(
309309
's3://bucket/path/to/iceberg_table'
310310
) SETTINGS
@@ -315,7 +315,7 @@ In order to make the most of asynchronous metadata prefetching at read operation
315315
By specifying tolerance to metadata staleness, the server is allowed to use the cached version of metadata snapshot without calling the remote catalog. If there's metadata version in cache, and it has been downloaded within the given window of staleness, it will be used to process the query.
316316
Otherwise the latest version will be fetched from the remote catalog.
317317

318-
```
318+
```sql
319319
SELECT count() FROM icebench_table WHERE ...
320320
SETTINGS iceberg_metadata_staleness_ms=120000
321321
```

src/Storages/ObjectStorage/DataLakes/Iceberg/IcebergMetadata.cpp

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -245,6 +245,13 @@ void IcebergMetadata::backgroundMetadataPrefetcherThread()
245245

246246
try
247247
{
248+
if (!Context::getGlobalContextInstance())
249+
{
250+
/// Should never happen, but if seen, this is clear indicator that the task should be started/stopped via startup/shutdown mechanism (check TODOs above)
251+
LOG_DEBUG(log, "backgroundMetadataPrefetcherThread: no global context - skipping");
252+
return;
253+
}
254+
248255
Stopwatch watch;
249256

250257
/// TODO: also we'd want to run all these download operations as separate scheduled tasks - to parallelize it and

0 commit comments

Comments
 (0)