Skip to content

Commit dd45bd9

Browse files
authored
Flink: Bundle flink-metrics-dropwizard in runtime jar (#16126)
Iceberg uses Dropwizard metrics for Hisograms. Flink does not ship this optional dependency by default. In order for histograms to continue to work, we should add back the runtime dependency removed in #16093.
1 parent d71583f commit dd45bd9

4 files changed

Lines changed: 19 additions & 2 deletions

File tree

‎docs/docs/flink-writes.md‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -207,6 +207,10 @@ They should have the following key-value tags.
207207
| dataFilesSizeHistogram | Histogram | Histogram distribution of data file sizes (in bytes). |
208208
| deleteFilesSizeHistogram | Histogram | Histogram distribution of delete file sizes (in bytes). |
209209

210+
The `Histogram` metrics above require `flink-metrics-dropwizard` on the classpath, which is not shipped
211+
by Flink by default. When using `iceberg-flink-runtime`, this dependency is already bundled. When using
212+
the `iceberg-flink` artifact directly, add `org.apache.flink:flink-metrics-dropwizard` as a dependency.
213+
210214
Committer metrics are added under the sub group of `IcebergFilesCommitter`.
211215
They should have the following key-value tags.
212216

‎flink/v2.1/build.gradle‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -33,7 +33,7 @@ project(":iceberg-flink:iceberg-flink-${flinkMajorVersion}") {
3333
implementation project(':iceberg-hive-metastore')
3434

3535
compileOnly libs.flink21.avro
36-
// for dropwizard histogram metrics implementation
36+
// dropwizard histogram metrics (optional in Flink)
3737
compileOnly libs.flink21.metrics.dropwizard
3838
compileOnly libs.flink21.streaming.java
3939
compileOnly "${libs.flink21.streaming.java.get().module}:${libs.flink21.streaming.java.get().getVersion()}:tests"
@@ -169,6 +169,9 @@ project(":iceberg-flink:iceberg-flink-runtime-${flinkMajorVersion}") {
169169
exclude group: 'com.google.code.findbugs', module: 'jsr305'
170170
}
171171

172+
// To support dropwizard histogram metrics (not shipped by Flink by default)
173+
implementation libs.flink21.metrics.dropwizard
174+
172175
// for integration testing with the flink-runtime-jar
173176
// all of those dependencies are required because the integration test extends FlinkTestBase
174177
integrationCompileOnly project(':iceberg-api')

‎flink/v2.1/flink-runtime/LICENSE‎

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -556,14 +556,22 @@ License: Apache License, Version 2.0 - https://www.apache.org/licenses/LICENSE-2
556556

557557
--------------------------------------------------------------------------------
558558

559-
This product bundles Codahale Metrics.
559+
This product bundles Dropwizard Metrics.
560560

561561
Copyright: (c) 2010-2013 Coda Hale, Yammer.com, 2014-2021 Dropwizard Team
562562
Project URL: https://github.com/dropwizard/metrics
563563
License: Apache License, Version 2.0 - https://www.apache.org/licenses/LICENSE-2.0
564564

565565
--------------------------------------------------------------------------------
566566

567+
This product bundles Apache Flink's optional support for Dropwizard Metrics.
568+
569+
Copyright: 2014-2026 The Apache Software Foundation
570+
Project URL: https://flink.apache.org/
571+
License: Apache License, Version 2.0 - https://www.apache.org/licenses/LICENSE-2.0
572+
573+
--------------------------------------------------------------------------------
574+
567575
This product bundles RoaringBitmap.
568576

569577
Copyright: (c) 2013-... the RoaringBitmap authors

‎flink/v2.1/flink-runtime/runtime-deps.txt‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,9 +6,11 @@ com.github.luben:zstd-jni:1.5.7-3
66
com.google.errorprone:error_prone_annotations:2.10.0
77
dev.failsafe:failsafe:3.3.2
88
io.airlift:aircompressor:2.0.3
9+
io.dropwizard.metrics:metrics-core:3.2.6
910
org.apache.avro:avro:1.12.1
1011
org.apache.datasketches:datasketches-java:6.2.0
1112
org.apache.datasketches:datasketches-memory:3.0.2
13+
org.apache.flink:flink-metrics-dropwizard:2.1.0
1214
org.apache.httpcomponents.client5:httpclient5:5.6
1315
org.apache.httpcomponents.core5:httpcore5-h2:5.4
1416
org.apache.httpcomponents.core5:httpcore5:5.4

0 commit comments

Comments
 (0)