Kafka Connect: Reconcile the Hive distribution LICENSE and NOTICE by kevinjqliu · Pull Request #18663 · apache/iceberg

kevinjqliu · 2026-10-27T03:28:12Z

Follow up to #18660

Fixes Kafka Connect runtime's hive distribution LICENSE and NOTICE. main distribution was fixed in #15449 and #18019.

The hive distribution is a superset of main: it bundles every main dep at the same version, plus 95 more for the Hive Metastore client. So hive/LICENSE and hive/NOTICE take all existing entries from main/LICENSE and main/NOTICE verbatim, and append a section for the 95 hive-only deps. main files are unchanged. All three hive files are add-only vs main:

cd kafka-connect/kafka-connect-runtime
diff main/runtime-deps.txt hive/runtime-deps.txt  # 95 added, 0 removed
diff main/LICENSE hive/LICENSE                    # 435 added, 0 removed
diff main/NOTICE hive/NOTICE                      # 307 added, 0 removed

logback-classic and slf4j-reload4j are SLF4J 1.x bindings that slf4j-api 2.0 ignores. Excluding them from hive will be a follow up.

How each added dep is attributed

Each of the 95 added deps maps to a hive/LICENSE entry; lines up to 2525 are inherited from main. Licenses were checked against each jar's POM and bundled LICENSE file, if any. Jars were scanned for shaded packages, native libs and web assets.

runtime-deps.txt additions → hive/LICENSE

Added deps	#	`hive/LICENSE` entry (line)	License
hive-common, hive-metastore, hive-serde, hive-shims, hive-shims-0.23/common/scheduler, hive-storage-api	8	Apache Hive (2534)	Apache-2.0
hadoop-auth, hadoop-client, hadoop-hdfs-client, hadoop-mapreduce-client-*, hadoop-yarn-api/client	8	Apache Hadoop (2541)	Apache-2.0
hadoop-yarn-common	1	Apache Hadoop (2541), jQuery/jQuery UI/DataTables/jsTree (2548)	Apache-2.0, MIT
libthrift, libfb303	2	Apache Thrift (2582)	Apache-2.0
zookeeper, zookeeper-jute	2	Apache ZooKeeper (2589)	Apache-2.0
curator-client/framework/recipes	3	Apache Curator (2596)	Apache-2.0
kerb-, kerby-	7	Apache Kerby (2603)	Apache-2.0
audience-annotations	1	Apache Yetus (2610)	Apache-2.0
derby	1	Apache Derby (2617)	Apache-2.0
jdo-api	1	Apache JDO (2624)	Apache-2.0
ant, ant-launcher	2	Apache Ant (2631)	Apache-2.0
commons-configuration2	1	Apache Commons Configuration (2638)	Apache-2.0
commons-dbcp	1	Apache Commons DBCP (2645)	Apache-2.0
commons-lang	1	Apache Commons Lang (1079, from main)	Apache-2.0
tomcat-embed-core	1	Apache Tomcat (2652), Java EE XML schemas (2659)	Apache-2.0, CDDL-1.0
tomcat-annotations-api	1	Apache Tomcat (2652)	Apache-2.0
jta	1	JTA (2666)	CDDL-1.0
reload4j	1	Reload4j (2673)	Apache-2.0
slf4j-reload4j	1	SLF4J (2372, from main)	MIT
logback-classic, logback-core	2	Logback (2680)	EPL-1.0 (or LGPL-2.1)
datanucleus-core	1	DataNucleus (2699), ASM (2706)	Apache-2.0, BSD-3
datanucleus-api-jdo, datanucleus-rdbms, javax.jdo	3	DataNucleus (2699)	Apache-2.0
jetty-, websocket-	12	Eclipse Jetty (2743)	Apache-2.0 (or EPL-1.0)
guice	1	Google Guice (2750), ASM (2706), cglib (2757)	Apache-2.0, BSD-3
javax.inject	1	javax.inject (2764)	Apache-2.0
aopalliance	1	AOP Alliance (2771)	Public Domain
bonecp	1	Jolbox BoneCP (2778)	Apache-2.0
joda-time	1	Joda-Time (2785)	Apache-2.0
nimbus-jose-jwt	1	Nimbus JOSE+JWT (2792), JCIP (2799), Gson (843, from main)	Apache-2.0
opencsv	1	OpenCSV (2806)	Apache-2.0
jline:jline	1	JLine (2813), Jansi and HawtJNI (2857)	BSD-3, Apache-2.0
org.jline:jline	1	JLine (2813)	BSD-3
antlr-runtime	1	ANTLR 3 runtime (2865)	BSD-3
javolution	1	Javolution (2899), FDLIBM (2913)	BSD, FDLIBM
metrics-json, metrics-jvm	2	Codahale Metrics (1158, from main)	Apache-2.0
jackson-jaxrs-base, jackson-jaxrs-json-provider, jackson-module-jaxb-annotations	3	Jackson JSON Processor (315, from main)	Apache-2.0
netty-all + 15 other netty codec/transport jars	16	Netty (1263, from main)	Apache-2.0
netty-codec-native-quic	1	Netty (1263, from main), quiche (2933), BoringSSL (1178, from main)	Apache-2.0, BSD-2, BSD-3

hive/NOTICE additions

Every non-ASF NOTICE line in the added jars is in hive/NOTICE, either inherited from main or in the appended section. The appended sections are copied verbatim from the jar's META-INF/NOTICE.

`hive/NOTICE` entry (line)	Source
netty-codec-native-quic (769)	lines not already in main's Netty NOTICE
Apache Derby (787)	header and IBM copyrights. Skips NIST tests, Felix, Lucene and json-simple, which are build-only and not in `derby.jar`
Apache Ant (806)	full
Apache Tomcat (816)	full
DataNucleus Core (847)	full
DataNucleus RDBMS (888)	full
Eclipse Jetty (931)	full, same NOTICE in all 12 jetty jars
Google Guice (1053)	full
Joda-Time (1062)	full

The rest have no NOTICE, ASF boilerplate only, or a NOTICE already covered by main (e.g. the Jackson JAX-RS/JAXB jars carry an older wording of the Jackson NOTICE).

AI Disclosure

Model: Claude Opus 5.5
Platform/Tool: GitHub Copilot
Human Oversight: [unknown - human to fill in]
Prompt Summary: Fix the Kafka Connect hive distribution LICENSE and NOTICE on top of the main distribution's, and verify every added runtime dependency is attributed

jbonofre · 2026-11-09T14:37:55Z

-
-Project URL: https://commons.apache.org/proper/commons-beanutils/
-License: Apache License, Version 2.0 - https://www.apache.org/licenses/LICENSE-2.0
+License: CDDL 1.0 - https://spdx.org/licenses/CDDL-1.0.html


This changes the JAXB license from CDDL 1.1 to CDDL 1.0, which doesn't match the jar we ship now.

jaxb-api 2.2.11 declares Bundle-License: https://glassfish.java.net/public/CDDL+GPL_1_1.html in its manifest. CDDL 1.0 was only correct for jaxb-api 2.2.2.
The previous hive entry was right.

main/LICENSE needs the same fix.

Suggested change

License: CDDL 1.0 - https://spdx.org/licenses/CDDL-1.0.html

License: CDDL 1.1 - https://glassfish.java.net/public/CDDL+GPL_1_1.html

this entry comes from main, will follow up there since it affects both

jbonofre · 2026-11-09T14:40:21Z

-
--------------------------------------------------------------------------------
-
-This product bundles reload4j with the following in its NOTICE file:


This reload4j section is removed, but reload4j and slf4j-reload4j are still in hive/runtime-deps.txt. The reload4j jar ships a META-INF/NOTICE.
Even if the NOTICE is the one from Apache, reload4j is not ASF project, so we should keep it.

Can this section be restored?

main skips the same case (analyticsaccelerator-s3's NOTICE is only an ASF copyright), so i left it out. will follow up for both

The Hive distribution inherits the main distribution's dependencies but was resolved on its own, so the Hive Metastore client's dependencies pulled newer commons-logging, jaxb-api and jakarta.activation-api and dropped stax-api. Resolve hive consistently with runtimeClasspath so it is the main distribution plus the Hive Metastore client and its dependencies. Generated-by: GitHub Copilot (Claude Opus 5.5) Co-authored-by: Copilot App <[email protected]>

The Hive distribution's LICENSE and NOTICE predate apache#18019 and no longer match what it bundles. The Hive distribution is now the main one plus the Hive Metastore client and its dependencies, so its LICENSE and NOTICE are the main ones followed by entries for those additions. The added entries cover the Hive Metastore client and its Hadoop, Thrift, ZooKeeper, DataNucleus, Derby, Jetty, Tomcat and logging dependencies, plus code bundled inside them, such as ASM, FDLIBM, quiche and the YARN web UI's jQuery, DataTables and jsTree. Generated-by: GitHub Copilot (Claude Opus 5.5) Co-authored-by: Copilot App <[email protected]>

kevinjqliu · 2026-11-13T21:57:19Z

 commons-io:commons-io:2.20
 commons-lang:commons-lang:2.6
-commons-logging:commons-logging:1.3
+commons-logging:commons-logging:1.2


align this with main distribution's runtime-deps.txt file

iceberg/kafka-connect/kafka-connect-runtime/main/runtime-deps.txt

Line 86 in d1a4229

commons-logging:commons-logging:1.2

kevinjqliu · 2026-11-13T21:57:43Z

 javax.servlet:javax.servlet-api:3.1
 javax.transaction:jta:1.1
 javax.xml.bind:jaxb-api:2.2
+javax.xml.stream:stax-api:1.0-2


align this with main distribution's runtime-deps.txt file

iceberg/kafka-connect/kafka-connect-runtime/main/runtime-deps.txt

Lines 161 to 163 in d1a4229

javax.xml.bind:jaxb-api:2.2

javax.xml.stream:stax-api:1.0-2

net.java.dev.jna:jna-platform:5.17

kevinjqliu · 2026-11-13T22:02:15Z

+      // keep the hive distribution a superset of main by resolving shared jars to the same versions
+      shouldResolveConsistentlyWith(runtimeClasspath)


@danielcweeks had a comment about this in #18660 (comment)

extendsFrom adds main's dependencies to hive, but hive still resolves them together with the Hive Metastore client's dependencies. When both ask for the same jar, Gradle picks the newer version, so hive ended up with different versions than main for a few shared jars.

This change ensures the versions are all aligned with main. See the diff in hive/runtime-deps.txt in this PR.

kevinjqliu · 2026-11-13T22:48:06Z

-
-Project URL: https://commons.apache.org/proper/commons-beanutils/
-License: Apache License, Version 2.0 - https://www.apache.org/licenses/LICENSE-2.0
+License: CDDL 1.0 - https://spdx.org/licenses/CDDL-1.0.html


this entry comes from main, will follow up there since it affects both

kevinjqliu · 2026-11-13T22:48:12Z

-
--------------------------------------------------------------------------------
-
-This product bundles reload4j with the following in its NOTICE file:


main skips the same case (analyticsaccelerator-s3's NOTICE is only an ASF copyright), so i left it out. will follow up for both

The Hive distribution bundled logback-classic and slf4j-reload4j, which are SLF4J 1.x bindings that slf4j-api 2.x ignores. Exclude them and drop the Logback LICENSE entry. Generated-by: GitHub Copilot (Claude Opus 5.5) Co-authored-by: Copilot App <[email protected]>

kevinjqliu · 2026-11-13T22:55:02Z

+      // SLF4J 1.x bindings are ignored by slf4j-api 2.x
+      exclude group: 'ch.qos.logback'
+      exclude group: 'org.slf4j', module: 'slf4j-reload4j'


@danielcweeks had a comment about this in #18660 (comment)

logback-classic and slf4j-reload4j are SLF4J 1.x bindings, which slf4j-api 2.0 ignores. Both are only in hive, so this excludes them and drops the Logback entry from hive/LICENSE. See the diff in hive/runtime-deps.txt in this PR.

reverted this to keep the PR focused on LICENSE/NOTICE. will exclude them from hive in a follow up

…ibution" Exclusion will be done in a follow up. This reverts commit 7a6a454. Generated-by: GitHub Copilot (Claude Opus 5.5) Co-authored-by: Copilot App <[email protected]>

github-actions Bot added build KAFKACONNECT labels Oct 27, 2026

kevinjqliu mentioned this pull request Oct 27, 2026

Kafka Connect: Publish the runtime distribution zip in the release process #15212

Open

jbonofre reviewed Nov 9, 2026

View reviewed changes

kevinjqliu mentioned this pull request Nov 12, 2026

Kafka Connect: Track Hive distribution runtime dependencies #18660

Merged

kevinjqliu and others added 2 commits November 13, 2026 13:54

kevinjqliu force-pushed the kevinjqliu-kafka-connect-hive-license-notice branch from f9fd933 to f6e244f Compare November 13, 2026 21:55

kevinjqliu commented Nov 13, 2026

View reviewed changes

kevinjqliu requested review from Fokko, danielcweeks and jbonofre November 13, 2026 22:48

kevinjqliu marked this pull request as ready for review November 13, 2026 22:49

kevinjqliu commented Nov 13, 2026

View reviewed changes

Revert "Kafka Connect: Exclude SLF4J 1.x bindings from the Hive distr…

dee1746

…ibution" Exclusion will be done in a follow up. This reverts commit 7a6a454. Generated-by: GitHub Copilot (Claude Opus 5.5) Co-authored-by: Copilot App <[email protected]>

Navigation Menu

Repository navigation

Kafka Connect: Reconcile the Hive distribution LICENSE and NOTICE - #18663

Kafka Connect: Reconcile the Hive distribution LICENSE and NOTICE#18663
kevinjqliu wants to merge 4 commits into
apache:mainfrom
kevinjqliu:kevinjqliu-kafka-connect-hive-license-notice

kevinjqliu commented Oct 27, 2026 •

edited

Loading

Uh oh!

jbonofre Nov 9, 2026

Uh oh!

kevinjqliu Nov 13, 2026

Uh oh!

jbonofre Nov 9, 2026

Uh oh!

kevinjqliu Nov 13, 2026

Uh oh!

Uh oh!

kevinjqliu Nov 13, 2026

Uh oh!

kevinjqliu Nov 13, 2026

Uh oh!

kevinjqliu Nov 13, 2026

Uh oh!

kevinjqliu Nov 13, 2026

Uh oh!

kevinjqliu Nov 13, 2026

Uh oh!

Uh oh!

kevinjqliu Nov 13, 2026 •

edited

Loading

Uh oh!

kevinjqliu Nov 13, 2026

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

	License: CDDL 1.0 - https://spdx.org/licenses/CDDL-1.0.html
	License: CDDL 1.1 - https://glassfish.java.net/public/CDDL+GPL_1_1.html


		--------------------------------------------------------------------------------

		This product bundles reload4j with the following in its NOTICE file:

	javax.xml.bind:jaxb-api:2.2
	javax.xml.stream:stax-api:1.0-2
	net.java.dev.jna:jna-platform:5.17

		// keep the hive distribution a superset of main by resolving shared jars to the same versions
		shouldResolveConsistentlyWith(runtimeClasspath)

Conversation

kevinjqliu commented Oct 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

How each added dep is attributed

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

kevinjqliu Nov 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

kevinjqliu commented Oct 27, 2026 •

edited

Loading

kevinjqliu Nov 13, 2026 •

edited

Loading