[Feature][Connectors-v2]Paimon version upgrade to 1.1.1 #8074
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Hisoka-X
reviewed
Nov 18, 2024
hawk9821
force-pushed
the
paimon_version_upgrade
branch
2 times, most recently
from
0292734 to
853960f
Compare
Hisoka-X
reviewed
Nov 20, 2024
...ctor-paimon-e2e/src/test/java/org/apache/seatunnel/e2e/connector/paimon/PaimonSinkCDCIT.java
Outdated
Show resolved
Hide resolved
...nel-connector-v2-e2e/connector-paimon-e2e/src/test/resources/fake_cdc_sink_paimon_case9.conf
Show resolved
Hide resolved
dailai
reviewed
Nov 22, 2024
...ctor-paimon-e2e/src/test/java/org/apache/seatunnel/e2e/connector/paimon/PaimonSinkCDCIT.java
Outdated
Show resolved
Hide resolved
hawk9821
force-pushed
the
paimon_version_upgrade
branch
3 times, most recently
from
5fa7e4e to
690c856
Compare
Contributor
|
I think we should keep the default behavior of seatunnel when writing to paimon which is that we still used fixed buckets and the default number of buckets is 1, and we should also mark dynamic buckets as experimental properties. |
hawk9821
force-pushed
the
paimon_version_upgrade
branch
5 times, most recently
from
060fd30 to
9c5377f
Compare
hawk9821
changed the title
Hisoka-X
reviewed
Feb 13, 2025
...on/src/main/java/org/apache/seatunnel/connectors/seatunnel/paimon/catalog/PaimonCatalog.java
Outdated
Show resolved
Hide resolved
| .forEach(field -> builder.column(field.name(), field.type(), field.description())); | ||
| builder.options(schema.options()); | ||
| Map<String, String> options = new HashMap<>(schema.options()); | ||
| options.remove(CoreOptions.PATH.key()); |
Member
There was a problem hiding this comment.
Please add a comment of why remove this?
Contributor
Author
There was a problem hiding this comment.
The paimon interface has changed in the new version. it will throw exception when not remove
java.lang.UnsupportedOperationException: The current catalog FileSystemCatalog does not support specifying the table path when creating a table.
at org.apache.paimon.catalog.AbstractCatalog.validateCustomTablePath(AbstractCatalog.java:582)
at org.apache.paimon.catalog.AbstractCatalog.createTable(AbstractCatalog.java:285)
at org.apache.paimon.catalog.DelegateCatalog.createTable(DelegateCatalog.java:106)
| this.newTableWrite(); | ||
| BucketMode bucketMode = this.paimonFileStoretable.bucketMode(); | ||
| this.dynamicBucket = | ||
| BucketMode.DYNAMIC == bucketMode || BucketMode.GLOBAL_DYNAMIC == bucketMode; |
Contributor
Author
There was a problem hiding this comment.
The paimon interface has changed in the new version
public enum BucketMode {
HASH_FIXED,
HASH_DYNAMIC,
CROSS_PARTITION,
BUCKET_UNAWARE;
...
}
...nel-connector-v2-e2e/connector-paimon-e2e/src/test/resources/fake_cdc_sink_paimon_case9.conf
Outdated
Show resolved
Hide resolved
hawk9821
force-pushed
the
paimon_version_upgrade
branch
from
a6c0a80 to
c284d30
Compare
hawk9821
force-pushed
the
paimon_version_upgrade
branch
from
c284d30 to
466ae0a
Compare
3 tasks
hawk9821
force-pushed
the
paimon_version_upgrade
branch
from
466ae0a to
97dbe2a
Compare
hawk9821
changed the title
hawk9821
force-pushed
the
paimon_version_upgrade
branch
3 times, most recently
from
dbd51bd to
85a4f65
Compare
hawk9821
force-pushed
the
paimon_version_upgrade
branch
from
87644b3 to
ac8ae4f
Compare
hawk9821
force-pushed
the
paimon_version_upgrade
branch
from
ac8ae4f to
d32128d
Compare
Hisoka-X
approved these changes
Aug 1, 2025
corgy-w
approved these changes
Aug 1, 2025
4 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Purpose of this pull request
The current dependency is set to
0.7.0-incubating, which is an incubating release of Apache Paimon, It is necessary to upgrade to the stable version 1.1.1.Does this PR introduce any user-facing change?
How was this patch tested?
PaimonSinkDynamicBucketIT#testParallelismWrite
Check list
New License Guide
release-note.