Skip to content

[Feature][connector][kafka] Support read debezium format message from kafka #3981

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
TaoZex wants to merge 49 commits into from
Closed

[Feature][connector][kafka] Support read debezium format message from kafka #3981

TaoZex wants to merge 49 commits into from

Conversation

@TaoZex
Copy link
Contributor

@TaoZex TaoZex commented Jan 19, 2023 •
edited
Loading

Purpose of this pull request

#3743
Support read debezium format message from kafka

Check list

@TaoZex
Copy link
Contributor Author

TaoZex commented Jan 19, 2023

I will add e2e test for this feature.

@hailin0
Copy link
Member

hailin0 commented Jan 19, 2023

I will add e2e test for this feature.

reference
https://github.com/apache/incubator-seatunnel/blob/dev/seatunnel-e2e/seatunnel-connector-v2-e2e/connector-cdc-sqlserver-e2e/src/test/java/org/apache/seatunnel/e2e/connector/cdc/sqlserver/SqlServerCDCIT.java

@hailin0
Copy link
Member

hailin0 commented Jan 19, 2023 •
edited
Loading

I will add e2e test for this feature.

reference https://github.com/apache/incubator-seatunnel/blob/dev/seatunnel-e2e/seatunnel-connector-v2-e2e/connector-cdc-sqlserver-e2e/src/test/java/org/apache/seatunnel/e2e/connector/cdc/sqlserver/SqlServerCDCIT.java

https://debezium.io/documentation/reference/2.1/tutorial.html#starting-services

@2000liux
Copy link

2000liux commented Jan 19, 2023

nice。

TyrantLucifer
TyrantLucifer reviewed Jan 23, 2023
View reviewed changes
Copy link
Member

@TyrantLucifer TyrantLucifer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you please add e2e test case about this pull request?

@hailin0
Copy link
Member

hailin0 commented Jan 30, 2023

Could you please add e2e test case about this pull request?

@TaoZex
reference check cdc sink output

https://github.com/apache/incubator-seatunnel/blob/dev/seatunnel-e2e/seatunnel-connector-v2-e2e/connector-jdbc-e2e/src/test/java/org/apache/seatunnel/connectors/seatunnel/jdbc/JdbcSinkCDCChangelogIT.java#L87

@hailin0
Copy link
Member

hailin0 commented Jan 30, 2023

reference this pr start mysql docker & kafka docker & debezium docker

#3950
https://debezium.io/documentation/reference/2.1/tutorial.html#starting-services

@hailin0
Copy link
Member

hailin0 commented Jan 30, 2023 •
edited
Loading

add docs into docs/en/connector-v2/formats/debezium-json.md and link to kafka source docs

reference
https://nightlies.apache.org/flink/flink-docs-release-1.16/docs/connectors/table/formats/debezium/

@TaoZex TaoZex closed this Feb 6, 2023
@TaoZex TaoZex reopened this Feb 6, 2023
hailin0
hailin0 reviewed Feb 13, 2023
View reviewed changes
Copy link
Member

@hailin0 hailin0 left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add docs & update release-note.md

@hailin0
Copy link
Member

hailin0 commented Apr 8, 2023

please rebase dev branch

@TaoZex
Copy link
Contributor Author

TaoZex commented Apr 9, 2023

Wait for #4524 merge.

@TyrantLucifer TyrantLucifer mentioned this pull request Apr 11, 2023
Merged
5 tasks
@TaoZex TaoZex closed this Apr 19, 2023
@TaoZex TaoZex reopened this Apr 19, 2023
@EricJoy2048
Copy link
Member

EricJoy2048 commented Apr 28, 2023

Can you rebase dev branch and push again?

hailin0
hailin0 reviewed May 12, 2023
View reviewed changes
Copy link
Member

@hailin0 hailin0 left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

check ci

https://github.com/apache/incubator-seatunnel/actions/runs/4959563825/jobs/8936251986?pr=3981

@hailin0
Copy link
Member

hailin0 commented May 31, 2023

image

@hailin0
Copy link
Member

hailin0 commented May 31, 2023

image

@zhilinli123
Copy link
Contributor

zhilinli123 commented Jun 13, 2023

check CI @TaoZex

@sunxiaojian sunxiaojian mentioned this pull request Jul 12, 2023
Merged
5 tasks
@hailin0
Copy link
Member

hailin0 commented Jul 25, 2023

Sorry, this PR has been merged

#5066

@TaoZex TaoZex closed this Jul 25, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@hailin0 hailin0 hailin0 left review comments

@TyrantLucifer TyrantLucifer TyrantLucifer left review comments

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@TaoZex @hailin0 @2000liux @EricJoy2048 @zhilinli123 @TyrantLucifer