Skip to content

[Improve][CDC] Optimize split state memory allocation in increment phase #6554

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
Hisoka-X merged 6 commits into from
Apr 2, 2024
Merged

[Improve][CDC] Optimize split state memory allocation in increment phase #6554

Hisoka-X merged 6 commits into from
Apr 2, 2024

Conversation

@hailin0
Copy link
Member

@hailin0 hailin0 commented Mar 21, 2024 •
edited
Loading

Purpose of this pull request

[CDC] Optimize memory allocation in increment phase

Does this PR introduce any user-facing change?

No

How was this patch tested?

seatunnel/seatunnel-e2e/seatunnel-connector-v2-e2e/connector-cdc-mysql-e2e/src/test/java/org/apache/seatunnel/connectors/seatunnel/cdc/mysql/MysqlCDCIT.java

Line 323 in 5f3c9c3

public void testMultiTableWithRestore(TestContainer container)

Added

Check list

@hailin0 hailin0 force-pushed the dev-optimize_increment_phase_memory branch 2 times, most recently from 5d65cdc to 0bb07db Compare March 21, 2024 07:11
@hailin0 hailin0 changed the title [Improve][CDC] Optimize memory allocation in increment phase [Improve][CDC] Optimize split state memory allocation in increment phase Mar 21, 2024
@hailin0 hailin0 force-pushed the dev-optimize_increment_phase_memory branch from 0bb07db to 1dabb41 Compare March 21, 2024 16:44
@hailin0
Copy link
Member Author

hailin0 commented Mar 21, 2024

image

@hailin0 hailin0 force-pushed the dev-optimize_increment_phase_memory branch from 1dabb41 to ec91c18 Compare March 22, 2024 10:01
@Carl-Zhou-CN
Copy link
Member

Carl-Zhou-CN commented Mar 23, 2024

good job,It seems that there is a lot of recycling inside

@hailin0 hailin0 marked this pull request as ready for review March 25, 2024 01:58
@hailin0 hailin0 force-pushed the dev-optimize_increment_phase_memory branch from ec91c18 to 4dc49dd Compare March 25, 2024 02:38
@hailin0 hailin0 added this to the 2.3.5 milestone Mar 25, 2024
@hailin0
Copy link
Member Author

hailin0 commented Mar 25, 2024 •
edited
Loading

image image

Carl-Zhou-CN
Carl-Zhou-CN previously approved these changes Mar 27, 2024
View reviewed changes
Copy link
Member

@Carl-Zhou-CN Carl-Zhou-CN left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Carl-Zhou-CN
Copy link
Member

Carl-Zhou-CN commented Mar 27, 2024

@hailin0 Please check ci

@hailin0
Copy link
Member Author

hailin0 commented Mar 27, 2024

image image

@Carl-Zhou-CN here

Carl-Zhou-CN
Carl-Zhou-CN approved these changes Apr 2, 2024
View reviewed changes
Copy link
Member

@Carl-Zhou-CN Carl-Zhou-CN left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@Hisoka-X Hisoka-X merged commit fe33422 into apache:dev Apr 2, 2024
chaorongzhi pushed a commit to chaorongzhi/seatunnel that referenced this pull request Aug 21, 2024
@hailin0
[Improve][CDC] Optimize split state memory allocation in increment ph…
14ebd20 
…ase (apache#6554)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Hisoka-X Hisoka-X Hisoka-X approved these changes

@Carl-Zhou-CN Carl-Zhou-CN Carl-Zhou-CN approved these changes

Assignees

No one assigned

Labels

approved cdc improve reviewed

Projects

None yet

Milestone

2.3.5

Development

Successfully merging this pull request may close these issues.

3 participants

@hailin0 @Carl-Zhou-CN @Hisoka-X