Skip to content

Support read from specific MySQL Binlog position #39

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
shizhengchao wants to merge 1 commit into from
Closed

Support read from specific MySQL Binlog position #39

shizhengchao wants to merge 1 commit into from

Conversation

@shizhengchao
Copy link
Contributor

@shizhengchao shizhengchao commented Sep 19, 2020 •
edited
Loading

Fix #21，
Support read from specific MySQL Binlog position

Add optional flag source-offset-file and source-offset-pos to specify the source offsets in mysql-cdc.
source-offset-file means the binlog-file name, source-offset-pos means the position of the binlog.
two steps to get the binlog-file and position:

  1. open the mysql command-cli, ：
show binary logs; // list the binlog-file
  1. get the position：
show binlog events in 'mysql-bin.xxxxxx'; // get the position

usage in flink sql:

CREATE TABLE flink_cdc_test (
  order_id INT,
  order_date TIMESTAMP(0),
  customer_name STRING,
  price DECIMAL(10, 5),
  product_id INT,
  order_status BOOLEAN
) WITH (
  'connector' = 'mysql-cdc',
  'hostname' = 'localhost',
  'port' = '3306',
  'username' = 'root',
  'password' = 'root123',
  'database-name' = 'flink',
  'table-name' = 'flink_cdc_test',
  'source-offset-file' = 'mysql-bin.000021',
  'source-offset-pos' = '30292'
);

@shizhengchao shizhengchao mentioned this pull request Oct 10, 2020
Closed
@wuchong
Copy link
Member

wuchong commented Feb 26, 2021

Thanks for the contribution. I have merged this PR in befb4ad.

@wuchong wuchong closed this Feb 26, 2021
morozov pushed a commit to morozov/flink-cdc that referenced this pull request Dec 11, 2024
@maxf-redis @yuxiqian @ruanhang1993
DE-7931: Backport FLINK-36326 (apache#39)
f11c3d7 
[FLINK-36326][cdc-connector][mysql] Send BinlogSplitUpdateRequestEvent only once to fix auto scan newly-added table failure (apache#3613)

Signed-off-by: yuxiqian <[email protected]>
Co-authored-by: yuxiqian <[email protected]>
Co-authored-by: Hang Ruan <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Read from specific MySQL Binlog position

2 participants

@shizhengchao @wuchong