[mysql] Fix the binlogSplit state incompatibility when upgrading from 2.2.0 #1211
Conversation
ruanhang1993
left a comment
ruanhang1993
left a comment
There was a problem hiding this comment.
Thanks for your work, @lzshlzsh . LGTM.
| if (version >= 3) { | ||
| totalFinishedSplitSize = in.readInt(); | ||
| if (version > 3) { | ||
| if (version > 3 && in.available() > 0) { |
There was a problem hiding this comment.
I think this code should be like :
if (version > 3) {
// Descirbe the bug in PR#996.
if (in.available() <= 0 && version != 4) {
throw new IllegalStateException("Expected more info for the split.");
}
isSuspended = in.readBoolean();
}
IMO, change like this will not hide bugs in other future versions. But the code I provide doesn't look pretty. ;)
|
Hi @lzshlzsh, sorry for the delay of this PR. Could you please rebase it to latest |
|
This pull request has been automatically marked as stale because it has not had recent activity for 60 days. It will be closed in 30 days if no further activity occurs. |
|
This pull request has been automatically marked as stale because it has not had recent activity for 60 days. It will be closed in 30 days if no further activity occurs. |
|
This pull request has been automatically marked as stale because it has not had recent activity for 60 days. It will be closed in 30 days if no further activity occurs. |
|
This pull request has been closed because it has not had recent activity. You could reopen it if you try to continue your work, and anyone who are interested in it are encouraged to continue work on this pull request. |
3 participants
EOFException will happen when upgrading from mysql-cdc 2.2.0 and restoring from a savepoint. The exception stack is ac follows:
java.io.EOFException at org.apache.flink.core.memory.DataInputDeserializer.readBoolean(DataInputDeserializer.java:125) at com.ververica.cdc.connectors.mysql.source.split.MySqlSplitSerializer.deserializeSplit(MySqlSplitSerializer.java:165) at com.ververica.cdc.connectors.mysql.source.split.MySqlSplitSerializer.deserialize(MySqlSplitSerializer.java:124)The reason is that isSuspended flag is serialized for MySqlBinlogSplit in #996 which results in state incompatibility and we should increase serializer version normally, but we did not.
We can fix the problem simply by checking whether there is data left in buffer as isSuspended flag is the last field.