ArrayIndexOutOfBoundsException in BufferedSocketInputStream.java:65 #66
Description
Hi @osheroff again
We got another exception in mysql-binlog-connector-java while using Debezium recently😅 This is the full error log we get.
org.apache.kafka.connect.errors.ConnectException: An exception occurred in the change event producer. This connector will be stopped.
at io.debezium.pipeline.ErrorHandler.setProducerThrowable(ErrorHandler.java:42)
at io.debezium.connector.mysql.MySqlStreamingChangeEventSource$ReaderThreadLifecycleListener.onEventDeserializationFailure(MySqlStreamingChangeEventSource.java:1193)
at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:958)
at com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:606)
at com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:850)
at java.lang.Thread.run(Thread.java:748)
Caused by: io.debezium.DebeziumException
at io.debezium.connector.mysql.MySqlStreamingChangeEventSource.wrap(MySqlStreamingChangeEventSource.java:1146)
... 5 more
Caused by: java.lang.ArrayIndexOutOfBoundsException
at com.github.shyiko.mysql.binlog.io.BufferedSocketInputStream.read(BufferedSocketInputStream.java:65)
at com.github.shyiko.mysql.binlog.io.ByteArrayInputStream.readWithinBlockBoundaries(ByteArrayInputStream.java:262)
at com.github.shyiko.mysql.binlog.io.ByteArrayInputStream.read(ByteArrayInputStream.java:241)
at java.io.InputStream.skip(InputStream.java:224)
at com.github.shyiko.mysql.binlog.io.ByteArrayInputStream.skipToTheEndOfTheBlock(ByteArrayInputStream.java:280)
at com.github.shyiko.mysql.binlog.event.deserialization.EventDeserializer.deserializeEventData(EventDeserializer.java:305)
at com.github.shyiko.mysql.binlog.event.deserialization.EventDeserializer.nextEvent(EventDeserializer.java:232)
at io.debezium.connector.mysql.MySqlStreamingChangeEventSource$1.nextEvent(MySqlStreamingChangeEventSource.java:233)
at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:945)
... 3 more
From the error log, it shows java.lang.ArrayIndexOutOfBoundsException in code
After digging more about the code, I believe the issue is caused by the value of limit in line
According to the doc of InputStream, read(...) method will return -1 if it reaches the end of stream, so the limit could be -1 here. Then bytesRemainingInBuffer will be a negative value which causes java.lang.ArrayIndexOutOfBoundsException in System.arraycopy method.
I wrote a test case for reproduce this error in BufferedSocketInputStreamTest
@Test
public void testReadToTheEnd() throws Exception {
BufferedSocketInputStream in = new BufferedSocketInputStream(new ByteArrayInputStream(new byte[]{
0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16}), 20);
assertEquals(in.read(), 0);
assertEquals(in.read(), 1);
byte[] buf = new byte[6];
assertEquals(in.read(buf, 0, buf.length), 6);
assertEquals(in.read(buf, 0, buf.length), 6);
assertEquals(in.read(buf, 0, buf.length), 3); // only 3 bytes left in stream
assertEquals(in.read(buf, 0, buf.length), -1); // got the end of stream, so return -1
}This test case will throw the same java.lang.ArrayIndexOutOfBoundsException as I mentioned above, and I have a patch for fixing this issue, could you help to review it? Thank you a lot.