My company (Neo4j) recently tried updating from Netty 4.2.12.Final to Netty 4.2.13.Final and one of our network protocol tests broke reproducibly. Upon further tracing it appears that an unfortunate combination of internal buffer offsets leads to the CompositeByteBuf.findComponentForRead returning the wrong Component buffer to read from and then corrupted data is read into the application protocol. This is similar to Netty Issue: GZIP trailer corruption with large payloads after upgrading from 4.2.9 to 4.2.11 #16656.
I have attached a minimal reproduction of the problem, which shows a simple test doing reading and writing using a CompositeByteBuf that works. Then a failing version that differs only in the consumed buffer offsets passed into the addFlattenedComponent call, but bypasses the cache state reset fixes in Fix component search fast path #16548
I also note that there are other situations where the resets of lastAccessedIndex might not be correct, because even when the lastAccessed doesn't need to be set to null the components array may be shifted up, or down by buffer inserts and deletes requiring corrections to lastAccessedIndex.
FastPathBug.zip
We are very keen to to be able to upgrade Netty soon as we want to address the CVE's included in the latest release, but are hesitant as we don't know how often we will encounter protocol errors in our production environment. I hope this repro will help towards a swift resolution of the problems.
Regards,
Matthew Nesbit
My company (Neo4j) recently tried updating from
Netty 4.2.12.FinaltoNetty 4.2.13.Finaland one of our network protocol tests broke reproducibly. Upon further tracing it appears that an unfortunate combination of internal buffer offsets leads to theCompositeByteBuf.findComponentForReadreturning the wrongComponentbuffer to read from and then corrupted data is read into the application protocol. This is similar to Netty Issue: GZIP trailer corruption with large payloads after upgrading from 4.2.9 to 4.2.11 #16656.I have attached a minimal reproduction of the problem, which shows a simple test doing reading and writing using a
CompositeByteBufthat works. Then a failing version that differs only in the consumed buffer offsets passed into theaddFlattenedComponentcall, but bypasses the cache state reset fixes in Fix component search fast path #16548I also note that there are other situations where the resets of
lastAccessedIndexmight not be correct, because even when thelastAccesseddoesn't need to be set to null the components array may be shifted up, or down by buffer inserts and deletes requiring corrections tolastAccessedIndex.FastPathBug.zip
We are very keen to to be able to upgrade Netty soon as we want to address the CVE's included in the latest release, but are hesitant as we don't know how often we will encounter protocol errors in our production environment. I hope this repro will help towards a swift resolution of the problems.
Regards,
Matthew Nesbit