can you add a comment about the patch? has it been submitted upstream?

Added the comment.
Changes were not submitted upstream yet. The patch modifies decodecorpus utility from zstd library. Unmodified tool allows generating valid zstd frames with randomized contents and size. With our modifications it is possibile to generate only some parts of the zstd frame. In this case we modify decodecorpus so that it allows generating only frame headers.

any reason we need to use a real file, rather than handling in memory buffer returned by libstd?

This was not possible in original decodecorpus and we wanted to provide the minimal changes required for generating only frame headers.

Would it be a "big change" to patch decodecorpus further so that it exposes its intermediate functions (IIRC they are currently static)?

We could remove the main from the XLS build and directly call them and consume their outputs (rather than dealing with processes and files).

I believe this was discussed with you on one of the meetings. We settled on leaving decodecorpus as it is.

is this redundant to what we're doing in GenerateRandomFrameHeader or does it have more coverage?

GenerateFrameHeader() calls decodecorpus in order to generate frame header which is valid in case of zstd frame header specification. The contents and length of those frame headers are randomized but the header is always valid - it should always be correctly parsed both by libzstd and by our decoder (frame header parser).

GenerateRandomFrameHeader() generates completely random vector of bytes of randomly picked size from range 0 to arbitrary max size of 32 bytes. With this generator we are able to test more 'negative' test cases, meaning:

• Not enough data in buffer for finishing parsing
• Corrupted header - e.g. reserved bit in Frame_Header_Descriptor is set
• Handling of buffers larger than bare minimum for parsing frame header

Make sense, thanks for the explanation!

Would it makes sense to add more docstring to the tests and helpers so that it's more obvious to future reader?

Added more comments

I wonder if #476 would help doing this kind of things?

This kind of testing approach would be great if we didn't have decodecorpus which always generate valid zstd frames/frame headers. We could say that decodecorpus has embedded in itself the properties that describe valid frame header.
Without it we would have to manually specify those properties in form of multiple RC_ASSERT() statements in rapidcheck test cases in order to generate valid test data. I think that when it comes to 'positive' tests (comparing against valid frame headers) it is better to use decodecorpus.
However, I'm not sure how would it look like for 'negative' tests (generating potentially invalid frame header data and comparing parsing through libzstd against simulation of our decoder).
Frame Header parsing should fail basically only in 2 situations:

• there was not enough data to finish the parsing
• reserved bit in frame header descriptor was set

Currently we generate random test vectors in order to check if decoder fails when those situations occur.

maybe something like https://github.com/google/fuzztest could be useful; I'd be curious to know what @ericastor as a migrated a lot of "random generation tests" to fuzztest in 1cfc4cc.

I agree that using GoogleFuzzTest here would probably result in better coverage - though it might end up discovering some valid headers!

Thanks for the suggestion, I've replaced generating random frame headers with FuzzTest. It is true that it can discover valid headers. We can work on constraining the tested domain further to precisely test only invalid frame headers but I believe it is better to do a general check with valid and invalid headers.