https://issues.apache.org/jira/browse/ARROW-13804

Thanks @zeroshade it looks like there might have been some large changes outside of flatbuf code unrelated to MonthDayNanoIntervals (it appears that the are formatting?) Would it pay to submit a PR doing the formatting and flatbuf update separately from the actual new logic?

@emkornfield So the formatting changes happened due to the length of the name MonthDayNanoInterval since I auto run go fmt on the files which will line things up. Since the name is so long, it ends up changing the formatting a bit in order to maintain the alignment of the code, that's all.

If you think that doing the flatbuf update separately from the actual new logic would be beneficial I can do that, mostly I didn't think anyone would be directly reviewing the generated flatbuf code so didn't consider it necessary to break it out as a separate change.

That said, the changes to the enum values to line up with the C++ type ids could be broken out as a separate change to be done first, before the flatbuf and MonthDayNanoInterval logic, which I can do.

the failure in the Archery & Crossbow test is unrelated to my changes as far as I can tell.

@emkornfield I broke the flatbuf update and the type id enum changes out to #11389 after that gets merged, i'll update this PR accordingly

I was able to narrow this down to being related to the application metadata. Unlike the C++ flight integration client, the Java one does not validate the metadata in the response to the DoGet, only in the DoPut, if I modify the Go flight integration server to not send the app metadata field with the DoGet responses, then it succeeds without issue. But when it does send the metadata, it fails claiming a memory leak of the exact number of bytes that were sent via app metadata (1 byte per message).

I don't know why it doesn't have this memory leak when consuming from the C++ flight integration server which appears to be sending the metadata in the same way unless i'm missing something. I'm not particularly well versed with java or the java implementation so any assistance at all in tracking down what's going on and whether it's a Go bug or a Java bug would be extremely helpful. Thanks much!

I figured out the cause of the issue, when using WriteWithMetadata from the Go flight library, if the schema message hadn't yet been sent yet it includes the metadata in the schema message in addition to the first record batch message. In the Java, it allocates the memory for the application metadata for the Schema message, but never cleans it up causing the allocator to see it as a memory leak and error out. So adding a check on schema messages to close out the app metadata solves that issue, which I've done here in order to fix the issue with Java as a flight consumer.

Success! Manually adding a step in the archery.yml to install pygit2 from a binary wheel fixes the missing header issue. Always nice to see all greens :)

@lidavidm I was told that I should tag you to check out the small java change here. Could you please take a look and just make sure i'm not doing anything terrible / that should be done a better way? Thanks!

Benchmark runs are scheduled for baseline = e379ee1 and contender = 12aaf2c. 12aaf2c is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished ⬇️25.0% ⬆️0.0%] ec2-t3-xlarge-us-east-2
[Finished ⬇️0.0% ⬆️0.0%] ursa-i9-9960x
[Finished ⬇️0.22% ⬆️0.0%] ursa-thinkcentre-m75q
Supported benchmarks:
ursa-i9-9960x: langs = Python, R, JavaScript
ursa-thinkcentre-m75q: langs = C++, Java
ec2-t3-xlarge-us-east-2: cloud = True