If JNA is compiled with a different compiler than your target library, you may see a crash when returning a struct of size greater than 8 bytes. This is because GCC, mingw-gcc, and MSVC don't necessarily agree on how to handle such a scenario.
This turned up when building and testing with mingw32 (x86 target) calling a callback with a returned struct. There is some accommodation within libffi to handle GCC vs MSVC calling conventions, but the JNA code should really test all permutations to ensure it works properly:
Simple call to native function returning struct:
- Java -> JNA (mingw) -> native (MSVC)
- Java -> JNA (MSVC) -> native (mingw)
- Java -> JNA (MSVC) -> native (MSVC) (tested, OK)
- Java -> JNA (mingw) -> native (mingw) (tested, OK)
Call to native function returning struct returned by a provided callback. Note that the calling and callback handling glue (libffi) will always be compiled by the same compiler:
- Java -> JNA (mingw) -> native (mingw) -> JNA (callback mingw) (tested, fails)
- Java -> JNA (MSVC) -> native (MSVC) -> JNA (callback MSVC) (tested, OK)
- Java -> JNA (MSVC) -> native (mingw) -> JNA (callback MSVC)
- Java -> JNA (mingw) -> native (MSVC) -> JNA (callback mingw)
If JNA is compiled with a different compiler than your target library, you may see a crash when returning a
structof size greater than 8 bytes. This is because GCC, mingw-gcc, and MSVC don't necessarily agree on how to handle such a scenario.This turned up when building and testing with mingw32 (x86 target) calling a callback with a returned
struct. There is some accommodation within libffi to handle GCC vs MSVC calling conventions, but the JNA code should really test all permutations to ensure it works properly:Simple call to native function returning struct:
Call to native function returning struct returned by a provided callback. Note that the calling and callback handling glue (libffi) will always be compiled by the same compiler: