-
Notifications
You must be signed in to change notification settings - Fork 623
Compilation failure on MSVC when changing the default calling convention #403
Description
Hello,
I'm trying to use meshoptimizer in a Visual Studio project that overrides the default calling convention with __vectorcall (/Gv), and seeing compilation failures in src/meshoptimizer.h:
1>meshoptimizer.h(732,23): error C2440: 'initializing': cannot convert from 'overloaded-function' to 'void (__vectorcall *)(void *)'
1>meshoptimizer.h(732,23): error C2440: template <typename T> void (*meshopt_Allocator::StorageT<T>::deallocate)(void*) = operator delete;
This appears to be because the MSVC C++ libs explicitly mark operator new and operator delete as __cdecl, rather than implicitly picking up the calling convention from the compiler settings. Since your allocator/deallocate free-function pointers don't specify a calling convention, they get __vectorcall implicitly in this scenario, and thus become incompatible with operator new.
Explicitly changing the calling convention back to __cdecl for the translation unit allows it to compile OK, but then causes linker errors because the other translation units that include meshoptimizer.h are still set to __vectorcall. The only path I can see to using the library in its current form is to change the whole project back to __cdecl, but that comes at a runtime performance cost I'd like to avoid.
(I could also get around it by wrapping the library it in a separately-compiled library and forwarding all the calls through shims with explicit calling conventions, but that's clearly a big headache.)
Would you be amenable to adding a user-configurable #define for explicitly specifying the calling convention of alloc+dealloc function pointers? I'd be happy to prepare a PR for this myself.