Background and motivation

The upcoming DllImport source generator will decouple the job of marshalling P/Invoke calls from the runtime, but in its current form it can't do much when native code is invoked via Marshal.GetDelegateForFunctionPointer. Function pointers can be used to call unmanaged code, but the built-in marshaller cannot be avoided if their arguments and return type are not blittable.

I propose a counterpart attribute to LibraryImportAttribute that instructs the generator to only marshal the parameters and the return type of the function, and call a user-specified function pointer instead of a P/Invoke. This attribute will provide the source-generated successor of Marshal.GetDelegateForFunctionPointer.

API Proposal

Updated to match the approved shape of LibraryImportAttribute.

namespace System.Runtime.InteropServices
{
    [AttributeUsage(AttributeTargets.Method, AllowMultiple = false, Inherited = false)]
    public sealed class NativeFunctionMarshalAttribute : Attribute
    {
        /// <summary>
        /// Constructor.
        /// </summary>
        public NativeFunctionMarshalAttribute();

        /// <summary>
        /// Indicates how to marshal string parameters to the method.
        /// </summary>
        /// <remarks>
        /// If this field is specified, <see cref="StringMarshallingCustomType" /> must not be specified.
        /// </remarks>
        public StringMarshalling StringMarshalling { get; set; }

        /// <summary>
        /// Indicates how to marshal string parameters to the method.
        /// </summary>
        /// <remarks>
        /// If this field is specified, <see cref="StringMarshalling" /> must not be specified.
        /// The type should be one that conforms to usage with the attributes:
        /// <see cref="System.Runtime.InteropServices.MarshalUsingAttribute"/>
        /// <see cref="System.Runtime.InteropServices.NativeMarshallingAttribute"/>
        /// </remarks>
        public Type? StringMarshallingCustomType { get; set; }

        /// <summary>
        /// Indicates whether the callee sents an error (SetLastError on Windows or errorno
        /// on other platforms) before returning from the attributed method.
        /// </summary>
        /// <see cref="System.Runtime.InteropServices.DllImportAttribute.SetLastError"/>
        public bool SetLastError { get; set; }
    }
}

The attribute resembles LibraryImportAttribute, with the difference that it lacks the constructor parameter for the library name and the EntryPoint property.

When it is applied to a partial method, the method's first parameter must be an IntPtr (otherwise it is an error) and is not passed to the native call; instead it contains the pointer to the unmanaged function; the generated method casts the IntPtr to the appropriate function pointer type and marshals and passes the other parameters to it.

In all cases that don't involve importing a DLL, the samantics of NativeFunctionMarshalAttribute are identical with LibraryImportAttribute and these two attributes will evolve in tandem. If LibraryImportAttribute gets a relevant new property or behavior, this attribute will get it as well.

API Usage

[NativeFunctionMarshal]
[UnmanagedCallConv(CallConvs = new[] {typeof(CallConvSuppressGCTransition)})]
public partial int QueryPerformanceCounter_wrapper(IntPtr nativeFunc, out long counter);

var kernel32 = NativeLibrary.Load("kernel32.dll");
var qpc_func = NativeLibrary.GetExport(kernel32, "QueryPerformanceCounter");

_ = QueryPerformanceCounter_wrapper(qpc_func, out long counter);

Console.WriteLine($"QPC returned {counter}");

Alternative Designs

• We could use a different attribute name.
• Instead of adding one function pointer argument to the method at the beginning, we could create source-generated specializations of Marshal.GetDelegateFromFunctionPointer<TDelegate>, for various TDelegates. While it would ease migration, it is less efficient and less flexible. And if somebody wants to create such delegates they can still be done with the proposed API.
• The first parameter could have been either of UIntPtr, nint, nuint and void* but there is no reason to be and it would be better to keep things simple. The functions in the NativeLibrary function return an IntPtr either way.

Risks

This attribute brings a conceptual change when dealing with P/Invokes; not all arguments on the managed function correspond to arguments on the native function. There might be some confusion, but this API is not intended for everyday use either way.