I am interfacing with a Nvidia DeepStream application and retrieving the pointer as described in: API

Using cudaMemcpy2D (API) I am able to copy the buffer to host device without problems.

unsafe static extern int cudaMemcpy2D(IntPtr dst, uint dpitch, IntPtr src, uint spitch, uint width, uint height, cudaMemcpyKind kind);

Example:

using var buffer = info.Buffer;

buffer.Map(out MapInfo map, MapFlags.Read);

NvBufSurface nvBufSurface = Marshal.PtrToStructure<NvBufSurface>(map.DataPtr);
IntPtr cudaPtr = (IntPtr)nvBufSurface.surfaceList[0].dataPtr;

byte[] buff = new byte[nvBufSurface.surfaceList[0].width * nvBufSurface.surfaceList[0].height * *nvBufSurface.surfaceList[0].planeParams.bytesPerPix];
fixed (byte* dst = &buff[0])
{
    int ret = cudaMemcpy2D((IntPtr)dst, nvBufSurface.surfaceList[0].width, cudaPtr, nvBufSurface.surfaceList[0].pitch, nvBufSurface.surfaceList[0].width, nvBufSurface.surfaceList[0].height, cudaMemcpyKind.cudaMemcpyDeviceToHost);

}

buffer.Unmap(map);

I notice that OrtValue has a IntPtr option:
OrtValue ortValue = OrtValue.CreateTensorValueWithData(cudaMemoryInfo, TensorElementType.Float, new long[] { 1, 3, 640, 640 }, cudaPtr, nvBufSurface.surfaceList[0].width * nvBufSurface.surfaceList[0].height * *nvBufSurface.surfaceList[0].planeParams.bytesPerPix);

However I don't see any way of passing an OrtValue to the inference session Run() or bind it with IOBinding.

The Python API describes in the Docs (Scenario 2) binds the X_ortvalue with io_binding.bind_input(name='input', device_type=X_ortvalue.device_name(), device_id=0, element_type=np.float32, shape=X_ortvalue.shape(), buffer_ptr=X_ortvalue.data_ptr()) however I don't see a similar API in C#?

Thanks,

Question: Inference from CUDA allocated memory #10180

Description

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions