I’m trying to call a video encoding function that I defined, but I can only invoke it up to 7 times—on the 8th call, it throws an error. I understand there’s a limit on the total number of encoders that can be created, but shouldn’t the encoder instance be destroyed once the function finishes executing? Version 1.0 does not have a problem.
Below is a minimal example that reproduces the issue:
import PyNvVideoCodec as nvc
def encode():
encoder = nvc.CreateEncoder(
width=1024,
height=1024,
fmt="NV12",
usecpuinputbuffer=True,
)
if __name__ == "__main__":
for i in range(8):
print(i)
encode()
The error message:
Traceback (most recent call last):
line 16, in <module>
enc = encode()
File "/home/jiang/Code/pynvvideocodec-test/test_encoder_num.py", line 4, in encode
encoder = nvc.CreateEncoder(
File "/home/jiang/anaconda3/envs/codec/lib/python3.10/site-packages/PyNvVideoCodec/__init__.py", line 237, in CreateEncoder
return PyNvEncoder(width, height, fmt, cudacontext, cudastream, usecpuinputbuffer, optional_args)
_PyNvVideoCodec.PyNvVCException: NvEncoder :
Error code : 21
Error Type : m_nvenc.nvEncOpenEncodeSessionEx(&encodeSessionExParams, &hEncoder) returned error 21
at /project/src/VideoCodecSDKUtils/helper_classes/NvCodec/NvEncoder/NvEncoder_130.cpp:76
I have the same issue. Calling del encoder afterwards doesn’t help either. I added prints in C++ wrapper code where it calls destructor and DestroyEncoder and it does run. I can’t find the code on how to properly release and reuse the encoder.
In NvEncoder_130.cpp line 543 where it has m_nvenc.nvEncDestroyEncoder(m_hEncoder);I added a print right above it and it prints. It seems that this is the place it should actually go and destroy the encoder session before reuse. So there isn’t anything I can actually go deeper on.
In my case I just need to reset the encoding for a new file output. I will try implementing functions in PyNvVideoCodec for that, as I don’t see any examples on how to do it. All examples use one encoder just for one file.
NOTE: To get it to even create an encoder on my end, I’ve had to patch the library like this:
Does it work fine for you without this patch @Dajuzi@Dalus ?
Another find:
With PyNvVideoCodec 2.0.0 I can only create up to 2 encoder sessions (the 3rd call to CreateEncoder ends up with a NV_ENC_ERR_OUT_OF_MEMORY) but with PyNvVideoCodec 1.0.2 I can create up to 5 encoder sessions (the 6th call fails, which I think is expected with my driver version)
EDIT: What if I told you this thread’s issue is related to the same method as mine?
The encoder session used to query capabilities PyNvEncoderCaps is being leaked there. It never gets freed.
Adding this line m_nvenc.nvEncDestroyEncoder(hEncoder); after the faulty (at least on my end) loop over the caps and before the call to cuCtxDestroy fixes everything. (NOTE: It will still leak if nvEncGetEncodeCaps fails & the exception is caught, along with the CUDA context & the dynamic library loaded)
Could someone provide an answer as to why NVIDIA decided to take this project off of GitHub? I think all these issues could’ve been solved faster if I, or others, could’ve created merge requests on this project.
I couldn’t find the error, so instead I just used multiprocessing to spawn encoder in a different process and then terminate the process. This of course complicated things, but it does also give me parallelization speedup, so that’s nice. Though I would like to fix this bug as then I could use threads in instead of processes and spawning a process takes 5s in my project.
I will try your fixes. And why this isn’t on github I don’t know. Doing anything with source is a pain right now, I agree with that. Would really love if anyone from nvidia would even read these threads though.
edit: Another issue I am dealing with right now is that encoder.Encode can ignore new data and duplicate frames with old data. Currently debugging if the issue is on my end, but everything I checked seems to indicate some issues with nvenc. Like I need to maybe call some synchronization step after each encode, though that doesn’t make much sense as it should wait until finish anyway.
edit2: It worked! Missing m_nvenc.nvEncDestroyEncoder(hEncoder); was the issue. Now I can use threads instead. Also was thinking about just reusing the encoder itself, though that failed for me as the stream got corrupted for the next videos (first one was correct). I don’t know how to do it properly. I tried setting first frame with FORCEIDR, then calling reconfigure in case that resets something, then I tried in C++ to reset frameCnt, but nothing worked.
I can’t find any changelogs. And the source is only for 2.0.0 in NGC.
But your fixes worked and allowed me to use regular threads, thus removing a long startup time. Thank you!
Edit: I couldn’t install 2.0.1 package:
error: Failed to install: pynvvideocodec-2.0.1-cp312-cp312-manylinux_2_28_x86_64.whl (pynvvideocodec==2.0.1)
Caused by: RECORD file doesn't match wheel contents: Could not find entry for PyNvVideoCodec-2.0.1.data/data/external/ffmpeg/src/ffmpeg-7.0.2.tar.xz (/worker/.venv/lib/python3.12/site-packages/PyNvVideoCodec-2.0.1.data/data/external/ffmpeg/src/ffmpeg-7.0.2.tar.xz)