Issue description

In version 2.8.3 I am receiving "could not find a KV slot for the batch (try reducing the size of the batch or increase the context)" error when performing a bunch of zero-shot prompts in a loop with the same context.

Expected Behavior

To be able to perform zero-shot prompts in a loop without failure.

Actual Behavior

I'm receiving the following error after a few iterations regardless of the number of layers or if I create multiple contexts.

Steps to reproduce

1. follow instructions on my OSS project for setting up genai-gamelist
2. update package.json to use 2.8.3 from 3.x beta for node-llama-cpp
3. run the script

My Environment

Additional Context

With only upgrading to the 3.x beta, I am now unable to reproduce the issue with the same script.

Relevant Features Used

• Metal support
• CUDA support
• Grammar

Are you willing to resolve this issue by submitting a Pull Request?

Yes, I have the time, but I don't know how to start. I would need guidance.