-
-
Notifications
You must be signed in to change notification settings - Fork 180
Unable to create a context #283
Description
Issue description
Create a context fail on Linux / AMD64 with CUDA
Expected Behavior
The following code returns a context:
const context = await model.createContext();
Actual Behavior
The following works fine on my Mac M1 environment but fails every time on linux/amd64 with CUDA. It returns the following error: Context is disposed.
I've tried adding:
console.log(this._ctx.getContextSize()
right after this line:
https://github.com/withcatai/node-llama-cpp/blob/beta/src/evaluator/LlamaContext/LlamaContext.ts#L82
And even there, I get the same error.
It works fine on 3.0.0-beta.32.
Steps to reproduce
On linux/amd64 with CUDA, run the following:
const context = await model.createContext();
My Environment
| Dependency | Version |
|---|---|
| Operating System | Linux |
| CPU | Intel Xeon |
| Node.js version | 20.11.0 |
| Typescript version | 5.5.3 |
node-llama-cpp version |
3.0.0-beta.40 |
Additional Context
No response
Relevant Features Used
- Metal support
- CUDA support
- Grammar
Are you willing to resolve this issue by submitting a Pull Request?
Yes, I have the time, but I don't know how to start. I would need guidance.