Run llm stuck while use jetson thor

[17816864266]

i run gpt-oss:20b step by step in Ollama - NVIDIA Jetson AI Lab

but while talk to llm, always stuck or shows unexpect EOF，how to solve it？in the same time，dmesg shows NVRM: nvAssertFailed: Assertion failed: 0 @ g_kern_bus_nvoc.h:2706

[CharlieHsu]

First, ensure your Jetson is operating at its maximum performance profile. This can prevent bottlenecks related to CPU, GPU, and memory clock speeds.

Bash

sudo jetson_clocks

Second, there appears to be a known issue on some Jetson platforms where Ollama may not release memory correctly, leading to instability. To mitigate this, you can manually clear the system’s page cache, dentries, and inodes right before running the model. This command will free up cached memory.

Bash

sudo sh -c 'echo 3 > /proc/sys/vm/drop_caches'

Recommended Workflow:

1. Run sudo jetson_clocks once after booting up.

2. Right before you start the model with ollama run ..., execute the drop_caches command to ensure maximum available memory.

Hope this helps solve the issue!

[17816864266]

thx, i ll try

[AastaLLL]

Hi,

Thanks for reporting this issue.

We are checking it internally.
Will provide more information with you later.

[AastaLLL]

Hi,

Thanks for your patience.
We did some test but not able to reproduce this issue in our environment.

Our Thor is set up with JetPack 7.0 + this ollama container.
The test of gpt-oss:20b runs smoothly and normally.

$ sudo docker run --ipc=host --net=host --gpus=all --runtime=nvidia --privileged -it --rm -u 0:0 --name=ollama ghcr.io/nvidia-ai-iot/ollama:r38.2.arm64-sbsa-cu130-24.04

root@tegra-ubuntu:/# ollama run gpt-oss:20b
pulling manifest
...
success
>>> test
Thinking...
The user just sent "test". They might be testing. Likely they want a response confirming receipt. We should respond politely and ask if they need help.
...done thinking.

Got it! How can I assist you today?

>>> Which number is larger, 9.11 or 9.8?
Thinking...
We need to answer which is larger: 9.11 or 9.8. Clearly 9.8 is larger.
...done thinking.

9.8 is larger.

>>> hello
Thinking...
User says "hello". They didn't ask a question. We should respond in a friendly manner. Possibly ask how to help.
...done thinking.

Hi there! How can I help you today?

>>> Send a message (/? for help)

In case there are some issues when downloading the model, could you re-run the model to see if it can work?

Thanks.

[alexjao]

I have found the following would also work

sudo jetson_clocks
sudo sh -c ‘echo 3 > /proc/sys/vm/drop_caches’
sudo systemctl restart ollama
free -h

[AastaLLL]

Hi,

Thanks for your information.

Just for your reference.
We also have cache cleaner script in the VSS release:

deploy/scripts/sys_cache_cleaner.sh

Thanks.