llama-cpp-python CUDA Installation for ComfyUI Portable

This guide helps you install llama-cpp-python with GPU (CUDA) support for ComfyUI's Windows portable version.

---

1. Open ComfyUI's Command Prompt

• Go to C:\ComfyUI_windows_portable\python_embeded
• In the address bar, type cmd and press Enter.

---

2. Install Build Tools (If you haven't already)

• Install Visual Studio Build Tools:
  • Download Visual Studio Build Tools 2022 [https://visualstudio.microsoft.com/downloads/](https://visualstudio.microsoft.com/downloads/](https://learn.microsoft.com/en-us/visualstudio/releases/2022/release-history#release-dates-and-build-numbers) (under 17.4 "Build Tools").
  • Run installer, select "Desktop development with C++" workload.
• Install NVIDIA CUDA Toolkit:
  • Download your CUDA version (e.g., 12.6) from https://developer.nvidia.com/cuda-toolkit-archive.
  • Run installer, ensure "Visual Studio Integration" is selected.

Automatic Installation Script (Recommended)

• Make sure the installation script is saved here:
  .\ComfyUI\custom_nodes\ComfyUI-JoyCaption\install_llama_official.py

• In the command prompt (opened in step 1), run the script with:

  .\python_embeded\python.exe llama_cpp_install.py

The script will:

• Upgrade pip
• Clear pip cache
• Install minimal build dependencies (scikit-build-core, cmake)
• Detect GPU and build llama-cpp-python with CUDA support if available

🕒 This process may take 5–20+ minutes depending on your system.

** Restart ComfyUI**

• Close ComfyUI.
• Restart ComfyUI.
• Hard refresh your browser with Ctrl+F5 (Windows) or Cmd+Shift+R (macOS).

Manuelly Installation

---

1. Prepare Python Environment

• In the command prompt, run:

  .\python.exe -m pip install --upgrade pip
  .\python.exe -m pip cache purge
  .\python.exe -m pip install scikit-build-core cmake

---

2. Compile and Install llama-cpp-python

• In the command prompt, run:

  set CMAKE_ARGS="-DGGML_CUDA=on" && .\python.exe -m pip install llama-cpp-python --no-cache-dir && set CMAKE_ARGS=

  • Wait patiently. This takes time (5-20+ minutes).

---

3. Restart ComfyUI

• Close ComfyUI.
• Restart ComfyUI.
• Hard refresh your web browser (Ctrl+F5 or Cmd+Shift+R).

---