Skip to content

About

A web interface for the Moondream3 vision-language model featuring image captioning, visual question answering, object detection, and object pointing.

Topics

Resources

Stars

6 stars

Watchers

0 watching

Forks

Latest commit

 

History

27 Commits

Folders and files

Repository files navigation

🌙 Moondream3 Gradio UI

A web interface for the Moondream3 vision-language model (9B total params, 2B active MoE) featuring image captioning, visual question answering, object detection, and object pointing.

Features

  • 📝 Image Captioning: Generate short, normal, or long captions with optional streaming
  • ❓ Visual Q&A: Ask questions about images with reasoning mode — also supports text-only queries
  • 🔍 Object Detection: Detect and localize specific objects with bounding boxes
  • 👆 Object Pointing: Point to specific objects in images
  • ⚙️ Advanced Settings: Temperature, max tokens, streaming, max objects controls

Model

This application uses the official moondream/moondream3-preview model from Hugging Face. The model will be automatically downloaded on first run.

Requirements

  • Python 3.10+
  • NVIDIA GPU with CUDA recommended (approx. 19 GB VRAM for full performance)
  • Also works on CPU/MPS, but slower

Installation

Pinokio (recommended)

This repository is a Pinokio script. Add it as a custom script in Pinokio and click Install — Pinokio creates the virtual environment at app/env, installs app/requirements.txt, and selects the right PyTorch build for your platform and GPU. Then click Start and Open Web UI.

Other menu entries: Update (git pull + reinstall dependencies), Save Disk Space (deduplicate library files), and Reset (delete the environment).

Manual installation

1. Clone Repository

git clone https://github.com/PierrunoYT/MoonDream-3-Pinokio.git
cd MoonDream-3-Pinokio

2. Create Virtual Environment

python -m venv .venv

# Windows (CMD)
.venv\Scripts\activate

# Windows (PowerShell)
.venv\Scripts\Activate.ps1

# Linux/macOS
source .venv/bin/activate

3. Install PyTorch

Visit pytorch.org/get-started or use:

# CUDA 12.8
pip install torch --index-url https://download.pytorch.org/whl/cu128

# CPU only
pip install torch --index-url https://download.pytorch.org/whl/cpu

4. Install Dependencies

pip install -r app/requirements.txt

Usage

Starting the Application

python app/app.py

Or from the app folder: cd app then python app.py.

The application will start at http://127.0.0.1:7860.

Steps to Use

  1. Open the URL in your browser
  2. Click "Load Model" to load Moondream3 (downloads ~18GB on first run)
  3. Select a tab for the desired function:
    • Image Captioning: Upload image, choose length, click "Generate Caption"
    • Visual Q&A: Upload image (optional), enter question, toggle reasoning, click "Ask Question"
    • Object Detection: Upload image, enter object type, click "Detect"
    • Object Pointing: Upload image, enter object type, click "Point"

Advanced Settings

  • Temperature: Controls randomness (0 = model default)
  • Max Tokens: Limit response length (0 = model default)
  • Stream Output: See responses generated token-by-token
  • Reasoning: Enable/disable chain-of-thought reasoning in Q&A
  • Max Objects: Limit detected objects count in detection

Environment Variables

Variable Default Description
MOONDREAM_COMPILE 0 Set to 1 to enable torch.compile optimization. Requires a working compiler toolchain and adds a long first-run warm-up.
MOONDREAM_REVISION pinned commit Hugging Face revision (commit, branch, or tag) of moondream/moondream3-preview to load. The model ships custom code that runs locally, so the app pins a known commit by default; set to main to follow upstream.
GRADIO_SERVER_NAME 127.0.0.1 Address to bind to. Use 0.0.0.0 to expose on the local network.
GRADIO_SERVER_PORT unset Pin a port. When unset, Gradio starts at 7860 and moves to the next free port if it is taken.

API

Once the server is running at http://127.0.0.1:7860, you can access the Gradio API programmatically.

JavaScript

import { Client } from "@gradio/client";
import fs from "node:fs";

const client = await Client.connect("http://127.0.0.1:7860");

// The model must be loaded once before any other endpoint is called.
await client.predict("/load_model", []);

// Image captioning: [image, length, temperature, max_tokens, stream]
const image = new Blob([fs.readFileSync("image.jpg")], { type: "image/jpeg" });
const result = await client.predict("/caption", [image, "normal", 0, 0, false]);
console.log(result.data);

Python

from gradio_client import Client, handle_file

client = Client("http://127.0.0.1:7860")

# The model must be loaded once before any other endpoint is called.
print(client.predict(api_name="/load_model"))

# Image captioning
result = client.predict(
    handle_file("image.jpg"),  # image
    "normal",                  # length: short | normal | long
    0,                         # temperature (0 = model default)
    0,                         # max_tokens (0 = model default)
    False,                     # stream
    api_name="/caption",
)
print(result)

Available endpoints: /load_model, /caption, /query, /detect, /point.

cURL

# Check available API endpoints
curl http://127.0.0.1:7860/info

Full interactive API docs are available at http://127.0.0.1:7860/?view=api once the app is running.

License

See the Moondream3 Model Card for license information.


Powered by Moondream3 & Gradio

About

A web interface for the Moondream3 vision-language model featuring image captioning, visual question answering, object detection, and object pointing.

Topics

Resources

Stars

6 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages