InferML GitHub Download

Open-source ML.
Ready to run.

Pick any model from the Hugging Face Hub. Download the app and run it. Detection, segmentation, VLMs, speech, diffusion, classification, depth, OCR, and more. Runs on CPU or GPU. Optional OpenAI-compatible API and MCP server. Nothing leaves your machine.

DETR SAM 2 SigLIP 2 InternVL Whisper FLUX SegFormer GPT-OSS Pixtral SDXL
Download InferML or
$ curl -fsSL https://inferml.vercel.app/install.sh | sh

Free and open source · all platforms · needs Python 3.10+

Not code-signed yet - Windows and macOS will warn on first launch. What to expect

InferML
5 local models
GPU: 0.01 / 8.00 GB | RAM: 9.08 / 15.71 GB | CPU: 24.00 %
Home
Installed only
Recents
detr-resnet-50
Object detection · 2m
sam2-hiera-large
Mask generation · 1h
Qwen2.5-VL-3B-Instruct
Image-text-to-text · 3h
segformer-b5-cityscapes
Segmentation · yesterday
TUESDAY, MAY 5
What would you like to run today?
Pick from your library, browse the Hub, or paste a model id.
⌕Search a task, family, or paste an HF id…
All VLM Text Segmentation SAM Detection Classify Diffusion Depth Docs / OCR ASR TTS
INSTALLED · 5
  • Florence-2-base
    microsoft · image-text-to-text
    462 MB
  • whisper-tiny
    openai · automatic-speech-recognition
    150 MB
  • detr-resnet-50
    facebook · object-detection
    165 MB
  • segformer-b0-ade-512
    nvidia · image-segmentation
    14 MB
  • Llama-3.2-1B-Instruct
    meta-llama · text-generation
    2.5 GB
SUGGESTED FOR YOU
  • InternVL2_5-1B
    OpenGVLab · image-text-to-text
    ↓ 1.9 GB
  • SmolLM3-3B
    HuggingFaceTB · text-generation
    ↓ 6.0 GB
  • sam2.1-hiera-tiny
    facebook · mask-generation
    ↓ 150 MB
  • stable-diffusion-xl-base-1.0
    stabilityai · text-to-image
    ↓ 6.9 GB
  • bark-small
    suno · text-to-speech
    ↓ 1.2 GB

See it in action.

Three different models, three different tasks, all running locally on the same machine.

Input photo
input a photo from your disk
Detection output with bounding boxes
output bounding boxes & labels
facebook/detr-resnet-50 · object-detection · 167 MB · runs on CPU or GPU
Input photo
input the same photo
Segmentation masks output
output per-pixel masks
facebook/sam-vit-base · mask-generation · 375 MB · runs on CPU or GPU
“Hi, how are you? How’s your day going?”
prompt your text
audio press play
facebook/mms-tts-eng · text-to-speech · 145 MB · runs on CPU or GPU

API and MCP Support

One switch in Settings turns your local models into an OpenAI endpoint - and into tools Claude can call. Off by default. Localhost only. It never serves the app's interface.

chat.py 
from openai import OpenAI

client = OpenAI(base_url="http://localhost:11500/v1", api_key="local")

resp = client.chat.completions.create(
    model="Qwen/Qwen2.5-0.5B-Instruct",
    messages=[{"role": "user", "content": "Hello!"}],
    stream=True,
)
  • ◇
    Drop-in OpenAI SDK. Set base_url, use any api key. Nothing else changes.
  • ◈
    LangChain & LangGraph. ChatOpenAI(base_url=…) just works, so your agents run on local weights.
  • ≋
    Streaming. stream=True streams tokens over SSE, exactly like the hosted API.
  • ⚙
    Tool & function calling. Parsed per family - Qwen · Hermes · Llama · Mistral.
  • ☰
    Shares warm models. It runs inside the engine, so it reuses what the app already loaded - not a second copy.

Claude, with your models.

Register the MCP server once - the app copies the exact command for your install - and Claude can run anything InferML has installed. Eleven tools.

▣detect_objects
▦segment_image
✦generate_image
♬transcribe_audio
♪text_to_speech
✎generate_text
⋮embed_text
⚲search_models
↯download_model
☰list_models
◉inferml_status

Not just LLMs.

Eleven task workspaces. Every major modality.

⊡

Detection

DETR, YOLOS, RT-DETR, D-FINE, Table Transformer. Draws labeled boxes server-side.

◉

Segmentation

SegFormer, Mask2Former, OneFormer, EoMT. Panoptic, instance, semantic. Composited overlays.

✦

Mask generation

SAM v1, SAM 2, SAM 2.1, SAM 3. Auto grid-sampling mode, full multi-region output.

☰

VLMs

Qwen-VL, LLaVA, Florence-2, Moondream, PaliGemma. Ask anything about an image.

♪

Speech

Whisper, Wav2Vec2, MMS for ASR. SpeechT5, Bark, VITS for TTS. Both directions, long-audio aware.

◈

Classification

ViT, ResNet, ConvNeXt, BEiT, SigLIP, CLIP. Image, zero-shot, audio. Confidence-ranked labels.

❋

Diffusion

Stable Diffusion, SDXL, FLUX, Kandinsky, PixArt. Text-to-image, img2img, inpaint.

⌨

Text generation

Llama, Mistral, Qwen, Gemma, Phi, DeepSeek. Chat-template aware, reasoning-model aware.

◐

Depth

DPT, MiDaS, ZoeDepth, Depth Anything v1/v2, Depth Pro. Single image → colorized depth map.

▤

Documents · OCR

TrOCR, Donut, LayoutLMv3, Pix2Struct. Read scanned pages, receipts, forms. Ask questions about them.

Everything in the Hub, ready to run.

200+ model families, each one verified against our architecture whitelist. If it shows up in InferML, it loads. No broken downloads, no missing packages, no guesswork.

Detection

DETRYOLOSRT-DETRRT-DETRv2D-FINEConditional-DETRDeformable-DETRTable-TransformerOWL-ViTOWLv2Grounding-DINO

Segmentation

SegFormerMaskFormerMask2FormerOneFormerEoMTUperNetBEiTDPTDETR-panopticMobileViT

Mask generation

SAMSAM 2SAM 2.1SAM 3MedSAM

VLMs

Qwen-VLQwen2.5-VLQwen3-VLLLaVALLaVA-NextViP-LLaVAFlorence-2MoondreamPaliGemmaIdefics 2/3SmolVLMKosmos-2InternVLPixtralFastVLMLFM2-VLDeepSeek-VLJanus-ProFuyuOvisAriaGLM4VCohere2-VisionEmu3

Text generation

Llama 3/4GPT-OSSMistral 3Qwen 2/3Gemma 2/3/3nPhi 3/4DeepSeekSmolLM3OLMo 3OLMoEFalcon-H1Nemotron-HBitNetStarCoder 2CohereGraniteMiniMax

ASR · TTS

WhisperDistil-WhisperWav2Vec2MMSMoonshineParakeetSpeechT5BarkVITS

Diffusion

SD 1.5SD 2.1SDXLSD 3 / 3.5FLUX.1KandinskyPixArtSanaKolors

Classification

ViTDeiTSwinConvNeXtBEiTResNetEfficientNetMobileNetCLIPSigLIPSigLIP 2

Depth

DPTGLPNZoeDepthDepth AnythingDepth Anything v2Depth ProMiDaS

Documents · OCR

TrOCRDonutLayoutLMLayoutLMv2LayoutLMv3Pix2Struct

Download and run.

An app you install, not a server you babysit. You just need Python 3.10+, which we don't bundle; grab it here if you're missing it.

Windows .exe installer · x64
macOS .dmg · Apple Silicon & Intel
Linux .AppImage & .deb · x64

InferML isn't code-signed yet, so your OS will question it the first time you open it. Nothing is wrong with the download - here is what you'll see, and how to get through it. (The one-line terminal install below sidesteps most of these prompts.)

Windows

A blue SmartScreen page: "Windows protected your PC." There's no visible Run button - click More info first, then Run anyway.

macOS

Gatekeeper refuses the app because Apple can't check it for malicious software. Open System Settings → Privacy & Security, scroll down, and click Open Anyway. If macOS instead calls it damaged, clear the download flag: xattr -dr com.apple.quarantine /Applications/InferML.app

Linux

No gatekeeper, just permissions. The AppImage needs the executable bit - chmod +x InferML.AppImage - and then runs. The .deb installs with sudo apt install ./InferML.deb.

or install from a terminal
Windows · PowerShell
> irm https://inferml.vercel.app/install.ps1 | iex
macOS & Linux
$ curl -fsSL https://inferml.vercel.app/install.sh | sh