Generate images and run vision models locally in the browser. Start with a model application; share its GPU device with your renderer when you need one.
Try the browser models · Start with Klein Image Studio · Choose a model application
Use the kit in your own application without the Kaminos editor. The Klein studio uses npm packages and downloads its hosted weights automatically: no Python environment, inference server or manual weight placement. Model-specific guides cover setup and download sizes for the other ports.
npm install @kaminos/webgpu-inference-kitThe Basin Maintenance Division monitors GitHub issues daily. Tell us what you tried and where you got stuck.
For image generation, material estimation and background removal, start with the model's application entry point. Your code supplies the input and uses the result; the application handles loading, queued inference, controls and cleanup. Install the matching model packages for the examples below.
Generate and display your first image with the complete Klein browser page, including setup, progress, repeated runs and model release. The short version:
import { createKleinApp } from '@kaminos/flux2-klein';
const klein = await createKleinApp();
try {
const image = await klein.generate({
prompt: 'a blue tank engine on a branch line', seed: 7,
width: 512, height: 512,
});
const canvas = document.createElement('canvas');
canvas.width = image.width;
canvas.height = image.height;
canvas.getContext('2d').putImageData(
new ImageData(image.rgba, image.width, image.height), 0, 0,
);
document.body.append(canvas);
} finally {
await klein.destroy();
}Klein downloads 3.2 GB of weights on first use and caches them for later visits. Keep one app alive for repeated generations, then destroy it when leaving the feature. The complete guide covers browser and memory requirements.
The same lifecycle supports SuperMat's image-to-material workload:
import { createSuperMatApp, decodeImageRgba } from '@kaminos/supermat';
const materials = await createSuperMatApp();
try {
const image = await decodeImageRgba(imageFile, materials.device);
const result = await materials.infer({ image, size: 512 });
// result.maps contains albedo, roughness, metallic and ORM images.
// result.planes retains the floating-point material outputs.
} finally {
await materials.destroy();
}imageFile is the browser File selected by your application. Both apps reuse loaded model state across sequential calls and expose load(), pause(), resume(), stop(), snapshot() and destroy(). Create with load: false to attach your renderer before weights load. Submit its frames through app.foreground.request(...); the same service accepts them before, during and after inference. Connect a renderer for the request loop and ownership rules.
Pass { device } to share your renderer's GPU, or { session, foreground } to share an existing coordinated host. Destruction preserves borrowed resources. Model options and outputs remain model-specific: Klein exposes pipelineOptions and app.pipeline; SuperMat exposes adapterOptions and app.adapter. Each accepts scheduling policy at creation or per run. Their application wrappers use the kit's shared admitted-run/control/cleanup runner, while each model retains its own loading and resource lifecycle.
UniMate uses the same runner through its existing host-injected createUniMateConsumer(host): provide a skeleton and prompt, receive an animated GLB, and retain solver controls and partial motion. Its run/read/togglePause/stop/destroy interface borrows your rendering host. Animate a skeleton shows the host integration.
U2Net uses createU2NetApp().removeForeground(...) for RGBA cutouts. It shares the application controls and preserves the loaded low-level runner at app.model.
Install the native-resolution model once with npx kaminos-sam-model --out public/models/sam3, then use it in an ordinary browser application:
import { createSam3Segmenter } from '@kaminos/webgpu-inference-kit/sam';
const sam = createSam3Segmenter({ manifestUrl: '/models/sam3/tensor-manifest.json' });
const result = await sam.segment(imageFile, 'windows');
const mask = result.mask(); // source-resolution Uint8Array: 0 or 1 per pixel
await sam.close();No Python, MLX, diagnostic server or Kaminos application is needed. Model setup downloads a pinned checkpoint and verifies every converted weight. The model files are about 3.4 GB and use Meta's SAM License. Start with SAM for setup, instance selection, resource ownership and shared-session integration.
For an initially selected object across successive frames, the source checkout also includes a SAM 3.1 video tracker and camera/video application. It carries temporal memory and shares the host's foreground scheduler; its prepared video model is separate from the image detector above.
Material Photograph brings two model ports and an interactive renderer together on one WebGPU device. MoGe recovers depth and normals; SuperMat estimates albedo, roughness and metalness. Rotate the four views together, move the light, and change the HDR environment to see what each model adds.
Explore the photograph in your browser with three included model results and no weight download. The local generation example keeps model resources resident, queues material inference through the kit, and restores completed outputs when you return to an image and resolution. Optional glow extraction and bloom complete the relighting experiment. Use your photograph locally and explore the composition.
Try the browser models: relight a photograph, generate an image with FLUX, animate a skeleton with UniMate, or reconstruct your hand. The generation demos run inference locally on your GPU.
Ride Thomas the Jank Engine: FLUX.2 [klein] 4B in your browser. Type a prompt and get an image generated entirely on your own GPU, at the jank level of your choosing. The Qwen3-4B text encoder, the 4-step diffusion transformer and the image decoder all run as WebGPU compute shaders, and nothing is sent to a server. A 512 × 512 image takes about 7 seconds on an Apple M4 Max and about 13 seconds on a 16 GB M2 Pro, while an animation on the same GPU keeps drawing at about 50 frames per second. The int4 weights download once (3.2 GB) and stay in the browser's cache.
The port is built on this kit: generation runs as a queued route invocation, its GPU work is submitted in command duties sized to measured throughput so the application's frames get through between them, and the inference control pauses, resumes or stops a run. The port's guide covers speed, memory and how to embed it.
For a visual first run, brighten a photo while a renderer stays active. The worked example walks through sharing a GPU device, queuing an operation, and displaying its result.
Kaminos WebGPU Inference Kit gives model ports a shared session and device lifecycle, persistent model routes, queued invocations, cooperative scheduling, progress and terminal state, resource residency, and runtime telemetry. Ports retain ownership of their weights, kernels, tensor semantics, execution order, and output construction.
For a model port or a host that owns its invocation, create a session, register a route, and queue model-owned work directly:
import { createWebGpuInferenceSession } from "@kaminos/webgpu-inference-kit/core";
const session = await createWebGpuInferenceSession({
sessionId: crypto.randomUUID(),
gpu: navigator.gpu,
adapterName: "browser-primary-adapter",
});
const route = await session.registerRoute({
routeId: "example.image-to-output.webgpu-local.v0",
});
const job = route.enqueue({
jobId: crypto.randomUUID(),
execute: invocation => runModel({ runtime: route.runtime, invocation }),
});
const completion = await job.completion;
if (completion.status === "succeeded") {
useModelOutput(completion.output);
}runModel is the port's model adapter. Reusable weights, pipelines, and buffers can remain resident through the registered route; the queued job captures invocation-specific input and resolves to an explicit terminal completion record.
Browser application
|
v
WebGpuInferenceSession
|
+-- Registered model route
| +-- reusable weights, pipelines, and buffers
| +-- queued invocation
| +-- model-specific output
|
+-- Registered model route
+-- reusable weights, pipelines, and buffers
+-- queued invocation
+-- model-specific output
The Klein and SuperMat application entry points and the UniMate generator assemble this session/route lifecycle for their callers. Their shared runner holds admission through control and foreground cleanup; ports retain model-specific loading and output construction. Application runner contract.
The session coordinates a shared WebGPU device, route lifecycle, global admission, and resource residency. A port typically places reusable model resources in route-scoped state and temporary run resources in invocation-scoped state.
A successful model port:
- Registers a route with the session
- Loads and retains its reusable model resources
- Accepts a well-defined invocation input
- Executes model-owned GPU, CPU, and worker work
- Reports meaningful progress and handles cancellation where its boundaries permit
- Retires temporary resources
- Returns a complete result the application can consume
Kaminos separates common runtime machinery from model implementation and product workflow:
| Inference kit | Model port | Application |
|---|---|---|
| Session and device coordination | Weights and pipelines | Product workflow |
| Route and invocation lifecycle | Tensor shapes and semantics | Route selection |
| Queued execution and admission | Preprocessing and postprocessing | Invocation inputs |
| Cooperative scheduling | Kernel dispatch and execution order | Progress presentation |
| Progress and terminal-state plumbing | Cooperative boundary placement | Foreground priorities |
| Timing and runtime telemetry | Output construction | Result presentation |
"Model port" names this integration role and architecture pattern. A port remains ordinary JavaScript, TypeScript, WGSL, and WebGPU code organized around the session, route, invocation, and output lifecycle.
The kit connects a growing family of browser model ports: generate an image from a text prompt, recover a scene's geometry, generate a textured object, turn an image into Gaussian splats, or animate a character from a text prompt. Each port brings its own model implementation and adopts shared runtime facilities where they serve its workload.
| Model Port | What You Can Build | Integration |
|---|---|---|
| FLUX.2 [klein] 4B | Images from a text prompt, generated on the visitor's GPU (live demo) | The text encoder, 4-step transformer and image decoder all run in the browser, with int4 weights cached after the first download. GPU work is submitted in duties sized to measured throughput, so the application keeps rendering, and the inference control can pause, resume or stop a run. |
| TRELLIS 2 | Textured 3D meshes from an image, exported as a GLB | Browser image-to-3D generation, stage-by-stage model loading, and a shared GPU device with the application renderer. |
| UniMate | Skeletal animation from a text prompt, generated for any rigged skeleton (a robot dog, a cat, a parrot, your own creature) | The Flan-T5 text encoder, the motion transformer, an adaptive ODE sampler and the decode to joint rotations all run in the browser. Each network evaluation is submitted as duties sized to measured throughput, so the application keeps rendering, and a run can be paused, resumed or stopped. |
| Hand reconstruction | Live hand meshes and landmarks from camera frames | WiLoR detection and MobRecon reconstruction use the shared application runner. The hand guide supplies checkout integration and model setup. |
| SuperMat · Application guide | Albedo, roughness and metalness for relighting a photograph | Shared-device material inference with reusable resources and queued invocations, composed with MoGe geometry and a live renderer. |
| U2Net | Foreground cutouts from opaque images | A background-removal application with loading, queued inference, controls and cleanup; lower-level runner and complete output provenance remain available. |
| SAM 3 | Instance masks from an image and text prompt | A complete browser WebGPU route with authenticated persistent model resources, cached image features, queued semantic requests, and same-device foreground submissions at phase boundaries. |
| MoGe | Depth maps, surface normals, and interactive point clouds from a single image | Shared device helpers, cooperative encoder and decoder work, bounded in-flight submissions, reusable GPU buffers, and a library build for embedding in a host application. |
| Stable Fast 3D | Textured, UV-unwrapped GLB meshes from a single image | Cooperative reconstruction and baking, bounded in-flight submissions, reusable scratch memory, worker offload, and a callable producer that can use the application's GPU device. |
| SHARP | Gaussian splat scenes from a single image | Adaptive cooperative scheduling, shared-device foreground rendering, staged output construction, and shared tensor-comparison helpers for port development. |
| Kimodo | Animated skeletal motion from a text prompt | Browser diffusion and motion decoding, bounded GPU submissions, reusable model resources, and a host-callable producer with rendering opportunities between transformer passes. Text embeddings come from an external server. |
Choose a port by its input, useful output and integration guide. Model implementations live either in this repository or in the linked standalone repositories; both use the same runtime family. Application interfaces, model setup and supported routes are described by each model guide. The kit package supplies the shared runtime; model packages and weights are installed separately.
TRELLIS 2 generates a textured 3D mesh from an image using WebGPU. It supports 512-resolution previews and higher-detail 1024-resolution generation, with local tools for background removal, mesh simplification and texture baking. Its model adapter can share the host application's GPU device and make room for rendering between model operations. The TRELLIS model guide explains the current developer setup and integration interfaces. The TRELLIS model code is in the Kaminos repository; its weights and local finishing tools are separate from this npm runtime.
The SAM 3 image detector finds object instances from an image and a text prompt. It runs in the browser, keeps reusable model weights in memory, and caches image features across prompts.
The SAM image example, included in kit
0.1.55, provides image upload, prompt entry, full-size masks and transparent
PNG cutouts. Applications can use the public ./core, ./sam and
./examples/sam-image entrypoints to build the same flow.
See the example guide for model setup, exports and performance measurements.
Ports can adopt a common application-facing shape:
shared session
-> persistent model route
-> queued invocation
-> model-owned work
-> cooperative boundaries where useful
-> complete model output
A new port identifies where reusable model state lives, what belongs to one invocation, how work enters the runtime, which execution boundaries are worth exposing, and what complete output the route returns.
Long WebGPU workloads can occupy the device or main thread long enough to make a functional model unpleasant to use inside an interactive product. A Kaminos port can expose work at model-meaningful boundaries such as transformer blocks, diffusion steps, decoder ranges, output batches, CPU phases, or worker jobs.
The runtime schedules those model duties so the browser can regain useful foreground opportunities between them. The port remains responsible for preserving model ordering and output semantics.
Ports can begin with direct execution and introduce cooperative boundaries where measurement shows that a phase is hostile to foreground responsiveness. The advanced integration reference covers scheduling policy, adaptive duty sizing, completion behavior, foreground opportunity donation, resources, multi-route admission, and runtime telemetry.
Using a Kaminos fire basin as the foreground workload? Follow Load an exported basin for a cooperative inference smoke for preset installation, exact-look verification, and the separate shared-device/frame-scheduling connection. Loading a preset alone does not configure cooperative inference.
In one measured run on an M4 Max in Chrome, SHARP generated 1,179,648 Gaussian splats in 185.3 seconds while a full Kaminos fire volume continued to simulate on every frame in the same browser and on the same GPU. Across 21,818 foreground frame intervals, p95 and p99 were 9.3ms and 10.0ms; 40 intervals exceeded 33.3ms.
Stable Fast 3D generated a complete textured GLB in 41.9 seconds while servicing 3,644 test host frames through the kit's shared-device foreground interlock. Page frame intervals had a p99 of 9.7ms and a maximum of 92.4ms. The GLB was byte-identical to the monolithic route's output.
Together, these examples show how model ports can expose useful scheduling boundaries, preserve their outputs, and make room for the application around them.
Per-kernel GPU timing is opt-in. Existing callers allocate no timing resources and encode no timestamp writes unless they explicitly request gpuTiming: true (or a timing options object) when creating a runtime. A session route accepts the same option through runtimeOptions:
const route = await session.registerRoute({
routeId: "example.image-to-output.webgpu-local.v0",
runtimeOptions: { gpuTiming: true },
});The device must expose timestamp-query; otherwise the timing summary reports unavailable. Call await route.runtime.settleGpuTiming() before reading completed gpuMs / gpuStatus stage records and the gpuTiming profile summary. Wall-clock stage timing stays separate. Invalid timestamps remain visibly invalid; observer failures are counted separately as listenerErrors.
Each physical timestamp pair is used once per query set. Exhausted sets retire after their readbacks settle, then a replacement begins the next epoch. That rollover can delay admission of the next kernel, so compare timing-enabled and timing-disabled execution when evaluating application performance. The standalone timer is exported as createWebGpuGpuTimer; explicit true, "auto", and options objects enable it, while omitted options, null, false and "off" leave it disabled.
- Follow Getting Started to run a complete minimal WebGPU model port with one reusable route and two queued invocations.
- Read the advanced integration reference for the complete current API manual and scheduling contracts.
- Inspect the public exports in
src/index.js. - Start with one session, one registered route, one complete invocation, and direct model execution. Add cooperative boundaries where measurements show the application needs them.
MIT. See LICENSE.

