Skip to content

feat(template): Add image pipeline template#7863

Merged
steveruizok merged 14 commits intomainfrom
claude/image-pipeline-starter-KDrtm
Feb 11, 2026
Merged

feat(template): Add image pipeline template#7863
steveruizok merged 14 commits intomainfrom
claude/image-pipeline-starter-KDrtm

Conversation

@steveruizok
Copy link
Copy Markdown
Collaborator

@steveruizok steveruizok commented Feb 7, 2026

This PR adds a new "Image pipeline" template to the create-tldraw CLI. In order to let users explore visual node-based AI image generation workflows, this PR adds a complete pipeline builder template with custom shapes, connections, and an execution engine.

Kapture.2026-02-08.at.15.52.23_1.32x.mp4
pr-7733-walkthrough.mp4

What's included

Core systems

  • Node system: Custom shape utilities for pipeline nodes with support for 17 different node types:
    • Inputs: Model, Prompt, Prompt Concat, Load Image, Number
    • Generation: Generate (image), Generate Text, ControlNet, IP Adapter, Style Transfer
    • Processing: Blend, Adjust, Upscale
    • Flow control: Router, Iterator
    • Output: Preview, Capture
  • Connection system: Visual bezier curve connections between nodes with type-checking and cycle detection to ensure valid pipelines
  • Port system: Typed input/output ports on nodes with data type validation (image, text, model, number, latent, any)
  • Execution engine: DAG-based execution engine that runs pipelines and tracks execution state across connected nodes

UI and interactions

  • Toolbar: Vertical toolbar with draggable node creation tools for all node types
  • On-canvas node picker: Interactive panel that appears when dragging a connection to empty space, showing only compatible node types
  • Pipeline regions: Automatic detection of connected node groups with bounding box overlays and play/stop buttons
  • Mid-connection insertion: Click the center handle of any connection to insert a compatible node
  • Visual feedback: Color-coded connections by data type, port compatibility indicators, and execution state visualization
  • Template system: Save and restore reusable subgraphs

Backend infrastructure

  • Cloudflare Worker backend: API routes for image generation, upscaling, style transfer, IP adapter, and text generation
  • Provider abstraction: Support for Replicate API (production) and placeholder providers (local development)
  • R2 image storage: Persistent image storage using Cloudflare R2
  • Default pipeline: Automatically creates a sample text-to-image pipeline (Model → Prompt → Generate → Preview) on first load

Documentation

  • Comprehensive README with setup instructions, architecture overview, and file structure
  • Deployment guide for Cloudflare Workers
  • Environment setup with .dev.vars configuration

API changes

  • Made Editor.getShapeIdsInsideBounds() public (removed @internal annotation) — this method is used by the template's execution engine to detect groups of connected nodes for pipeline region rendering

Change type

  • feature

Test plan

  1. Run yarn create tldraw and select the "Image pipeline" template
  2. Verify the default pipeline loads with model, prompt, generate, and preview nodes
  3. Create new nodes using the toolbar (drag or click to place)
  4. Connect nodes by dragging from output ports to input ports
  5. Verify color-coded connections based on data type
  6. Click the play button on a pipeline region to execute it
  7. Insert nodes mid-connection by clicking the center handle on a connection
  8. Drag a connection to empty space and verify the on-canvas node picker shows compatible nodes
  9. Verify type checking prevents incompatible connections (e.g., can't connect image to model)
  10. Verify cycle detection prevents circular connections
  11. Test with and without Replicate API key (should show placeholder images without key)
  12. Test saving and loading templates
  13. Verify execution state is properly tracked (loading, success, error states)
  • Unit tests
  • End to end tests

Release notes

  • Add new "Image pipeline" starter template with visual node-based AI image generation workflows

Note

Medium Risk
Large addition of a new template plus a small but user-visible editor API surface change; core runtime behavior is mostly isolated, but public API/promoted method could create compatibility expectations.

Overview
Introduces a new templates/image-pipeline starter kit: a node-and-connection pipeline canvas with typed ports, drag-to-connect UX (including on-canvas node insertion/picker), region-based play/stop controls, template saving, and a DAG execution runner wired to a Cloudflare Worker API client for generation/processing.

Updates the create-tldraw CLI template registry to include Image pipeline (shifting ordering of existing templates), and adds docs content/landing updates for the new starter kit.

Promotes Editor.getShapeIdsInsideBounds() from @internal to @public (and updates the API report) to support template functionality; also bumps Gemini model strings in docs and existing chat/branching-chat templates to gemini-3-flash-preview.

Written by Cursor Bugbot for commit d3f2a00. This will update automatically on new commits. Configure here.

A foundational starter kit for image generation pipeline apps, inspired
by ComfyUI and Flora. Features a typed port system with color-coded
data types (image, text, model, number, latent), type-safe connections,
image previews in nodes, and pipeline execution with DAG traversal.

Includes 8 node types: Model, Prompt, Generate, Load Image, Preview,
Blend, Adjust, and Number — covering the core building blocks of
image generation workflows.

https://claude.ai/code/session_019Kbhuu8tocPuvSd6sNZx1D
@huppy-bot huppy-bot bot added the feature New feature label Feb 7, 2026
@vercel
Copy link
Copy Markdown

vercel bot commented Feb 7, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
examples Ready Ready Preview Feb 11, 2026 10:46am
5 Skipped Deployments
Project Deployment Actions Updated (UTC)
analytics Ignored Ignored Preview Feb 11, 2026 10:46am
tldraw-docs Ignored Ignored Preview Feb 11, 2026 10:46am
chat-template Skipped Skipped Feb 11, 2026 10:46am
tldraw-shader Skipped Skipped Feb 11, 2026 10:46am
workflow-template Skipped Skipped Feb 11, 2026 10:46am

Request Review

@steveruizok steveruizok marked this pull request as draft February 7, 2026 20:20
…or, multi-model, templates

New nodes:
- Router: fan-out utility with 'any' typed ports
- Prompt Concat: merge prefix/main/suffix text inputs with separator
- Upscale: 2x/4x resolution with method selection
- ControlNet: image-guided generation with canny/depth/pose/segmentation
- Iterator: batch-process items through pipeline with progress bar

Enhanced:
- ModelNode now supports multiple providers (SD, Flux, DALL-E, Midjourney)
- 'any' port type that matches all data types
- Reusable workflow templates (save selection, stamp from picker)
- Template picker UI with save/delete/stamp actions

https://claude.ai/code/session_019Kbhuu8tocPuvSd6sNZx1D
@steveruizok steveruizok changed the title Add image pipeline template feat(template): Add image pipeline template Feb 7, 2026
claude and others added 8 commits February 7, 2026 22:12
Wire Generate, ControlNet, and Upscale nodes to a real backend with
support for Stability AI, fal.ai (Flux), and OpenAI DALL-E providers.
Falls back to local placeholder SVGs when no API keys are configured.

- worker/worker.ts: itty-router entry point with /api/generate, /api/upscale, /api/images endpoints
- worker/routes/generate.ts: Multi-provider image generation (Stability, fal.ai, OpenAI)
- worker/routes/upscale.ts: AI-enhanced upscaling via fal.ai and Stability AI
- worker/routes/images.ts: R2 bucket image persistence with edge caching
- src/api/pipelineApi.ts: Frontend API client with graceful local fallback
- wrangler.toml: Cloudflare Worker config with R2 bucket binding

https://claude.ai/code/session_019Kbhuu8tocPuvSd6sNZx1D
Introduce a full image-pipeline template: add README, frontend UI, new nodes, icons, API clients, worker routes/providers, and styling.

Highlights:
- New README with setup, providers, nodes, and deployment docs.
- App layout: add ImagePipelineSidebar and integrate it into the canvas; expose editor state for the sidebar.
- Sidebar: draggable node palette to create nodes on the canvas.
- Toolbar: reorganized PipelineToolbar, moved template picker into a popover and added default quick actions and actions menu.
- Node UI: Node footer with Play/Stop execution controls, duplicate/download/clear actions, and related dropdown menu.
- Connection logic: ConnectionShapeUtil now tracks and replaces pending connections when reconnecting to occupied ports to avoid duplicates and cycles.
- API: pipelineApi adds apiIPAdapter and apiStyleTransfer with local SVG placeholders for offline dev.
- New node types: IP Adapter and Style Transfer registered; node height/constants updated to account for node footers.
- Icons: IPAdapterIcon, StyleTransferIcon (and other small icon additions referenced in nodes).
- CSS: extensive styles for node footer, sidebar, template popover, load image node, and overall image-pipeline layout.
- Worker: new provider modules (openai, replicate, stability, google, placeholder) and routes (ipAdapter, styleTransfer) plus related worker changes to support the new endpoints.

These changes add image-guided and style-transfer capabilities, a sidebar node picker, improved node controls and UX, and backend worker plumbing for multiple providers.
Introduce a new GenerateText node (UI, icon, CSS, and worker route) and backend API to generate text from multimodal inputs with a placeholder fallback. Add multi-connection/input support across the pipeline: port multi flag, InfoValue multi types, input aggregation, execution graph handling, and UI/display adjustments (copy text, result area). Refactor image providers to consolidate on Replicate (remove several provider files), add Replicate handlers for SD and Google models, and simplify provider selection. Misc: worker config env var cleanup, executionState race-safety fix, minor node layout/ordering tweaks, and other small fixes.
Add typed port compatibility checks and tooling for the image-pipeline template. Introduces a new portCompatibility util and updates connection/insert/pointer flows to choose the first compatible port when creating nodes mid-connection (prevents invalid connections and enables type-aware node insertion).

Other updates:
- Add dynamic node toolbar items (auto-generate ToolbarItem per node definition).
- Improve node result handling: unify result keys, support multiple output types, and clear all result fields.
- Make Iterator node return current item and parse multiline items correctly.
- Change default model provider to `flux:flux-dev` and randomize seed for generated nodes.
- Documentation: overhaul README (features, setup, endpoints, models, deploy instructions).

These changes improve UX when wiring typed nodes and make the template docs and defaults clearer for local development.
Introduce explicit ordering for connections and robust input coercion, and add per-node resultKeys.

- ConnectionBindingUtil: add optional order prop, auto-assign order for end-terminal bindings, and persist order on create/update.
- ExecutionGraph & nodePorts: sort inbound connections by order when assembling inputs so multi-input ports are deterministic.
- NodeShapeUtil & node types: replace a global RESULT_KEYS with per-definition resultKeys and use node definition defaults to detect/clear results. Update many node definitions to declare resultKeys.
- shared.tsx: add NodeDefinition.resultKeys and helper utilities (coerceToText/coerceToNumber, getInput, getInputMulti, getInputText, getInputNumber) for safe input handling.
- GenerateText and worker route: coerce input to string everywhere (api + worker) to avoid runtime errors when treating inputs as text, and use coerced values in placeholders and prompt/image handling.
- Minor: guard against missing outputs when collecting port values.

These changes make input handling safer, allow ordered multi-input connections, and provide a consistent way to track and clear node results.
Add a reference-image port to the Generate node and pass referenceImageUrl through to providers; default model changed to flux:flux-dev. Propagate reference image into Flux/Google Replicate requests and remove older Stable Diffusion handling from the Replicate provider. Mark image previews and text results as loading when a node is out-of-date (shimmer animation + placeholder styling), update NodeImagePreview to use contain, and add conditional classes to many node components. UI tweaks: adjust node row/header sizing, button/icon sizes and paddings, stop context-menu propagation for input/select/textarea inside nodes, and small color tweak for the generic port. Also improve prompt handling and adjust Generate node body height to accommodate the new image port.
Introduce a new Image Pipeline starter kit: adds docs (starter-kits/image-pipeline.mdx), bento UI entry, and overview listing. Add template files under templates/image-pipeline (package.json, icon, CSS, node utilities and types) including a new resizable Capture node that can capture a canvas region to an image URL. Update NodeShapeUtil and node infrastructure to support variable node widths and optional resizable nodes (getNodeWidthPx, canResizeNode, onResize changes), and add styling for the Capture node. Also bump template ordering in create-tldraw and package.jsons for image-pipeline, branching-chat, and shader.
@cloudflare-workers-and-pages
Copy link
Copy Markdown

cloudflare-workers-and-pages bot commented Feb 8, 2026

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Updated (UTC)
✅ Deployment successful!
View logs
image-pipeline-template d3f2a00 Feb 11 2026, 10:47 AM

@mimecuvalo
Copy link
Copy Markdown
Member

shiiiiip

Switch references from gemini-2.5-flash to gemini-3 variants (docs, chat/worker routes, and Replicate endpoint). Broaden image detection to treat http/https URLs as images in several image-pipeline handlers. Add a reusable resizeNode helper and integrate it into NodeShapeUtil, update CaptureNode to use editor geometry for correct sizing, simplify App imports/UI (remove custom Style/Menu panels), prefer exact port type matches in portCompatibility, and make Editor.getShapeIdsInsideBounds public. These changes update the model usage and improve node resizing, image input handling, and UI simplification in the image-pipeline template.
@huppy-bot
Copy link
Copy Markdown
Contributor

huppy-bot bot commented Feb 9, 2026

API Changes Check Passed

Great! The PR description now includes the required "### API changes" section. This helps reviewers and SDK users understand the impact of your changes.

@steveruizok
Copy link
Copy Markdown
Collaborator Author

@claude add the api changes section to the PR description. And update the PR description in general to be more complete.

@tldraw tldraw deleted a comment from claude bot Feb 9, 2026
@steveruizok steveruizok marked this pull request as ready for review February 11, 2026 10:22
Replace Path2D.roundRect with an explicit arcTo-based path that computes width, height, and a constrained corner radius (r = min(scale, w/2, h/2)). This prevents an oversized corner radius when the note is scaled or when its height is small, ensures a properly closed rounded rectangle, and makes the indicator rendering more robust across sizes.
Remove the local placeholder image provider and inline placeholder generators, and update the Replicate provider to throw when REPLICATE_API_TOKEN is not configured. Add a NodeImage component that hides itself on load errors and replace many <img> usages across node components to use NodeImage. Sidebar and on-canvas picker behavior adjusted to skip hidden/empty node categories and move prompt_concat to the appropriate group; NodeDefinitions ordering updated accordingly. Update README to reflect that generation will fail without a token.
Copy link
Copy Markdown

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 2 potential issues.

Bugbot Autofix is OFF. To automatically fix reported issues with Cloud Agents, enable Autofix in the Cursor dashboard.

},
},
])
}
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Default pipeline creates disconnected nodes without connections

High Severity

createDefaultPipeline creates four node shapes but never creates any connection shapes or bindings between them, so the "default pipeline" is actually four disconnected nodes. The stampTemplate function in templateActions.ts shows the correct pattern: create a connection shape, then create two bindings (start and end) linking it to the source and target nodes. Without connections, pipeline region overlays with play buttons won't appear (they require groups of 2+ connected nodes), and the first-run experience doesn't match what the README and PR description promise.

Fix in Cursor Fix in Web

return (await response.json()) as GenerateResult
} catch (e) {
throw new Error(`Backend unavailable: ${e instanceof Error ? e.message : e}`)
}
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

API catch block wraps all errors as unavailable

Medium Severity

The catch block in every API function catches errors thrown by the !response.ok check and actual network failures, wrapping both with "Backend unavailable: ". A 400 response with {"error": "Invalid prompt"} becomes "Backend unavailable: Invalid prompt", which is misleading since the backend is available — it just rejected the request. This affects all five API functions: apiGenerate, apiUpscale, apiIPAdapter, apiStyleTransfer, and apiGenerateText.

Additional Locations (2)

Fix in Cursor Fix in Web

@steveruizok steveruizok added this pull request to the merge queue Feb 11, 2026
Merged via the queue into main with commit 2078ec4 Feb 11, 2026
20 checks passed
@steveruizok steveruizok deleted the claude/image-pipeline-starter-KDrtm branch February 11, 2026 10:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feature New feature

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants