Inference Provider API (IPA)

A proposed browser standard for provider-agnostic AI inference.

Status: Experimental Draft

Spec: SPEC.md

Try It

Build an app with a coding agent

Install the build-with-ipa skill in your app project:

npx skills add SamSamskies/inference-provider-api --skill build-with-ipa

Then ask your agent to add an IPA-powered feature to your app. The skill guides integration with the browser API and points to the spec, helpers, and examples.

Reference Implementation

Inference Bridge is the reference Chrome and Firefox extension that injects window.inference and connects to local or cloud providers.

Install from the Chrome Web Store or Firefox Add-ons. See its installation guide for development builds.

Example Applications

• Examples index — gallery of demo apps (source)
• Topic Explorer — a browser-native research agent built with ipa-tools createAgent that searches Wikipedia, reads articles, and answers with sources and live activity (source)
• Chat demo — minimal chat UI that uses the API (source)
• Social demo — post + replies with a Grok-like Ask AI panel (source)
• Translate demo — short haiku translated with ipa-tools complete (source)
• Avatar demo — describe a character, generate an SVG avatar with ipa-tools complete, download PNG (source)
• Nostr feed demo — a feed of Nostr notes filtered with natural language via IPA tool calling (source)
• Charts demo — natural-language charts via IPA tool calling (render_chart generative UI) (source)
• Speech demo — transcribe an uploaded file, generate speech, and play MP3 or WAV (source)
• Decisions demo — triage a message with typed category, urgency, and reply-needed judgments (source)
• Image decisions demo — choose and preview a local image, then inspect it for visible damage when decision-image support is advertised (source)
• Prompt API fallback demo — today-in-history fact via createInference with ipa-prompt-api-fallback when no extension is present (source)

Apps using IPA

IPA-first, with a hosted fallback if no extension is installed:

• Trending Nostr — trending notes with Ask AI (source)
• Advanced Nostr Search — search plus Ask AI (source)

Motivation

AI-powered web applications often have to:

• Ask users for API keys
• Integrate every inference provider separately
• Proxy requests through their own backend
• Build custom permission systems

IPA lets applications request inference through a user-controlled browser or extension without handling the user's API keys.

Inspired by NIP-07, IPA separates applications from providers, giving users complete control over where inference is performed.

Design Principles

1. Users own their API keys; applications never access them.
2. Applications use a provider-agnostic inference API.
3. Users choose providers and models.
4. Local and remote inference are first-class citizens.
5. Permission is explicit.

Example

request is required. Check window.inference?.request before enabling inference UI; a compatible browser or extension may be missing.

for await (const chunk of window.inference.request({
  method: "chat",
  messages: [
    {
      role: "user",
      content: `Is this true?:\n\nNostr is dead.`
    }
  ]
})) {
  if (chunk.type === "accepted") {
    // permission resolved; provider call may begin
  } else if (chunk.type === "reasoning_delta") {
    // optional reasoning output
  } else if (chunk.type === "delta") {
    // append chunk.content to the reply UI
  } else if (chunk.type === "done") {
    // final message / usage; message.reasoning when reasoning was streamed
  }
}

Use ipa-tools (npm install ipa-tools) when you only need the final chat result or a page-executed tool loop:

import { complete } from "ipa-tools";

const { model, message, usage } = await complete({
  method: "chat",
  messages: [{ role: "user", content: "Is this true?:\n\nNostr is dead." }],
});

The helpers are application code. The injected API stays on request and optional getFeatures.

Optional capabilities

getFeatures returns a synchronous snapshot of supported API features. Missing flags mean unsupported; flags do not guarantee that the user's selected provider or model can fulfill a request.

const features = window.inference.getFeatures?.() ?? {};

Capability	Feature gate	Request and result
Function tools	features.toolCalling	Function tools in tools; the page executes done.message.toolCalls, or uses ipa-tools's runTools
Hosted web search	features.webSearch	tools: [{ type: "web_search" }]; the injector or provider executes search
Image input	features.imageInput	Image parts in user or assistant messages
Image output	features.imageOutput	output: { images: true }; image parts appear on done.message.content
Typed decisions	features.methods?.decide	method: "decide" with JSON state and named noul / choice / score questions; accepted then done.answers, including per-question refusals
Decision images	features.methods?.decide === true and features.decisionMedia?.imageMediaTypes	Optional shared inline image attachments; native media, separate consent, same typed answers
Transcription	features.methods?.transcribe?.acceptedMedia	method: "transcribe" with completed media; text on done.transcript
Speech synthesis	features.methods?.synthesize	method: "synthesize" with text; MP3 by default, optional advertised WAV in audio_delta chunks
Chat options	features.options?.reasoningEffort / features.options?.temperature	Generation preferences in options; unsupported keys are ignored

Optional capabilities are independent. Unadvertised tools, hosted search, images, and optional methods are rejected with invalid_request. See the speech examples for uploads, format discovery and playback, URL input, and cancellation.

Typed decisions require nonempty questions; question-count and payload-size maxima are documented and enforced by each implementation, and selected providers/models may impose smaller limits. methods.decide: true advertises method and primitive support, not a request-size guarantee. Handle invalid_request for exceeded limits; implementations must not silently truncate input or automatically split a request to fit them. Applications may explicitly submit multiple requests when appropriate. See the decision validation rules.

Permissions

The injector obtains permission for the calling origin before sending content to a provider. The user chooses the provider and model. Remembered grants bind that route to the origin and method (plus voice for synthesis); changing a global default does not rewrite existing grants. A remembered denial blocks the origin.

Function tools, hosted search, image input/output, typed decisions, and speech require their own approval; a plain chat grant does not cover them. The approval UI must identify offered tools and capabilities, but a request preview is optional. Decision approval identifies question ids/types. Applications should disclose what page tools may send before the first request because later tool results can be sent without another preview.

Decision images require a separate MIME-scoped media grant; existing text-only decision or chat-image grants do not cover them. Approval lists each attachment's type and decoded size. See the native image demo. Unadvertised or malformed images are invalid_request; an advertised type unsupported on the selected route is unavailable. State URLs remain ordinary data.

An accepted chunk means permission has resolved, including through an existing grant; it does not prove a prompt appeared. See the specification's behavior, security, and capability-specific permission rules for details.

An inference/tool-description grant does not approve every side effect an application might execute. Page-owned tools can use ipa-tools' optional argument validation and app approval hooks before executing consequential actions.

Non-goals

• Replacing provider SDKs
• Billing
• Authentication
• Defining inference protocols
• Choosing the "best" model

Open Questions

Some topics that still need community discussion:

• Which further capability constraints, if any, do applications need beyond tools and options?
• Are "auto" | "none" | "low" | "medium" | "high" the right options.reasoningEffort levels, or should the field become a provider-mapped budget/token object?
• Which further keys belong under options (for example maxTokens)?
• Should embeddings use this API or a separate API?
• Should other hosted / provider-executed tools (for example MCP) be specified, or remain implementation-specific?
• Should tool calls stream as their own chunk type, or stay on done.message.toolCalls only?
• Should arbitrary structured chat outputs (e.g. JSON Schema / responseFormat, beyond typed decisions) be part of IPA, or left to prompt engineering until providers converge?

Contributing

For package development, use Node.js 22 (22.12 or later), 24, or 26+ and npm. From the repository root, run npm ci, then npm test to build both packages and run their tests and TypeScript checks.

Open an issue or pull request for spec feedback, API proposals, examples, reference implementations, or related standards and prior art.

License

This project is licensed under the MIT License. See LICENSE for details.