Azure OpenAI
The Azure OpenAI provider contains language model support for the Azure OpenAI chat API.
Start with AI Gateway for models in its catalog (recommended), or use @ai-sdk/azure to connect directly to Azure OpenAI.
Using AI Gateway
The AI SDK uses AI Gateway by default for model strings. Gateway provides access to models across providers through a single API key, with usage and cost tracking. Install ai:
$ pnpm add ai
$ npm install ai
$ yarn add ai
$ bun add ai
Set an AI Gateway API key in your environment:
Vercel deployments can also use OIDC authentication. See the AI Gateway provider documentation for configuration and the model catalog for available models.
The example sets providerOptions.gateway.order to ['azure']. This sends the request to Azure OpenAI first, then falls back to other available providers of the same model if Azure OpenAI is unavailable or the request fails. order sets a preference; it does not restrict fallback to Azure OpenAI. See provider ordering.
The dropdown lists models served by Azure OpenAI through Gateway with a corresponding direct-provider example. Provider availability depends on the model and your team's Provider Allowlist.
Using @ai-sdk/azure
For direct access to Azure OpenAI, install ai and @ai-sdk/azure:
$ pnpm add ai @ai-sdk/azure
$ npm install ai @ai-sdk/azure
$ yarn add ai @ai-sdk/azure
$ bun add ai @ai-sdk/azure
Set your Azure API key and resource name:
The direct example uses an Azure deployment name. Replace it with the deployment name configured in your resource.
Use the direct provider for your Azure deployments, Microsoft Entra ID authentication, or Azure-specific APIs. Replace the deployment name in the Azure example with one from your resource.
The following sections document the direct @ai-sdk/azure provider, including
its model factories and settings. Gateway model and feature availability can
differ.
Provider Instance
You can import the default provider instance azure from @ai-sdk/azure:
If you need a customized setup, you can import createAzure from @ai-sdk/azure and create a provider instance with your settings:
You can also use a complete Microsoft Foundry OpenAI v1 base URL:
For Microsoft Entra ID authentication, you can provide a token provider.
Install @azure/identity separately if you want to use its credential helpers:
You can use the following optional settings to customize the OpenAI provider instance:
-
resourceName string
Azure resource name. It defaults to the
AZURE_RESOURCE_NAMEenvironment variable.The resource name is used in the assembled URL:
https://{resourceName}.openai.azure.com/openai/v1{path}. It must be a single DNS label (letters, digits, and hyphens). You can usebaseURLinstead to specify the URL prefix. -
apiKey string | PromiseLike<string> | (() => string | PromiseLike<string>)
Pass a callback to resolve the API key before each request. The callback can return a string or a promise; token caching and renewal are handled by the caller.
API key that is being sent using the
api-keyheader. It defaults to theAZURE_API_KEYenvironment variable. -
tokenProvider () => Promise<string>
Microsoft Entra ID token provider that returns an access token to be sent using the
Authorizationheader. When provided, this is used instead ofapiKey. -
apiVersion string
Sets a custom api version. Defaults to
v1. This setting is applied when the provider constructs the v1 path or uses deployment-based URLs. Complete v1 base URLs are used as-is. -
baseURL string
Use a different URL prefix for API calls, e.g. to use proxy servers.
Either this or
resourceNamecan be used. When a baseURL is provided, the resourceName is ignored.For Azure-hosted URLs ending in
.openai.azure.com,.services.ai.azure.com, or.cognitiveservices.azure.com, you can pass either an unversioned OpenAI prefix such ashttps://your-resource.services.ai.azure.com/openaior a complete v1 base URL such ashttps://your-resource.services.ai.azure.com/openai/v1. The provider appends/v1and the configuredapi-versionto unversioned resource URLs. Complete v1 URLs are used as-is.Complete Microsoft Foundry project v1 URLs, such as
https://your-resource.services.ai.azure.com/api/projects/your-project/openai/v1, are also used as-is. If/v1is omitted from a Foundry project URL, the provider appends it without adding anapi-versionquery parameter.With a non-Azure custom gateway baseURL, the resolved URL is
{baseURL}{path}; the SDK does not append/v1or anapi-versionquery parameter in this mode. -
headers Record<string,string>
Custom headers to include in the requests.
-
fetch (input: RequestInfo, init?: RequestInit) => Promise<Response>
Custom fetch implementation. Defaults to the global
fetchfunction. You can use it as a middleware to intercept requests, or to provide a custom fetch implementation for e.g. testing. -
useDeploymentBasedUrls boolean
Use deployment-based URLs for API calls. Set to
trueto use the legacy deployment format:{baseURL}/deployments/{deploymentId}{path}?api-version={apiVersion}instead of{baseURL}/v1{path}?api-version={apiVersion}. Defaults tofalse.This option is useful for compatibility with certain Azure OpenAI models or deployments that require the legacy endpoint format.
-
speechBaseURL string
URL prefix for Azure Speech requests (MAI-Transcribe transcription and MAI-Voice speech generation), for example a regional endpoint such as
https://eastus.api.cognitive.microsoft.com. Defaults tohttps://{resourceName}.cognitiveservices.azure.com. Speech requests do not usebaseURLorapiVersion. -
maiBaseURL string
URL prefix for MAI requests (MAI-Image and MAI-Transcribe-2-Streaming). Defaults to
https://{resourceName}.services.ai.azure.com/mai/v1. -
webSocket WebSocketConstructor
Custom WebSocket implementation for MAI streaming transcription, for example the
WebSocketexport of thewspackage in Node.js. With a custom implementation the API key is sent in theapi-keyheader; with the nativeWebSocketit is sent as theapi-keyquery parameter. Microsoft Entra ID (tokenProvider) authentication requires a custom implementation; without one, streaming fails with anInvalidArgumentErrorbefore connecting.
Language Models
The Azure OpenAI provider instance is a function that you can invoke to create a language model:
You need to pass your deployment name as the first argument.
You can optionally pass the underlying model ID in a settings object:
The SDK uses the model ID as the model identity, including in telemetry, and to detect model capabilities such as reasoning support. API requests still use the deployment name. The model ID returned by Azure in response metadata is preserved.
If omitted, the deployment name is used for model identity and capability detection. Keep the supplied model ID aligned with the model running in your deployment.
Reasoning Models
Azure exposes the thinking of DeepSeek-R1 in the generated text using the <think> tag.
You can use the extractReasoningMiddleware to extract this reasoning and expose it as a reasoning property on the result:
You can then use that enhanced model in functions like generateText and streamText.
The Azure provider calls the Responses API by default (unless you specify e.g.
azure.chat).
Example
You can use OpenAI language models to generate text with the generateText function:
OpenAI language models can also be used in the streamText function
and support structured data generation with Output
(see AI SDK Core).
Azure OpenAI sends larger chunks than OpenAI. This can lead to the perception that the response is slower. See Troubleshooting: Azure OpenAI Slow To Stream
Provider Options
When using OpenAI language models on Azure, you can configure provider-specific options using providerOptions.openai. More information on available configuration options are on the OpenAI provider page.
The same options are also accepted under providerOptions.azure. Responses models use providerOptions.azure when it is present and providerOptions.openai otherwise. Chat and completion models merge both, and providerOptions.azure wins when both set the same option.
Chat Models
The URL for calling Azure chat models will be constructed as follows:
https://RESOURCE_NAME.openai.azure.com/openai/v1/chat/completions?api-version=v1
You can create models that call the Azure OpenAI chat completions API using the .chat() factory method:
Azure OpenAI chat models support also some model specific settings that are not part of the standard call settings. You can pass them as an options argument:
The following optional provider options are available for OpenAI chat models:
-
logitBias Record<number, number>
Modifies the likelihood of specified tokens appearing in the completion.
Accepts a JSON object that maps tokens (specified by their token ID in the GPT tokenizer) to an associated bias value from -100 to 100. You can use this tokenizer tool to convert text to token IDs. Mathematically, the bias is added to the logits generated by the model prior to sampling. The exact effect will vary per model, but values between -1 and 1 should decrease or increase likelihood of selection; values like -100 or 100 should result in a ban or exclusive selection of the relevant token.
As an example, you can pass
{"50256": -100}to prevent the token from being generated. -
logprobs boolean | number
Return the log probabilities of the tokens. Including logprobs will increase the response size and can slow down response times. However, it can be useful to better understand how the model is behaving.
Setting to true will return the log probabilities of the tokens that were generated.
Setting to a number will return the log probabilities of the top n tokens that were generated.
-
parallelToolCalls boolean
Whether to enable parallel function calling during tool use. Default to true.
-
user string
A unique identifier representing your end-user, which can help OpenAI to monitor and detect abuse. Learn more.
DeepSeek Chat Models
You can create Azure-hosted DeepSeek chat models using the .deepseek() factory method:
Use this factory for Azure DeepSeek models that support DeepSeek's chat
reasoning fields, such as deepseek-v4-pro and deepseek-v4-flash. The
factory maps top-level reasoning to DeepSeek reasoning_effort request
settings, and parses streamed reasoning_content as AI SDK reasoning parts.
Responses Models
Azure OpenAI uses responses API as default with the azure(deploymentName) factory method.
Further configuration can be done using OpenAI provider options.
You can validate the provider options using the OpenAILanguageModelResponsesOptions type.
In the Responses API, use azure as the provider name in providerOptions
instead of openai. The openai key is still supported for providerOptions
input.
The following provider options are available:
-
parallelToolCalls boolean Whether to use parallel tool calls. Defaults to
true. -
store boolean Whether to store the generation. Defaults to
true. -
metadata Record<string, string> Additional metadata to store with the generation.
-
previousResponseId string The ID of the previous response. You can use it to continue a conversation. Defaults to
undefined. -
instructions string Instructions for the model. They can be used to change the system or developer message when continuing a conversation using the
previousResponseIdoption. Defaults toundefined. -
user string A unique identifier representing your end-user, which can help OpenAI to monitor and detect abuse. Defaults to
undefined. -
reasoningEffort 'low' | 'medium' | 'high' Reasoning effort for reasoning models. Defaults to
medium. If you useproviderOptionsto set thereasoningEffortoption, this model setting will be ignored. -
strictJsonSchema boolean Whether to use strict JSON schema validation. Defaults to
false.
The Azure OpenAI provider also returns provider-specific metadata:
For Responses models (azure(deploymentName)), you can type this metadata using AzureResponsesProviderMetadata:
The following Azure-specific metadata may be returned:
- responseId string | null | undefined The ID of the response. Can be used to continue a conversation.
- logprobs (optional) Log probabilities of output tokens (when enabled).
- serviceTier (optional) Service tier information returned by the API.
The providerMetadata is only returned with the default responses API, and is not supported when using 'azure.chat' or 'azure.completion'
Changing Reasoning Effort Mid-Conversation
Azure Responses models support message-level reasoning effort updates through
providerOptions.azure.reasoningEffortUpdate. The openai namespace is also
accepted when the message has no azure options. Use the
OpenAIResponsesSystemMessageOptions type exported from @ai-sdk/azure to
validate these options:
Keep the request-level reasoning unchanged throughout the conversation and
retain earlier effort updates when resending message history. These updates
also work with streamText. See the OpenAI reasoning effort
guide
for continuation, compaction, and configuration restrictions.
The SDK infers support for effort updates from the optional modelId setting
passed to azure.responses(), falling back to the deployment name. The ID must identify
a supported GPT-6-or-later model family, such as gpt-6-astra. A custom name
such as production is not recognized, even when the underlying deployment
runs GPT-6. Pass the underlying model ID, for example
azure.responses('production', { modelId: 'gpt-6-astra' }), to enable detection
for a custom deployment name.
Web Search Tool
The Azure OpenAI responses API supports web search through the azure.tools.webSearch tool.
azure.tools.webSearchPreview() is deprecated. Use azure.tools.webSearch()
instead. The existing searchContextSize and userLocation options are
supported by both tools.
The web search tool supports the following configuration options:
- externalWebAccess boolean - Whether to use external web access for fetching live content. Defaults to
true. - searchContextSize 'low' | 'medium' | 'high' - Controls the amount of context used for the search. Higher values provide more comprehensive results but may have higher latency and cost.
- userLocation - Optional location information to provide geographically relevant results. Includes
type(always'approximate'),country,city,region, andtimezone. - filters - Optional filter configuration to restrict search results.
- allowedDomains string[] - Up to 100 allowed domains for the search.
- blockedDomains string[] - Up to 100 blocked domains for the search.
Omit the HTTP or HTTPS prefix from domain filters. Subdomains of configured domains are automatically included or excluded.
For detailed information on configuration options see the Azure OpenAI Web Search Tool documentation.
On Microsoft Foundry environment , Live internet access is not supported. The
parameter externalWebAccess if passed will be ignored.
File Search Tool
The Azure OpenAI provider supports file search through the azure.tools.fileSearch tool.
You can force the use of the file search tool by setting the toolChoice parameter to { type: 'tool', toolName: 'file_search' }.
The tool must be named file_search when using Azure OpenAI's file search
functionality. This name is required by Azure OpenAI's API specification and
cannot be customized.
The 'file_search' tool is only supported with the default responses API, and is not supported when using 'azure.chat' or 'azure.completion'
Image Generation Tool
Azure OpenAI's Responses API supports multi-modal image generation as a provider-defined tool.
Availability is restricted to specific models (for example, gpt-5 variants).
The tool must be named image_generation when using Azure OpenAI's image
generation functionality. This name is required by Azure OpenAI's API
specification and cannot be customized.
The 'image_generation' tool is only supported with the default responses API, and is not supported when using 'azure.chat' or 'azure.completion'
To use image_generation, you must first create an image generation model. You
must add a deployment specification to the header
x-ms-oai-image-generation-deployment. Please note that the Responses API
model and the image generation model must be in the same resource.
When you set store: false, then previously generated images will not be
accessible by the model. We recommend using the image generation tool without
setting store: false.
Code Interpreter Tool
The Azure OpenAI provider supports the code interpreter tool through the azure.tools.codeInterpreter tool. This allows models to write and execute Python code.
The code interpreter tool can be configured with:
-
container: Either:
- a container ID (
string) to reuse an existing container, or - a configuration
objectto create and configure a new container.
- a container ID (
-
container configuration object
- fileIds string[] — File IDs to make available in the container at startup
- memoryLimit '1g' | '4g' | '16g' | '64g' — Memory limit for the container
The shared networkPolicy container option is exposed by
azure.tools.codeInterpreter, but outbound networking availability is
controlled by Azure OpenAI. Check the supported options for your deployment
and API version in the Azure Responses API
documentation.
The tool must be named code_interpreter when using Azure OpenAI's code
interpreter functionality. This name is required by Azure OpenAI's API
specification and cannot be customized.
The 'code_interpreter' tool is only supported with the default responses API, and is not supported when using 'azure.chat' or 'azure.completion'
When working with files generated by the Code Interpreter, reference
information can be obtained from both annotations in Text
Parts and providerMetadata in
Source Document Parts.
Custom Tool
Use azure.tools.customTool to send OpenAI-style
custom tools
with Azure Responses models. Availability depends on your Azure model deployment
and API version; see Microsoft's reasoning model documentation.
Custom tool calls contain a raw string input instead of JSON, optionally constrained
to a regex or Lark grammar. This is useful for SQL queries, code snippets, or other
text that must match a specific pattern.
Replace your-deployment-name below with an Azure deployment that supports
custom tools. These tools use the Responses API, rather than azure.chat or
azure.completion.
Custom tools also work with streamText:
The key in the tools object (write_sql above) names the tool.
The custom tool can be configured with:
- description string (optional) - A description of what the tool does, to help the model understand when to use it.
- format object (optional) - The output format constraint. Omit for unconstrained text output.
- type 'grammar' | 'text' - The format type. Use
'grammar'for constrained output or'text'for explicit unconstrained text. - syntax 'regex' | 'lark' - (grammar only) The grammar syntax. Use
'regex'for regular expression patterns or'lark'for Lark parser grammar. - definition string - (grammar only) The grammar definition string (a regex pattern or Lark grammar).
- type 'grammar' | 'text' - The format type. Use
- execute function (optional) - An async function that receives the raw string input and returns the tool result. Enables multi-turn tool calling.
Apply Patch Tool
Use azure.tools.applyPatch to configure the OpenAI apply patch tool for Azure Responses models.
Availability depends on your model deployment, region, and API version; see the
Azure Responses API documentation.
The apply patch tool lets the model create, update, and delete files in your codebase using structured diffs.
Instead of just suggesting edits, the model emits patch operations that your application applies and reports back on,
enabling iterative, multi-step code editing workflows.
Register this tool with the apply_patch key so patch results are returned in the
format expected by the Responses API.
Your execute function must return:
- status 'completed' | 'failed' - Whether the patch was applied successfully
- output string (optional) - Human-readable log text (e.g., results or error messages)
Shell Tool
Configure the shell tool for Azure Responses models with azure.tools.shell.
Availability of shell execution, hosted containers, container references, and skills
depends on your model deployment, region, and API version. See the
Azure Responses API documentation.
The shell tool allows running bash commands and interacting with a command line.
The model proposes shell commands; your application executes them in local mode,
while the Azure OpenAI service executes them in hosted mode.
Running arbitrary shell commands can be dangerous. Always sandbox execution or add strict allow-/deny-lists before forwarding a command to the system shell.
The shell tool supports three environment modes that control where commands are executed:
Local Execution (default)
When no environment is specified (or type: 'local' is used), commands are executed locally via your execute callback.
Connect it to your sandbox; the placeholder below returns a failed result without executing commands:
Hosted Container (auto)
Set environment.type to 'containerAuto' to run commands in a container hosted by the Azure OpenAI service. No execute callback is needed — Azure OpenAI handles execution server-side:
The Azure OpenAI Responses API shell tool currently does not support
networkPolicy. The shared factory accepts and forwards this option, so check
your deployment's service support before using it.
The containerAuto environment supports:
- fileIds string[] - File IDs to make available in the container
- memoryLimit '1g' | '4g' | '16g' | '64g' - Memory limit for the container
Existing Container Reference
Set environment.type to 'containerReference' to use an existing container by ID:
Execute Callback
For local execution (default or type: 'local'), your execute function must return an output array with results for each command:
- stdout string - Standard output from the command
- stderr string - Standard error from the command
- outcome - Either
{ type: 'timeout' }or{ type: 'exit', exitCode: number }
Skills
Skills are versioned bundles of files with a SKILL.md manifest that extend the shell tool's capabilities. They can be attached to both containerAuto and local environments.
Azure skill-ID resolution is not implemented by the shared shell adapter. Use local skill directories or inline archives instead of OpenAI platform skill references.
Local skills point to a directory on disk containing a SKILL.md file:
Inline container skills provide a ZIP archive containing the skill's SKILL.md
and supporting files. For a deployment that supports hosted shell skills:
Tool Search Tool
The Azure OpenAI provider supports tool search through the azure.tools.toolSearch tool.
Tool search lets the model load deferred tools only when it needs them. It requires gpt-5.4 or later.
Mark the tools to defer with deferLoading: true:
azure.tools.toolSearch accepts the same options as openai.tools.toolSearch, including client-executed tool search and namespaces.
PDF support
The Azure OpenAI provider supports reading PDF files.
You can pass PDF files as part of the message content using the file type:
The model will have access to the contents of the PDF file and
respond to questions about it.
The PDF file should be passed using the data field,
and the mediaType should be set to 'application/pdf'.
Reading PDF files are only supported with the default responses API, and is not supported when using 'azure.chat' or 'azure.completion'
Typed providerMetadata in Text Parts
When using the Azure OpenAI Responses API, the SDK attaches Azure OpenAI-specific metadata to output parts via providerMetadata.
This metadata can be used on the client side for tasks such as rendering citations or downloading files generated by the Code Interpreter. To enable type-safe handling of this metadata, the AI SDK exports dedicated TypeScript types.
For text parts, when part.type === 'text', the providerMetadata is provided in the form of AzureResponsesTextProviderMetadata.
This metadata includes the following fields:
-
itemId
The ID of the output item in the Responses API. -
annotations(optional) An array of annotation objects generated by the model. If no annotations are present, this property itself may be omitted (undefined).Each element in
annotationsis a discriminated union with a requiredtypefield. Supported types include, for example:url_citationfile_citationcontainer_file_citationfile_path
These annotations directly correspond to the annotation objects defined by the Responses API and can be used for inline reference rendering or output analysis. For details, see the official OpenAI documentation: Responses API – output text annotations.
When implementing file downloads for files generated by the Code Interpreter,
the container_id and file_id available in providerMetadata can be used
to retrieve the file content. For details, see the Retrieve container file
content
API.
Typed providerMetadata in Reasoning Parts
When using the Azure OpenAI Responses API, reasoning output parts can include provider metadata.
To handle this metadata in a type-safe way, use AzureResponsesReasoningProviderMetadata.
For reasoning parts, when part.type === 'reasoning', the providerMetadata is provided in the form of AzureResponsesReasoningProviderMetadata.
This metadata includes the following fields:
itemId
The ID of the reasoning item in the Responses API.reasoningEncryptedContent(optional)
Encrypted reasoning content (only returned when requested viainclude: ['reasoning.encrypted_content']).
Typed providerMetadata in Source Document Parts
For source document parts, when part.type === 'source' and sourceType === 'document', the providerMetadata is provided as AzureResponsesSourceDocumentProviderMetadata.
This metadata is also a discriminated union with a required type field. Supported types include:
file_citationcontainer_file_citationfile_path
Each type includes the identifiers required to work with the referenced resource, such as fileId and containerId.
Annotations in text parts follow the OpenAI Responses API specification and
therefore use snake_case properties (e.g. file_id, container_id). In
contrast, providerMetadata for source document parts is normalized by the
SDK to camelCase (e.g. fileId, containerId). Fields that depend on the
original text content, such as start_index and end_index, are omitted, as
are fields like filename that are directly available on the source object.
Completion Models
You can create models that call the completions API using the .completion() factory method.
The first argument is the model id.
Currently only gpt-35-turbo-instruct is supported.
OpenAI completion models support also some model specific settings that are not part of the standard call settings. You can pass them as an options argument:
The following optional provider options are available for Azure OpenAI completion models:
-
echo: boolean
Echo back the prompt in addition to the completion.
-
logitBias Record<number, number>
Modifies the likelihood of specified tokens appearing in the completion.
Accepts a JSON object that maps tokens (specified by their token ID in the GPT tokenizer) to an associated bias value from -100 to 100. You can use this tokenizer tool to convert text to token IDs. Mathematically, the bias is added to the logits generated by the model prior to sampling. The exact effect will vary per model, but values between -1 and 1 should decrease or increase likelihood of selection; values like -100 or 100 should result in a ban or exclusive selection of the relevant token.
As an example, you can pass
{"50256": -100}to prevent the <|endoftext|> token from being generated. -
logprobs boolean | number
Return the log probabilities of the tokens. Including logprobs will increase the response size and can slow down response times. However, it can be useful to better understand how the model is behaving.
Setting to true will return the log probabilities of the tokens that were generated.
Setting to a number will return the log probabilities of the top n tokens that were generated.
-
suffix string
The suffix that comes after a completion of inserted text.
-
user string
A unique identifier representing your end-user, which can help OpenAI to monitor and detect abuse. Learn more.
Embedding Models
You can create models that call the Azure OpenAI embeddings API
using the .embedding() factory method.
Azure OpenAI embedding models support several additional settings. You can pass them as an options argument:
The following optional provider options are available for Azure OpenAI embedding models:
-
dimensions: number
The number of dimensions the resulting output embeddings should have. Only supported in text-embedding-3 and later models.
-
user string
A unique identifier representing your end-user, which can help OpenAI to monitor and detect abuse. Learn more.
Image Models
You can create models that call the Azure OpenAI image generation API using the .image() factory method. The first argument is your deployment name.
MAI-Image models use the MAI image API instead.
Azure image models (including MAI-Image) download remote reference images and
masks using the fetch option of createAzure; provider credentials are not
sent to those hosts. For untrusted URLs on Node.js, pass a guarded-fetch
transport there. See Secure URL
Fetching.
Azure OpenAI image models support several additional settings. You can pass them as providerOptions.openai or providerOptions.azure when generating the image (azure wins when both set the same option):
Example
You can use Azure OpenAI image models to generate images with the generateImage function:
Model Capabilities
The table lists supported DALL-E backing models and Azure MAI image IDs. For OpenAI-backed deployments, the supported sizes depend on the model version behind your deployment.
| Model | Sizes |
|---|---|
dall-e-3 | 1024x1024, 1792x1024, 1024x1792 |
dall-e-2 | 256x256, 512x512, 1024x1024 |
mai-image-2.5 | Sides must be multiples of 16 and at least 768 pixels. |
mai-image-2.5-flash | Sides must be multiples of 16 and at least 768 pixels. |
mai-image-2.5-pro | Sides must be multiples of 16 and at least 768 pixels. |
mai-image-2.6 | Sides must be multiples of 16 and at least 768 pixels. |
mai-image-2.6-flash | Sides must be multiples of 16 and at least 768 pixels. |
mai-image-2.7 | Sides must be multiples of 16 and at least 768 pixels. |
mai-image-2.7-flash | Sides must be multiples of 16 and at least 768 pixels. |
Azure deployment names are user-defined and do not identify the backing model.
Consequently, supportsFileInputs and supportsMaskInputs are undefined
(unknown) for Azure image model instances. If your application knows the
capabilities of a deployment, you can advertise them with the
overrideSupportsFileInputs and overrideSupportsMaskInputs options of
wrapImageModel.
DALL-E models do not support the aspectRatio parameter. Use the size
parameter instead.
When creating your Azure OpenAI deployment, make sure to set the DALL-E model version you want to use.
MAI-Image
MAI-Image models (MAI-Image-2.7, MAI-Image-2.7-Flash, MAI-Image-2.6,
MAI-Image-2.6-Flash, MAI-Image-2.5, MAI-Image-2.5-Flash,
MAI-Image-2.5-Pro) use the MAI image API at
https://{resourceName}.services.ai.azure.com/mai/v1 (override with
maiBaseURL). Deployments named after the model are detected automatically,
case-insensitively. For other deployment names, set
providerOptions.azure.api to 'mai'.
MAI-Image-2.7 models can return an image with a transparent background:
Pass reference images to edit or combine them (up to five JPEG or PNG images). Masks are not supported.
Each request returns one PNG image; generateImage sends one request per image.
size is sent as width and height. Each side must be at least 768 pixels,
and each output is at most 2048x2048 for MAI-Image-2.7 models, 2,359,296 pixels
for MAI-Image-2.6 models, and 1,048,576 pixels for MAI-Image-2.5 models. Without size, aspectRatio is mapped to
dimensions of about one megapixel; without either, the model returns 1024x1024.
seed is not supported. Token usage is returned in usage, with the input
text and image token split in providerMetadata.azure.images.
The following provider options are available:
-
api 'mai' | 'openai'
API to use. Defaults to
'mai'for MAI-Image models,'openai'otherwise. -
autoAspectRatio boolean
Let the model choose the output aspect ratio from the prompt and reference images. MAI-Image-2.6 and MAI-Image-2.7 models only.
-
webGrounding boolean
Ground generation in Bing web search results. MAI-Image-2.6 and MAI-Image-2.7 models only.
-
background 'transparent' | 'opaque'
Image background for generations and edits.
transparentreturns a PNG with an alpha channel; the API defaults toopaque. MAI-Image-2.7 models only. The same option is passed through to OpenAI image deployments, which also accept'auto'; on the MAI image API,'auto'is not sent and produces a warning.
Transcription Models
You can create models that call the Azure OpenAI or Azure Speech transcription API using the .transcription() factory method.
The first argument is the model id e.g. whisper-1.
MAI-Transcribe
MAI-Transcribe models are multilingual speech-to-text models from Microsoft AI, served through Azure. The following model IDs use the Azure Speech API. They are matched case-insensitively and sent to Azure under their Azure model names:
| Model ID | Azure model | Notes |
|---|---|---|
mai-transcribe-2 | MAI-Transcribe-2 | Supports every option below. |
mai-transcribe-1.5 | MAI-Transcribe-1.5 | Supports locales and phraseList. |
mai-transcribe-2-streaming uses the MAI realtime API; see
MAI-Transcribe-2-Streaming. All other IDs use the
OpenAI API by default, including custom deployment names.
MAI-Transcribe models require a Microsoft Foundry (Speech) resource in a region where they are available:
centralindia, eastus, northeurope, southeastasia, westus, or westus2
(see Speech service regions).
Resources in other regions reject the request.
Set AZURE_RESOURCE_NAME and AZURE_API_KEY to that resource's name and key,
or pass resourceName and apiKey to createAzure. tokenProvider, custom headers,
and custom fetch are also supported. Keys are sent as Ocp-Apim-Subscription-Key.
Speech requests use https://{resourceName}.cognitiveservices.azure.com/speechtotext/transcriptions:transcribe
with API version 2025-10-15. Use speechBaseURL to supply a different Speech URL prefix
(without /speechtotext/transcriptions:transcribe), for example a regional endpoint:
Speech requests do not use baseURL, apiVersion, or useDeploymentBasedUrls;
those settings only apply to Azure OpenAI models.
The following options are available through providerOptions.azure:
-
api 'openai' | 'speech' Overrides model-based API selection for this request. Set
openaifor an OpenAI deployment named after one of the model IDs above, orspeechto send another model ID to Speech. Other Speech model IDs are sent unchanged. -
timestamps 'word' | 'segment' | 'none' Timing granularity. For MAI-Transcribe-2 the AI SDK defaults to
segmentso that results include timed segments (Azure's own default isnone).segmentreturns one segment per sentence or speaker turn,wordadditionally returns word timing in provider metadata, andnonereturns a single segment that spans the whole audio. MAI-Transcribe-1.5 only supportsnone, so the AI SDK does not send a default for it. -
transcribeStyle 'verbatim' | 'clean'
verbatimkeeps fillers and false starts,cleanremoves them. Defaults toverbatim.cleanrequires MAI-Transcribe-2. -
locales string[] Exactly one language code, such as
['en']. This is a strong hint; omit it for automatic language detection and code switching. -
diarization object Set
enabledtotrueto identify speakers. Speaker IDs are returned in provider metadata. Requires MAI-Transcribe-2.maxSpeakersis not supported. -
phraseList object Set
phrasesto a list of names and domain terms to bias recognition toward them.
Explicit api takes precedence over the model ID. OpenAI transcription options,
such as language and prompt, can be passed under providerOptions.openai or
providerOptions.azure; azure wins when both set the same option. Speech-only
Azure options produce warnings when used with the OpenAI API, and OpenAI-only
options produce warnings when used with the Speech or MAI APIs. Unknown providerOptions.azure keys
(for example diarization.maxSpeakers) are rejected before the request is sent.
Options that a model does not support, such as timestamps: 'word' with
MAI-Transcribe-1.5, are sent unchanged, and Azure rejects the request with an
error that names the option.
For Azure Speech, the shared timestampGranularities: ['word', 'segment']
setting requests word timestamps, which also include phrase timing.
providerOptions.azure.timestamps takes precedence. Requests for timings that
are disabled by that override or unsupported by the model produce a warning.
MAI-Transcribe-2 exposes normalized word timing in result.words and phrase
timing in result.segments, both in seconds. MAI-Transcribe-1.5 does not support
timestamp requests. Azure OpenAI models use the OpenAI timing settings and
precedence instead.
Results include transcript text, timed segments, duration, and an ISO-639-1
language code when the reported phrase locales identify one language. Mixed-language
results, and languages without an ISO-639-1 code such as Filipino (fil) or
Cantonese (yue), leave language undefined. Speaker IDs, exact locales, and word
timing are preserved in providerMetadata.azure.phrases; use the exported
AzureTranscriptionProviderMetadata type to access them. Phrase timing is in
milliseconds in metadata and seconds in normalized segments.
MAI-Transcribe models are file transcription models: they accept WAV, MP3, or FLAC files shorter than 2 hours and smaller than 250 MB, and do not support streaming transcription; use MAI-Transcribe-2-Streaming to stream. The models are in public preview. See Azure's MAI-Transcribe guide for supported languages and diarization limits.
AI Gateway forwards these provider options, including azure.api.
MAI-Transcribe-2-Streaming
MAI-Transcribe-2-Streaming is a low-latency streaming speech-to-text model from Microsoft AI.
azure.transcription('mai-transcribe-2-streaming') streams audio to the MAI realtime API
(wss://{resourceName}.services.ai.azure.com/mai/v1/realtime) with
experimental_streamTranscribe.
The model ID is sent to Azure as the Foundry deployment name, so deploy the model
as mai-transcribe-2-streaming, or set api: 'mai' for other deployment names.
The model requires a Microsoft Foundry resource with a MAI-Transcribe-2-Streaming
deployment in a supported region, such as centralus.
Input audio must be 16-bit mono PCM (audio/pcm) at 16000 or 24000 Hz; the default is 24000 Hz.
The stream emits transcript-delta parts for finalized text and transcript-partial
parts for the provisional text that follows the latest delta. MAI does not detect
pauses on the server, so the stream commits once when the input audio ends and then
emits a transcript-final part and the finished transcript. durationInSeconds is
computed from the streamed audio.
The following options are available through providerOptions.azure:
-
api 'openai' | 'speech' | 'mai' Set
maito stream with a deployment that has a different name. -
language string Language hint such as
en. Omit for automatic language detection.
File transcription options such as timestamps produce warnings when streaming.
MAI-Transcribe-2-Streaming does not support file transcription.
OpenAI transcription
If you encounter a "DeploymentNotFound" error with transcription models, try enabling deployment-based URLs:
This uses the legacy endpoint format which may be required for certain Azure OpenAI deployments.
When using useDeploymentBasedUrls, the default api-version is not valid. You must set it to 2025-04-01-preview or an earlier value.
You can also pass additional provider-specific options using the providerOptions argument. For example, supplying the input language in ISO-639-1 (e.g. en) format will improve accuracy and latency.
The following provider options are available:
-
timestampGranularities string[] The granularity of the timestamps in the transcription. Defaults to
['segment']. Possible values are['word'],['segment'], and['word', 'segment']. Note: There is no additional latency for segment timestamps, but generating word timestamps incurs additional latency. -
language string The language of the input audio. Supplying the input language in ISO-639-1 format (e.g. 'en') will improve accuracy and latency. Optional.
-
prompt string An optional text to guide the model's style or continue a previous audio segment. The prompt should match the audio language. Optional.
-
temperature number The sampling temperature, between 0 and 1. Higher values like 0.8 will make the output more random, while lower values like 0.2 will make it more focused and deterministic. If set to 0, the model will use log probability to automatically increase the temperature until certain thresholds are hit. Defaults to 0. Optional.
-
include string[] Additional information to include in the transcription response.
Model Capabilities
| Model | Transcription | Streaming | Duration | Segments | Language |
|---|---|---|---|---|---|
whisper-1 | |||||
gpt-4o-mini-transcribe | |||||
gpt-4o-transcribe | |||||
mai-transcribe-2 | |||||
mai-transcribe-1.5 | |||||
mai-transcribe-2-streaming | Conditional: Requires an audio format with a known byte rate. | Conditional: Reports the configured language hint. |
Speech Models
You can create models that call the Azure OpenAI speech API or Azure Speech text to speech using the .speech() factory method.
The first argument is your deployment name for the text-to-speech model (e.g., tts-1),
or a MAI-Voice model ID.
Example
You can also pass additional provider-specific options using the providerOptions argument:
The following provider options are available:
-
instructions string Control the voice of your generated audio with additional instructions e.g. "Speak in a slow and steady tone". Does not work with
tts-1ortts-1-hd. Optional. -
speed number The speed of the generated audio. Select a value from 0.25 to 4.0. Defaults to 1.0. Optional.
Model Capabilities
The table lists OpenAI TTS backing models and Azure MAI voice IDs. For OpenAI-backed deployments, instruction support depends on the model version behind your deployment.
| Model | Instructions |
|---|---|
tts-1 | |
tts-1-hd | |
gpt-4o-mini-tts | |
mai-voice-2.1-flash | |
mai-voice-2.1 | |
mai-voice-2-flash | |
mai-voice-2 |
MAI-Voice
MAI-Voice models are multilingual text-to-speech models from Microsoft AI, served through Azure. The following model IDs use the Azure Speech API:
| Model ID | Notes |
|---|---|
mai-voice-2.1-flash | Low latency, for voice agents and assistants. |
mai-voice-2.1 | Highest fidelity, for long-form narration. |
mai-voice-2-flash | Low latency, for voice agents and assistants. |
mai-voice-2 | Highest fidelity, for long-form narration. |
All other IDs use the OpenAI API by default, including custom deployment names.
MAI-Voice models require a Microsoft Foundry (Speech) resource in a region where they are available
(see Speech service regions).
They use the same settings as MAI-Transcribe: resourceName (or AZURE_RESOURCE_NAME),
apiKey (or AZURE_API_KEY), tokenProvider, and speechBaseURL.
- voice is a prebuilt voice ID that includes its locale, such as
en-US-Harper(the default),de-DE-Mia, ores-MX-Valeria. The voice locale selects the language. Full Azure voice names such ases-MX-Valeria:MAI-Voice-2are also accepted. Available voices vary by model; see Azure's MAI-Voice guide. - language picks a default voice when
voiceis not set (see the table below). Other languages andautouseen-US-Harperwith a warning. Whenvoiceis set, its locale selects the language, and a conflictinglanguageproduces a warning. - outputFormat accepts
mp3(default, 24 kHz),wav,pcm(headerless 24 kHz, 16-bit, mono),opus(Ogg), or any AzureX-Microsoft-OutputFormatvalue such asaudio-48khz-192kbitrate-mono-mp3. Other values fall back tomp3with a warning. Azure labels raw formats such aspcmasaudio/basic. - speed is a rate multiplier, for example
0.5for half speed or2for double speed. instructionsis not supported and produces a warning.
| Language | Default voice | Language | Default voice |
|---|---|---|---|
de | de-DE-Mia | nl | nl-NL-Fleur |
en | en-US-Harper | pt | pt-BR-Luana |
es | es-MX-Valeria | ro | ro-RO-Elena |
fr | fr-FR-Soleil | ru | ru-RU-Masha |
hi | hi-IN-Kavya | th | th-TH-Krit |
hu | hu-HU-Lilla | tr | tr-TR-Elif |
it | it-IT-Rosa | zh | zh-CN-Mei |
ko | ko-KR-Haena |
The following options are available through providerOptions.azure:
-
api 'openai' | 'speech' Overrides model-based API selection for this request. Set
openaifor an OpenAI deployment named after one of the model IDs above, orspeechto send another model ID to Speech. -
style string Speaking style, such as
excitedorwhispering. Supported styles vary by voice and model. -
styleDegree number Style intensity from 0.01 to 2. Azure defaults to 1. Requires
style.
OpenAI speech options (instructions and speed) can be passed under
providerOptions.openai or providerOptions.azure; azure wins when both set
the same option. They produce warnings when used with the Speech API.
Text is always spoken as plain text; SSML input is not supported.
The models are in public preview. AI Gateway forwards these provider options, including azure.api.
Provider Option Reference
The following option definitions are generated from the provider catalog.
| Operation | Option | Description |
|---|---|---|
| image | api | Select the OpenAI or MAI image API. |
| image | autoAspectRatio | Choose the aspect ratio automatically. |
| image | webGrounding | Ground image generation in web search. |
| image | background | Transparent or opaque image background. |
| speech | api | Select the OpenAI or Azure Speech API. |
| speech | style | Speaking style supported by the selected voice. |
| speech | styleDegree | Intensity of the speaking style. |
| transcription | api | Select the OpenAI, Azure Speech, or MAI API. |
| transcription | language | Language hint for MAI streaming transcription. |
| transcription | timestamps | Word, segment, or no timestamps. |
| transcription | transcribeStyle | Verbatim or cleaned transcription. |
| transcription | locales | Language to use for transcription. |
| transcription | diarization | Enable speaker diarization. |
| transcription | phraseList | Phrases to bias transcription toward. |