Skip to content

feat(tts): add Inworld as built-in speech provider#55972

Merged
steipete merged 9 commits into
openclaw:mainfrom
cshape:feat/inworld-tts
Apr 25, 2026
Merged

feat(tts): add Inworld as built-in speech provider#55972
steipete merged 9 commits into
openclaw:mainfrom
cshape:feat/inworld-tts

Conversation

@cshape

@cshape cshape commented Mar 27, 2026

Copy link
Copy Markdown
Contributor

Summary

Add Inworld as a built-in OpenClaw speech provider plugin, alongside ElevenLabs, Google Gemini, Gradium, Local CLI, Microsoft, MiniMax, OpenAI, Vydra, xAI, and Xiaomi MiMo. Inworld is a streaming TTS API; this PR wires it into messages.tts with MP3, OGG_OPUS (voice notes), and PCM (telephony) outputs.

New files

  • extensions/inworld/speech-provider.ts — speech-provider plugin entry, including audio format negotiation (MP3 / OGG_OPUS / PCM) and voice listing.
  • extensions/inworld/tts.ts — Inworld API client: streaming voice:stream (newline-delimited JSON, base64 audio chunks) and voices/v1/voices listing. All HTTP requests routed through fetchWithSsrFGuard.
  • extensions/inworld/speech-provider.test.ts, extensions/inworld/tts.test.ts — 22 unit tests covering streaming chunk parsing, error paths, voice mapping, telephony PCM, and request body shape.
  • extensions/inworld/index.ts — plugin entry point.
  • extensions/inworld/package.json, extensions/inworld/tsconfig.json — workspace package metadata.
  • extensions/inworld/openclaw.plugin.json — plugin manifest with name, description, providerAuthEnvVars: { inworld: ["INWORLD_API_KEY"] }, contracts.speechProviders, and a real configSchema for apiKey, baseUrl, voiceId, modelId, and temperature (0..2 bounded).
  • docs/providers/inworld.md — provider page modeled on docs/providers/deepgram.md: setup, config table, models, audio outputs, custom endpoints, and the Base64 / HTTP Basic auth callout. Wired into docs/docs.json Providers nav.

Modified files

  • .env.exampleINWORLD_API_KEY placeholder.
  • .github/labeler.ymlextensions: inworld label rule covering extensions/inworld/** and docs/providers/inworld.md (the matching GitHub label needs to be created repo-side, color #0E8A16 to match other extensions: * labels).
  • CHANGELOG.md — new Inworld TTS entry under the Unreleased ## Changes section.
  • docs/tools/tts.md — Inworld added to provider list, optional-keys section, primary config example, and field reference; auth note explicitly states the apiKey must be the Base64 dashboard credential sent verbatim as Authorization: Basic <apiKey> (no bearer-style normalization, no extra encoding).
  • docs/.generated/config-baseline.sha256 — regenerated for the manifest configSchema additions.

Inworld API details

  • TTS endpoint: POST /tts/v1/voice:stream — newline-delimited JSON, each line carries base64 audio in result.audioContent.
  • Voices endpoint: GET /voices/v1/voices (current Voices API; old /tts/v1/voices is deprecated).
  • Auth: Authorization: Basic <apiKey> where apiKey is the Base64-encoded credential string copied verbatim from the Inworld dashboard. Plugin sends it without re-encoding; users should not pass a raw bearer token. Fallback resolution: messages.tts.providers.inworld.apiKeyINWORLD_API_KEY env.
  • Default voice: Sarah. Default model: inworld-tts-1.5-max.
  • Supported models: inworld-tts-1.5-max, inworld-tts-1.5-mini, inworld-tts-1-max, inworld-tts-1.
  • Audio formats: MP3 default; OGG_OPUS for target: "voice-note"; raw PCM at 22050 Hz for telephony.
  • Config knobs: apiKey, baseUrl, voiceId, modelId, temperature (0..2).

Config example

{
  "messages": {
    "tts": {
      "auto": "always",
      "provider": "inworld",
      "providers": {
        "inworld": {
          "voiceId": "Sarah",
          "modelId": "inworld-tts-1.5-max"
        }
      }
    }
  }
}

Set INWORLD_API_KEY in env (Base64 credential from the Inworld dashboard, used as HTTP Basic).

Notes for reviewers

  • Imports use the post-extraction openclaw/plugin-sdk/speech-core subpath, matching the merged Gemini TTS template (feat(google): add Gemini TTS provider #67515).
  • This is a speech-only plugin, so the manifest mirrors the Gradium/ElevenLabs precedent (no providerAuthChoices / uiHints — those are model-provider onboarding fields).

Test plan

  • pnpm test extensions/inworld — 22/22 pass
  • pnpm test:bundled — pass (bundled plugin invariants)
  • pnpm config:docs:check — pass after manifest configSchema regen
  • pnpm build — pass
  • Local Telegram live test — Inworld TTS reply with voiceId: "Sarah" round-trips correctly

@cshape
cshape requested a review from a team as a code owner March 27, 2026 18:29
@openclaw-barnacle openclaw-barnacle Bot added docs Improvements or additions to documentation docker Docker and sandbox tooling extensions: openai size: L labels Mar 27, 2026
@greptile-apps

greptile-apps Bot commented Mar 27, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR adds Inworld AI as a fourth built-in TTS provider alongside OpenAI, ElevenLabs, and Microsoft. The integration follows the existing provider plugin pattern well: a new extensions/inworld/ package implements SpeechProviderPlugin with streaming synthesis, voice listing, and telephony support; the core src/tts/tts.ts wires in auto-detection and config resolution; and all the surrounding plumbing (Zod schema, secrets collection, /tts command status, generated baselines) is updated consistently.

Key findings:

  • Temperature schema off-by-one (gt(0) → should be gte(0)): The types.tts.ts JSDoc documents the valid range as 0–2, but the Zod schema uses .gt(0), silently rejecting the minimum documented value. Users who set temperature: 0 for deterministic synthesis will hit a schema error.
  • pull_policy: never in docker-compose.yml: This unrelated change, applied to both openclaw-gateway and openclaw-cli, prevents Docker from pulling pre-built images and will break docker-compose up for users without locally-built images.
  • Unguarded JSON.parse in the NDJSON streaming loop: A non-JSON line from the API would produce an opaque SyntaxError instead of a descriptive Inworld-specific message.

Confidence Score: 4/5

Safe to merge after addressing the temperature schema bug and reverting the unrelated pull_policy: never docker-compose change.

Two P1 findings remain: a schema validation bug that rejects a documented valid input value, and an unrelated docker-compose change that breaks image-pull deployments. The core implementation is otherwise solid and well-tested.

src/config/zod-schema.core.ts (temperature range) and docker-compose.yml (pull_policy: never).

Important Files Changed

Filename Overview
extensions/inworld/speech-provider.ts Core provider implementation — streaming NDJSON parser, voice listing, telephony support. Logic is correct; minor robustness gap with unguarded JSON.parse.
src/config/zod-schema.core.ts Inworld Zod schema added; temperature validation uses gt(0) instead of gte(0), contradicting the documented 0–2 range.
docker-compose.yml Adds INWORLD_API_KEY env passthrough (correct), but also adds pull_policy: never to both services — an unrelated change that breaks image-pull deployments.
src/tts/tts.ts Inworld wired into resolveTtsConfig, resolveTtsApiKey, getTtsProvider auto-detection, and TTS_PROVIDERS — all consistent with existing provider patterns.
extensions/inworld/speech-provider.test.ts 17 unit tests covering synthesis, voice listing, error handling, format selection, and provider metadata — good coverage.

Comments Outside Diff (1)

  1. docker-compose.yml, line 31 (link)

    P1 pull_policy: never breaks image-pull deployments

    Adding pull_policy: never to both openclaw-gateway (line 31) and openclaw-cli (line 48) means Docker will never attempt to pull a pre-built image from a registry. Any user who does not already have the image locally and runs docker-compose up will get an immediate "unable to find image ... locally" error instead of pulling the image.

    This appears to be a local-development convenience setting that leaked into the shared docker-compose.yml. It is also unrelated to the Inworld TTS feature and should either be reverted or kept only in a local override file (docker-compose.override.yml).

    Prompt To Fix With AI
    This is a comment left during a code review.
    Path: docker-compose.yml
    Line: 31
    
    Comment:
    **`pull_policy: never` breaks image-pull deployments**
    
    Adding `pull_policy: never` to both `openclaw-gateway` (line 31) and `openclaw-cli` (line 48) means Docker will never attempt to pull a pre-built image from a registry. Any user who does not already have the image locally and runs `docker-compose up` will get an immediate `"unable to find image ... locally"` error instead of pulling the image.
    
    This appears to be a local-development convenience setting that leaked into the shared `docker-compose.yml`. It is also unrelated to the Inworld TTS feature and should either be reverted or kept only in a local override file (`docker-compose.override.yml`).
    
    How can I resolve this? If you propose a fix, please make it concise.
Prompt To Fix All With AI
This is a comment left during a code review.
Path: src/config/zod-schema.core.ts
Line: 455

Comment:
**Temperature validation rejects documented minimum value**

The Zod schema uses `.gt(0)` (strictly greater than 0), but the JSDoc comment in `src/config/types.tts.ts` defines the valid range as `"0–2"`, meaning `temperature: 0` is a documented and plausible value (e.g. for deterministic, low-variance synthesis). Any user who tries to set `temperature: 0` will get a schema validation error, silently contradicting the documented range.

```suggestion
        temperature: z.number().gte(0).lte(2).optional(),
```

How can I resolve this? If you propose a fix, please make it concise.

---

This is a comment left during a code review.
Path: docker-compose.yml
Line: 31

Comment:
**`pull_policy: never` breaks image-pull deployments**

Adding `pull_policy: never` to both `openclaw-gateway` (line 31) and `openclaw-cli` (line 48) means Docker will never attempt to pull a pre-built image from a registry. Any user who does not already have the image locally and runs `docker-compose up` will get an immediate `"unable to find image ... locally"` error instead of pulling the image.

This appears to be a local-development convenience setting that leaked into the shared `docker-compose.yml`. It is also unrelated to the Inworld TTS feature and should either be reverted or kept only in a local override file (`docker-compose.override.yml`).

How can I resolve this? If you propose a fix, please make it concise.

---

This is a comment left during a code review.
Path: extensions/inworld/speech-provider.ts
Line: 88-94

Comment:
**Unguarded `JSON.parse` in streaming loop**

If the Inworld API returns a line that is not valid JSON (e.g., a rate-limit HTML error page that slips through after a `200` status, or a partial flush), `JSON.parse(trimmed)` will throw a raw `SyntaxError`. The error will propagate out of `inworldTTS`, be caught by the provider-level try/catch in `synthesizeSpeech`, and surface as a cryptic `"Unexpected token …"` message rather than something actionable.

Wrapping the parse in a try/catch with a descriptive re-throw would make debugging much easier:

```ts
let parsed: { result?: { audioContent?: string }; error?: { code?: number; message?: string } };
try {
  parsed = JSON.parse(trimmed) as typeof parsed;
} catch {
  throw new Error(`Inworld TTS stream parse error: unexpected non-JSON line: ${trimmed.slice(0, 80)}`);
}
```

How can I resolve this? If you propose a fix, please make it concise.

Reviews (1): Last reviewed commit: "feat(tts): add Inworld as built-in speec..." | Re-trigger Greptile

Comment thread src/config/zod-schema.core.ts Outdated
Comment thread extensions/inworld/speech-provider.ts Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 56464e254c

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread src/secrets/target-registry-data.ts Outdated
Comment thread src/config/zod-schema.core.ts Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 67307c31d6

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread src/secrets/target-registry-data.ts Outdated
@cshape
cshape force-pushed the feat/inworld-tts branch from c162c63 to 2fe6fa5 Compare April 17, 2026 01:02
@openclaw-barnacle openclaw-barnacle Bot removed the docker Docker and sandbox tooling label Apr 17, 2026

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 49374a424a

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread extensions/inworld/tts.ts
@steipete

Copy link
Copy Markdown
Contributor

Deep review against current main (5163a2f): this is a real missing provider, but the PR still needs work before land.

Current state:

  • main has no Inworld plugin/provider (rg "inworld|INWORLD" returns nothing).
  • .profile does not currently expose INWORLD_API_KEY, so I could not run a live synthesis probe here.
  • Official Inworld docs still match the basic API shape used here: POST https://api.inworld.ai/tts/v1/voice:stream, Authorization: Basic $INWORLD_API_KEY, newline JSON chunks with result.audioContent.

Good changes since the earlier bot review:

  • The current diff no longer shows the unrelated docker-compose.yml pull_policy: never change.
  • The stream parser now wraps JSON.parse and reports a provider-specific parse error.
  • This includes docs, changelog, provider tests, and uses the current plugin speech-provider contract.

Remaining blockers:

  1. CI is still red on this head: checks-node-agentic-control-plane and aggregate checks-node-core failed. That has to be green before land.
  2. New bundled plugin surface needs repo plumbing check: add/update labeler coverage for extensions/inworld/** if missing, and make sure bundled plugin metadata generation/checks are current.
  3. Add a small live/contract note or test fixture for the exact Inworld auth expectation. The docs say Authorization: Basic $INWORLD_API_KEY; users often interpret API keys as raw bearer-style tokens, so docs should state the value must be the Inworld Basic credential exactly as copied/generated.
  4. Consider a provider-specific docs page (docs/providers/inworld.md) or at least link the Inworld config from provider docs. docs/tools/tts.md is useful, but new first-class providers usually deserve a provider page/discoverable entry.

So: valid provider, not solved on main, and the implementation is close, but do not merge with current red CI.

@cshape
cshape force-pushed the feat/inworld-tts branch 2 times, most recently from f3e9c11 to ff27a84 Compare April 25, 2026 09:25
@cshape

cshape commented Apr 25, 2026

Copy link
Copy Markdown
Contributor Author

@steipete Thanks for the detailed review. I've pushed updates addressing all four blockers:

  1. CI — rebased the branch onto current origin/main (was 3,600+ commits behind, which is what was driving the conflicts and likely some of the red checks). The earlier checks-node-agentic-control-plane / checks-node-core failures should now be re-evaluated against current main. Locally I'm green on pnpm test extensions/inworld (22/22), pnpm test:bundled, pnpm config:docs:check, and pnpm build. The remaining pnpm tsgo failures I see locally are all in unrelated files (src/gateway/server-methods/*, src/plugin-sdk/provider-tools.ts, ui/src/ui/*) — none mention Inworld and they reproduce on stock origin/main.

  2. Repo plumbing — added extensions: inworld to .github/labeler.yml (covers extensions/inworld/** and docs/providers/inworld.md). Heads up: the matching GitHub label needs to be created on the repo side before the workflow can apply it — would you mind adding extensions: inworld with the existing extensions: * color (#0E8A16)? Also enriched extensions/inworld/openclaw.plugin.json with name, description, providerAuthEnvVars, and a real configSchema (apiKey/baseUrl/voiceId/modelId/temperature with descriptions and bounds), and regenerated docs/.generated/config-baseline.sha256 to match.

  3. Auth docs — strengthened the callout in docs/tools/tts.md and added a matching inline comment in extensions/inworld/tts.ts near the Authorization header. Both make it explicit that apiKey must be the Base64 credential from the Inworld dashboard, sent verbatim as Authorization: Basic <apiKey>, and to not pass a raw bearer token or Base64-encode it again.

  4. Provider docs page — added docs/providers/inworld.md (modeled on docs/providers/deepgram.md) and wired it into the Providers nav in docs/docs.json alphabetically.

One incidental: while rebasing, I also realigned the SDK imports from openclaw/plugin-sdk/speech to openclaw/plugin-sdk/speech-core to match the merged Gemini TTS template (#67515) — same exports, just the post-extraction canonical path.

@cshape

cshape commented Apr 25, 2026

Copy link
Copy Markdown
Contributor Author

@obviyus I saw you merged in a Gemini TTS PR (#67515) recently.

If you could take a look at this PR for Inworld TTS (currently #1 on Artificial Analysis TTS arena) that would be much appreciated! Thanks!

cshape added 4 commits April 25, 2026 22:22
Registers a new bundled speech plugin at extensions/inworld/ that follows the
openclaw/plugin-sdk/* contract: the plugin entry registers an Inworld
SpeechProviderPlugin via api.registerSpeechProvider, and the manifest declares
the speechProviders contract with enabledByDefault=true so core picks it up
through normal plugin discovery.

Provider covers MP3 output, OGG_OPUS voice-notes, PCM telephony, listVoices,
and API-key resolution via messages.tts.providers.inworld.apiKey or
INWORLD_API_KEY. Uses the shared openclaw/plugin-sdk/speech coerce helpers to
stay consistent with the OpenAI and ElevenLabs bundled providers.

Docs, .env.example, and the Unreleased changelog are updated; no core
src/tts, src/config, or src/secrets edits are needed under the current
plugin-owned pattern.
CI lint:tmp:no-raw-channel-fetch flags new raw fetch() callsites in
extension runtime code. Replace the Inworld streaming-TTS and voices
calls with fetchWithSsrFGuard from openclaw/plugin-sdk/ssrf-runtime, so
SSRF policy, dispatcher pinning, and proxy-capture run through the
shared guard instead of a plugin-local AbortController.

Tests now mock the ssrf-runtime module export instead of globalThis.fetch
and assert the guarded request shape + dispatcher release for both
success and failure paths.
- Switch extensions/inworld imports from openclaw/plugin-sdk/speech to
  openclaw/plugin-sdk/speech-core to match the merged Gemini TTS template.
- Enrich extensions/inworld/openclaw.plugin.json with name, description,
  providerAuthEnvVars (INWORLD_API_KEY), and a real configSchema so the
  control-plane can validate and surface field-level help.
- Add the matching .github/labeler.yml entry for extensions: inworld.
- Document the HTTP Basic / Base64-credential auth expectation in
  docs/tools/tts.md and inline near the Authorization header in tts.ts.
- Add docs/providers/inworld.md provider page and wire it into the
  Providers nav in docs/docs.json.
- Regenerate docs/.generated/config-baseline.sha256 for the manifest
  configSchema change.
Update DEFAULT_INWORLD_VOICE_ID, the corresponding default-voice
test expectation, manifest configSchema description, and all docs
(messages.tts.providers.inworld example, the inworld provider page,
and the field reference table) to point at Sarah by default.
@openclaw-barnacle openclaw-barnacle Bot added the cli CLI command changes label Apr 25, 2026
@steipete
steipete merged commit 0bcb4c9 into openclaw:main Apr 25, 2026
65 checks passed
@steipete

Copy link
Copy Markdown
Contributor

Landed in 0bcb4c9.

Maintainer proof:

  • CI green on source SHA e2fbe6a.
  • Local targeted tests: pnpm test src/plugin-sdk/facade-loader.test.ts src/plugins/setup-registry.test.ts src/plugins/doctor-contract-registry.test.ts src/channels/plugins/module-loader.test.ts src/cli/program/preaction.test.ts src/infra/diagnostic-events.test.ts extensions/inworld.
  • Local typecheck: pnpm tsgo:all.
  • Live .profile Inworld check: OPENCLAW_LIVE_TEST_QUIET=0 pnpm test:live extensions/inworld/inworld.live.test.ts.

Also verified no open Inworld duplicate issues/PRs remain.

ayesha-aziz123 pushed a commit to ayesha-aziz123/openclaw that referenced this pull request Apr 26, 2026
Adds the bundled Inworld speech provider with docs, config surface, SSRF-guarded fetches, directive overrides, native voice-note/telephony output coverage, and live `.profile` verification.

Co-authored-by: cshape <[email protected]>
ogt-redknie pushed a commit to ogt-redknie/OPENX that referenced this pull request May 2, 2026
Adds the bundled Inworld speech provider with docs, config surface, SSRF-guarded fetches, directive overrides, native voice-note/telephony output coverage, and live `.profile` verification.

Co-authored-by: cshape <[email protected]>
github-actions Bot pushed a commit to Desicool/openclaw that referenced this pull request May 9, 2026
Adds the bundled Inworld speech provider with docs, config surface, SSRF-guarded fetches, directive overrides, native voice-note/telephony output coverage, and live `.profile` verification.

Co-authored-by: cshape <[email protected]>
globalcaos pushed a commit to globalcaos/tinkerclaw that referenced this pull request May 13, 2026
Adds the bundled Inworld speech provider with docs, config surface, SSRF-guarded fetches, directive overrides, native voice-note/telephony output coverage, and live `.profile` verification.

Co-authored-by: cshape <[email protected]>
github-actions Bot pushed a commit to Desicool/openclaw that referenced this pull request May 24, 2026
Adds the bundled Inworld speech provider with docs, config surface, SSRF-guarded fetches, directive overrides, native voice-note/telephony output coverage, and live `.profile` verification.

Co-authored-by: cshape <[email protected]>
jameslcowan pushed a commit to jameslcowan/openclaw that referenced this pull request Jun 2, 2026
Adds the bundled Inworld speech provider with docs, config surface, SSRF-guarded fetches, directive overrides, native voice-note/telephony output coverage, and live `.profile` verification.

Co-authored-by: cshape <[email protected]>
sablehead pushed a commit to sablehead/openclaw that referenced this pull request Jun 10, 2026
Adds the bundled Inworld speech provider with docs, config surface, SSRF-guarded fetches, directive overrides, native voice-note/telephony output coverage, and live `.profile` verification.

Co-authored-by: cshape <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cli CLI command changes docs Improvements or additions to documentation extensions: tts-local-cli size: XL

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants