Skip to content

feat(tts): add Volcengine (ByteDance) speech provider#55641

Merged
steipete merged 1 commit into
openclaw:mainfrom
xuruiray:feat/volcengine-tts-provider
Apr 25, 2026
Merged

feat(tts): add Volcengine (ByteDance) speech provider#55641
steipete merged 1 commit into
openclaw:mainfrom
xuruiray:feat/volcengine-tts-provider

Conversation

@xuruiray

@xuruiray xuruiray commented Mar 27, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Add Volcengine (ByteDance/豆包) as a new TTS speech provider, integrated into the existing volcengine plugin alongside the LLM provider
  • Support Chinese voice synthesis via the ByteDance Speech API (openspeech.bytedance.com/api/v1/tts)
  • Add core type updates (TtsConfig, ResolvedTtsConfig, zod schema) for messages.tts.volcengine configuration

Details

New files:

  • extensions/volcengine/tts.ts — raw Volcengine TTS API call (HTTPS, returns base64-decoded audio buffer)
  • extensions/volcengine/speech-provider.tsSpeechProviderPlugin implementation
  • extensions/volcengine/tts.test.ts — unit tests for provider configuration and voice listing

Modified files:

  • extensions/volcengine/index.ts — register speech provider alongside existing LLM provider
  • extensions/volcengine/openclaw.plugin.json — declare speechProviders contract and TTS env vars
  • src/config/types.tts.ts — add volcengine section to TtsConfig
  • src/config/zod-schema.core.ts — add volcengine zod validation
  • src/tts/tts.ts — add volcengine section to ResolvedTtsConfig and resolveTtsConfig()

Features:

  • OGG Opus output for voice notes (Feishu/Telegram/WhatsApp compatible), MP3 for audio files
  • Auth via messages.tts.volcengine.appId/token config or VOLCENGINE_TTS_APPID/VOLCENGINE_TTS_TOKEN env vars
  • 6 built-in Chinese voices (default: zh_female_xiaohe_uranus_bigtts)
  • Speed ratio (0.2-3.0) and emotion control
  • Provider aliases: bytedance, doubao

Example config:

{
  messages: {
    tts: {
      auto: "always",
      provider: "volcengine",
      volcengine: {
        appId: "your-app-id",
        token: "your-token",
        voice: "zh_female_xiaohe_uranus_bigtts",
        speedRatio: 0.9
      }
    }
  }
}

Test plan

  • Unit tests for isConfigured (config vs env var fallback)
  • Unit tests for listVoices (locale, gender)
  • Manual test with Volcengine credentials on Feishu channel
  • Verify OGG Opus output plays as native voice message

@greptile-apps

greptile-apps Bot commented Mar 27, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR adds Volcengine (ByteDance/豆包) as a TTS speech provider, integrating it into the existing volcengine plugin alongside the LLM provider. The implementation is well-structured, follows existing provider conventions, and includes type definitions, Zod schema validation, and unit tests.

Key changes:

  • New extensions/volcengine/tts.ts — raw HTTPS client for the ByteDance Speech API, returning base64-decoded audio buffers
  • New extensions/volcengine/speech-provider.tsSpeechProviderPlugin implementation with 6 built-in Chinese voices, OGG Opus/MP3 encoding, and credential resolution from config or env vars
  • Core type updates in src/config/types.tts.ts, src/config/zod-schema.core.ts, and src/tts/tts.ts

Issues found:

  • P1 — Missing res.on("error") handler in tts.ts: Response-stream errors are not caught. If the IncomingMessage emits "error" (e.g. a mid-stream TCP reset), the Node.js process receives an uncaught exception and the returned Promise hangs forever since neither resolve nor reject is called.
  • P2 — req referenced in timer callback before const declaration: Technically safe at runtime but unconventional and may trigger linter warnings.
  • P2 — Unused imports in tts.test.ts: vi, beforeEach, and afterEach are imported but never used.

Confidence Score: 4/5

Safe to merge after fixing the missing response error handler, which can cause unhandled exceptions on network failures.

The overall implementation is clean, well-typed, and consistent with the existing codebase. The only actionable bug is the missing res.on('error') listener in the HTTP client, which is a reliability gap on network fault paths. The remaining comments are style-level. One targeted fix resolves the primary concern.

extensions/volcengine/tts.ts — needs the response stream error handler before merge.

Important Files Changed

Filename Overview
extensions/volcengine/tts.ts Core TTS HTTP client — missing res.on("error") handler can cause uncaught exceptions or hung promises on response-stream failures; req referenced in timer closure before declaration.
extensions/volcengine/speech-provider.ts Clean SpeechProviderPlugin implementation; credential resolution, encoding selection, and voice listing look correct.
extensions/volcengine/tts.test.ts Good unit test coverage for isConfigured and listVoices; three unused imports (vi, beforeEach, afterEach) should be removed.
extensions/volcengine/index.ts Adds registerSpeechProvider call alongside the existing LLM registration — correct and minimal.
extensions/volcengine/openclaw.plugin.json Declares speechProviders contract and new TTS env vars correctly.
src/config/types.tts.ts Adds optional volcengine block to TtsConfig with appropriate types including SecretInput for the token.
src/config/zod-schema.core.ts Adds correct Zod schema for volcengine TTS config with sensitive registration on the token and range validation on speedRatio.
src/tts/tts.ts Adds volcengine to ResolvedTtsConfig and resolveTtsConfig() with sensible defaults for voice and cluster.
Prompt To Fix All With AI
This is a comment left during a code review.
Path: extensions/volcengine/tts.ts
Line: 73-91

Comment:
**Missing `res.on("error")` handler leaks promise**

The response stream (`res`) has no `"error"` listener. In Node.js, if an `IncomingMessage` stream emits `"error"` (e.g. due to a mid-stream connection reset), the event has no handler, which causes an uncaught exception in the process. The outer `req.on("error")` only covers request-side errors, not response-side stream errors. The promise would also hang indefinitely because neither `resolve` nor `reject` would be called.

Add an `"error"` listener on `res`:

```suggestion
      (res) => {
        const chunks: Buffer[] = [];
        res.on("data", (chunk: Buffer) => chunks.push(chunk));
        res.on("error", (e) => {
          clearTimeout(timer);
          reject(new Error(`Volcengine TTS response error: ${e.message}`));
        });
        res.on("end", () => {
          clearTimeout(timer);
          try {
            const body = JSON.parse(Buffer.concat(chunks).toString());
            if (body.code === 3000 && body.data) {
              resolve(Buffer.from(body.data, "base64"));
            } else {
              reject(
                new Error(`Volcengine TTS error ${body.code}: ${body.message ?? "unknown"}`),
              );
            }
          } catch (e) {
            reject(new Error(`Volcengine TTS: failed to parse response: ${e}`));
          }
        });
      },
```

How can I resolve this? If you propose a fix, please make it concise.

---

This is a comment left during a code review.
Path: extensions/volcengine/tts.ts
Line: 57-61

Comment:
**`req` captured in timer closure before declaration**

`req` is referenced inside the `setTimeout` callback on line 58, but it's only declared with `const` on line 62. This works at runtime because `setTimeout` is asynchronous and `req` will always be set before the timer fires. However, it creates a temporal dead zone reference that some linters (e.g. `no-use-before-define`) may flag, and it's easy to misread.

Consider swapping the ordering — assign `req` first, then create the timer:

```typescript
const req = https.request({ ... }, (res) => { ... });
const timer = setTimeout(() => {
  req.destroy();
  reject(new Error(`Volcengine TTS timeout after ${timeoutMs}ms`));
}, timeoutMs);
req.write(payload);
req.end();
```

How can I resolve this? If you propose a fix, please make it concise.

---

This is a comment left during a code review.
Path: extensions/volcengine/tts.test.ts
Line: 1

Comment:
**Unused imports: `vi`, `beforeEach`, `afterEach`**

`vi`, `beforeEach`, and `afterEach` are imported from `vitest` but never used in the test file. These can be removed to avoid linting warnings.

```suggestion
import { describe, it, expect } from "vitest";
```

How can I resolve this? If you propose a fix, please make it concise.

Reviews (1): Last reviewed commit: "feat(tts): add Volcengine (ByteDance) sp..." | Re-trigger Greptile

Comment thread extensions/volcengine/tts.ts Outdated
Comment thread extensions/volcengine/tts.ts Outdated
Comment thread extensions/volcengine/tts.test.ts Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e366e78233

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread extensions/volcengine/tts.ts Outdated
Comment thread extensions/volcengine/openclaw.plugin.json Outdated
Comment thread src/config/zod-schema.core.ts Outdated
@xuruiray
xuruiray force-pushed the feat/volcengine-tts-provider branch from e366e78 to f75f77d Compare March 27, 2026 08:07

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f75f77d76f

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread src/config/types.tts.ts Outdated
Comment thread extensions/volcengine/openclaw.plugin.json Outdated
@xuruiray
xuruiray force-pushed the feat/volcengine-tts-provider branch from f75f77d to 4e2b54f Compare March 27, 2026 08:13

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d8cb91c9f2

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread extensions/volcengine/speech-provider.ts Outdated
@xuruiray
xuruiray force-pushed the feat/volcengine-tts-provider branch from d8cb91c to e99babc Compare March 27, 2026 08:27

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ca1dfe0a2f

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread extensions/volcengine/speech-provider.ts
@xuruiray
xuruiray force-pushed the feat/volcengine-tts-provider branch from ca1dfe0 to 84d00d5 Compare March 27, 2026 08:38

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b810a7262d

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread extensions/volcengine/speech-provider.ts Outdated
@xuruiray
xuruiray force-pushed the feat/volcengine-tts-provider branch from b810a72 to 720c60e Compare March 27, 2026 08:53

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 720c60e36f

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread extensions/volcengine/speech-provider.ts Outdated
@xuruiray
xuruiray force-pushed the feat/volcengine-tts-provider branch from 720c60e to 9f85d9f Compare March 27, 2026 09:09
@xuruiray

Copy link
Copy Markdown
Contributor Author

Note on voice catalog: The current listVoices implementation returns 6 hardcoded popular Chinese voices. Volcengine actually offers 100+ voices across multiple languages and styles (see voice catalog). Happy to add dynamic voice listing via the Volcengine API in a follow-up if that would be useful.

@xuruiray

Copy link
Copy Markdown
Contributor Author

@steipete Would love to get your review on this when you have a moment. This adds Volcengine (ByteDance) as a TTS speech provider — it's the major speech API in China and provides high-quality Chinese neural voices. We're using it in production with OpenClaw on Feishu (Lark) and it works well for native voice message delivery. All CI checks are passing. Thanks!

@steipete
steipete force-pushed the feat/volcengine-tts-provider branch from 7f1ec14 to 6eab06e Compare April 25, 2026 22:17
@steipete

Copy link
Copy Markdown
Contributor

Rebased/reworked this PR onto current main and pushed maintainer edits to the PR branch.

What changed:

  • Ported the Volcengine speech provider onto the current plugin/manifest shape.
  • Swapped the raw https client for the repo SSRF-guarded fetch helper.
  • Added payload/auth/error coverage for the Volcengine TTS request shape, including native ogg_opus voice-note output.
  • Added a gated live test at extensions/volcengine/tts.live.test.ts.
  • Updated TTS/provider docs, i18n glossary, and changelog.

Local proof on pushed head 6eab06e3aa:

  • pnpm test extensions/volcengine/tts.test.ts
  • OPENCLAW_LIVE_TEST=1 pnpm test:live extensions/volcengine/tts.live.test.ts (skips cleanly without credentials)
  • pnpm check:changed
  • pnpm check:docs
  • pnpm build

I still need a matching Volcengine Speech AppID + Access Token to do the real live API call. The ModelArk/OpenAI-compatible key is not enough for this endpoint. I tried the AppID Peter had available with the local token candidates and Volcengine returned 401 / load grant: requested grant not found in SaaS storage, which means the AppID/token pair is not valid for Speech TTS or the TTS grant is not enabled.

Please provide a limited test VOLCENGINE_TTS_APPID and VOLCENGINE_TTS_TOKEN pair, or confirm a Speech Console app/token that can synthesize TTS, and I can run the live test before landing.

@openclaw-barnacle openclaw-barnacle Bot added docs Improvements or additions to documentation extensions: tts-local-cli labels Apr 25, 2026
@steipete

Copy link
Copy Markdown
Contributor

Maintainer follow-up pushed in 0be34792b1.

What changed:

  • Reworked TTS auth to support the current BytePlus Seed Speech API-key flow (VOLCENGINE_TTS_API_KEY / BYTEPLUS_SEED_SPEECH_API_KEY) via https://voice.ap-southeast-1.bytepluses.com/api/v3/tts/unidirectional.
  • Kept the legacy Volcengine AppID/token path for older Speech Console applications.
  • Made resourceId, appKey, baseUrl, and voice configurable.
  • Updated the default resource to seed-tts-1.0: the default BytePlus project key live-tested here is valid, but seed-tts-2.0 returns requested resource not granted unless the project has TTS 2.0 entitlement.
  • Removed generic BYTEPLUS_API_KEY fallback from TTS detection because it can be a ModelArk key and caused a false-positive auth path here.

Validation:

  • pnpm test extensions/volcengine/tts.test.ts
  • OPENCLAW_LIVE_TEST=1 pnpm test:live extensions/volcengine/tts.live.test.ts with .profile Seed Speech key; returned real MP3 bytes (ID3 header)
  • pnpm check:docs
  • pnpm check:changed
  • pnpm build

Dependency/source check: verified against BytePlus Seed Speech unidirectional HTTP docs and voice list; the live API behavior matches the API-key header flow and current default-project entitlement.

Add Volcengine/BytePlus Seed Speech as a bundled TTS provider with current API-key auth, legacy AppID/token fallback, native Ogg/Opus voice-note output, and MP3 audio-file output.

Co-authored-by: Peter Steinberger <[email protected]>
@steipete
steipete force-pushed the feat/volcengine-tts-provider branch from 0be3479 to 115f0cf Compare April 25, 2026 22:35
@steipete

Copy link
Copy Markdown
Contributor

Rewrote this PR branch history to one clean commit: 115f0cf794 feat(tts): add BytePlus Seed Speech provider.

The tree is identical to the previously validated head (0be34792b1); this was history cleanup only. The squashed commit keeps Rui as author and Peter as co-author.

@steipete
steipete merged commit 1531123 into openclaw:main Apr 25, 2026
64 checks passed
@steipete

Copy link
Copy Markdown
Contributor

Landed via GitHub rebase onto main.

  • Source SHA: 115f0cf
  • Landed SHA: 1531123
  • Gates: pnpm test extensions/volcengine/tts.test.ts; OPENCLAW_LIVE_TEST=1 pnpm test:live extensions/volcengine/tts.live.test.ts; live .profile Ogg/Opus probe (OggS bytes); pnpm check:changed; pnpm build; exact-head PR CI green.
  • API check: verified against BytePlus Seed Speech docs for the unidirectional HTTP API and voice list.

Thanks @xuruiray!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants