Skip to content

Commit 930ea46

Browse files
uezouezo
committed
Add support for AIAvatarKit as an AI agent backend service
- Added `AIAvatarKitService` for integrating `AIAvatarKit` as a backend AI agent. - Allows using AI agent behaviors on the server side through the standard LLM service interface. - Users can simply attach `AIAvatarKitService` without worrying about the underlying agentic operations.
1 parent f48bb69 commit 930ea46

File tree

9 files changed

+773
-0
lines changed

9 files changed

+773
-0
lines changed

‎Plugins/AIAvatarKitServiceWebGL.jslib‎

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
mergeInto(LibraryManager.library, {
2+
StartAIAvatarKitMessageStreamJS: function(targetObjectNamePtr, sessionIdPtr, urlPtr, aakStreamRequestPtr) {
3+
let targetObjectName = UTF8ToString(targetObjectNamePtr);
4+
let sessionId = UTF8ToString(sessionIdPtr);
5+
let url = UTF8ToString(urlPtr);
6+
let aakStreamRequest = UTF8ToString(aakStreamRequestPtr);
7+
let decoder = new TextDecoder("utf-8");
8+
9+
if (document.aakAbortController == null) {
10+
document.aakAbortController = new AbortController();
11+
}
12+
13+
fetch(url, {
14+
headers: {
15+
"Content-Type": "application/json"
16+
},
17+
method: "POST",
18+
body: aakStreamRequest,
19+
signal: document.aakAbortController.signal
20+
})
21+
.then((response) => response.body.getReader())
22+
.then((reader) => {
23+
const readChunk = function({done, value}) {
24+
if(done) {
25+
reader.releaseLock();
26+
return;
27+
}
28+
SendMessage(targetObjectName, "SetAIAvatarKitMessageStreamChunk", sessionId + "::" + decoder.decode(value));
29+
reader.read().then(readChunk);
30+
}
31+
reader.read().then(readChunk);
32+
})
33+
.catch((err) => {
34+
console.error(`Error at fetch: ${err.message}`);
35+
});
36+
},
37+
38+
AbortAIAvatarKitMessageStreamJS: function() {
39+
console.log("Abort AIAvatarKit at AbortAIAvatarKitMessageStreamJS");
40+
document.aakAbortController.abort();
41+
}
42+
});

‎Plugins/AIAvatarKitServiceWebGL.jslib.meta‎

Lines changed: 2 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎README.ja.md‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -590,6 +590,13 @@ LLMでTool Call（Function Calling）がサポートされている場合、そ
590590

591591
**NOTE**: プロジェクトにLLMFunctionSkillを使用している場合、[Migration from FunctionSkill to Tool](#migration-from-functionskill-to-tool)も参照してください。
592592

593+
### Integration with Remote AI Agents
594+
595+
ChatdollKitは標準でシンプルなTool Callをサポートしていますが、よりAgenticな振る舞いを実現するため、サーバーサイドのAIエージェントとの連携機能も提供しています。
596+
597+
具体的には、AIエージェントを`LLMService`として登録することでRESTful API経由で呼び出すことができます。これにより、裏側で行われているエージェント的なプロセスを意識することなく、ChatdollKitのフロント側ではリクエストの送信とレスポンスの受信と発話・モーション等の表現に注力することができます。
598+
現在のところ [Dify](https://dify.ai) と [AIAvatarKit](https://github.com/uezo/aiavatarkit) に対応しています。`DifyService`または`AIAvatarKitService`をアタッチし、設定を行ったうえで`IsEnabled`をオンにすることで利用できます。
599+
593600

594601
## 🎙️ Devices
595602

‎README.md‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -590,6 +590,14 @@ By creating a component that implements `ITool` or extends `ToolBase` and attach
590590
**NOTE**: See [Migration from FunctionSkill to Tool](#migration-from-functionskill-to-tool) if your project has custom LLMFunctionSkills.
591591

592592

593+
### Integration with Remote AI Agents
594+
595+
While ChatdollKit natively supports simple tool calls, it also provides integration with server-side AI agents to enable more agentic behaviors.
596+
597+
Specifically, ChatdollKit allows you to call AI agents through RESTful APIs by registering them as an `LLMService`. This lets you send requests and receive responses without needing to be aware of the agentic processes happening behind the scenes.
598+
Currently, [Dify](https://dify.ai) and [AIAvatarKit](https://github.com/uezo/aiavatarkit) are supported. You can use them by attaching either `DifyService` or `AIAvatarKitService`, configuring their settings, and enabling the `IsEnabled` flag.
599+
600+
593601
## 🎙️ Devices
594602

595603
We provide a device control mechanism. Currently, microphones and cameras are supported.

‎Scripts/LLM/AIAvatarKit.meta‎

Lines changed: 8 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

0 commit comments

Comments
 (0)