feat(hooks): add message:transcribed and message:preprocessed internal hooks#9859
feat(hooks): add message:transcribed and message:preprocessed internal hooks#9859Drickon wants to merge 2 commits intoopenclaw:mainfrom
Conversation
| // Trigger message:transcribed hook after media understanding completes | ||
| // At this point, audio has been transcribed and Transcript field is populated | ||
| await triggerInternalHook( | ||
| createInternalHookEvent("message", "transcribed", finalized.SessionKey ?? "", { | ||
| from: finalized.From, | ||
| to: finalized.To, | ||
| body: finalized.Body, | ||
| bodyForAgent: finalized.BodyForAgent, | ||
| transcript: finalized.Transcript, | ||
| timestamp: finalized.Timestamp, | ||
| channelId: finalized.ChannelId, | ||
| conversationId: finalized.ConversationId, | ||
| messageId: finalized.MessageSid, | ||
| senderId: finalized.SenderId, | ||
| senderName: finalized.SenderName, | ||
| senderUsername: finalized.SenderUsername, | ||
| provider: finalized.Provider, | ||
| surface: finalized.Surface, | ||
| mediaPath: finalized.MediaPath, | ||
| mediaType: finalized.MediaType, | ||
| cfg, | ||
| }), | ||
| ); | ||
|
|
There was a problem hiding this comment.
Hook fires in tests
message:transcribed is triggered unconditionally even when OPENCLAW_TEST_FAST==="1" skips applyMediaUnderstanding/applyLinkUnderstanding (src/auto-reply/reply/get-reply.ts:119-155). That means in fast tests the hook will fire without any transcription/link processing having happened, contradicting the event’s meaning and docs. Gate this hook behind the same !isFastTestEnv condition or otherwise ensure it only fires after the processing steps actually ran.
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/auto-reply/reply/get-reply.ts
Line: 132:155
Comment:
**Hook fires in tests**
`message:transcribed` is triggered unconditionally even when `OPENCLAW_TEST_FAST==="1"` skips `applyMediaUnderstanding`/`applyLinkUnderstanding` (`src/auto-reply/reply/get-reply.ts:119-155`). That means in fast tests the hook will fire without any transcription/link processing having happened, contradicting the event’s meaning and docs. Gate this hook behind the same `!isFastTestEnv` condition or otherwise ensure it only fires after the processing steps actually ran.
How can I resolve this? If you propose a fix, please make it concise.There was a problem hiding this comment.
Good catch on the original concern, but this is gated by the transcript check:
if (finalized.Transcript) {
void triggerInternalHook(
createInternalHookEvent("message", "transcribed", ...)
).catch(...);
}In fast test mode, applyMediaUnderstanding is skipped, so finalized.Transcript remains falsy and the hook never fires. The guard ensures message:transcribed only triggers when transcription actually occurred.
| // Trigger internal hook for message:received | ||
| const messageEvent = createInternalHookEvent("message", "received", ctx.SessionKey ?? "", { | ||
| from: ctx.From, | ||
| to: ctx.To, | ||
| content, | ||
| timestamp, | ||
| channelId, | ||
| conversationId, | ||
| messageId: messageIdForHook, | ||
| senderId: ctx.SenderId, | ||
| senderName: ctx.SenderName, | ||
| senderUsername: ctx.SenderUsername, | ||
| senderE164: ctx.SenderE164, | ||
| provider: ctx.Provider, | ||
| surface: ctx.Surface, | ||
| threadId: ctx.MessageThreadId, | ||
| originatingChannel: ctx.OriginatingChannel, | ||
| originatingTo: ctx.OriginatingTo, | ||
| }); | ||
| await triggerInternalHook(messageEvent); | ||
|
|
There was a problem hiding this comment.
Hook can block pipeline
dispatchReplyFromConfig now does await triggerInternalHook(messageEvent) for message:received (src/auto-reply/reply/dispatch-from-config.ts:201-221). Unlike the existing plugin hook runner call (which is fire-and-forget), this will delay message handling for every inbound message whenever any internal message/message:received hook is registered, and can also throw and fail the whole dispatch path if a handler rejects. If these hooks are intended to be “critical points” but non-blocking like the other hook system, consider making this non-blocking (or explicitly isolating errors) so user message processing isn’t gated on hook runtime/failures.
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/auto-reply/reply/dispatch-from-config.ts
Line: 201:221
Comment:
**Hook can block pipeline**
`dispatchReplyFromConfig` now does `await triggerInternalHook(messageEvent)` for `message:received` (`src/auto-reply/reply/dispatch-from-config.ts:201-221`). Unlike the existing plugin hook runner call (which is fire-and-forget), this will delay message handling for every inbound message whenever any internal `message`/`message:received` hook is registered, and can also throw and fail the whole dispatch path if a handler rejects. If these hooks are intended to be “critical points” but non-blocking like the other hook system, consider making this non-blocking (or explicitly isolating errors) so user message processing isn’t gated on hook runtime/failures.
How can I resolve this? If you propose a fix, please make it concise.There was a problem hiding this comment.
Thanks for flagging! This is actually already addressed in the current code — the hook is fired as fire-and-forget with error isolation:
void triggerInternalHook(messageEvent).catch((err) => {
logVerbose(\`dispatch-from-config: message:received internal hook failed: ${String(err)}\`);
});The void prefix ensures it does not block the dispatch pipeline, and the .catch() prevents unhandled rejections from propagating. Message processing continues immediately without waiting for hook completion.
Strong support + use case: Event-driven agent notifications强烈支持 + 使用场景:事件驱动的 Agent 主动通知 This PR addresses a critical gap. I built an entire external events-framework (1500 lines of Python: event inbox → router → rate-limit/dedupe/quiet-hours → Telegram) just because OpenClaw has no native event-driven hook for "something happened → notify user." 这个 PR 解决了一个关键缺口。我构建了一整套外部 events-framework(1500 行 Python:事件收件箱 → 路由器 → 限频/去重/静默时段 → Telegram),仅仅因为 OpenClaw 没有原生的事件驱动钩子。 My use case / 我的使用场景I run a multi-agent system (1 PM + 8 sub-agents). When sub-agents complete work, cost thresholds are hit, or errors occur, I need immediate, deduplicated, rate-limited notifications — not cron polling every N minutes. 我运行一个多 Agent 系统(1 个项目经理 + 8 个子 Agent)。当子 Agent 完成工作、费用阈值触发或发生错误时,我需要即时、去重、限频的通知——而不是 cron 每 N 分钟轮询。 The
On the Greptile performance concern / 关于 Greptile 的性能顾虑Greptile flagged that triggerInternalHook(event).catch(err => logger.warn("hook error", err));Note on competing PRs / 关于竞争 PRI see #7545 and #9387 also implement similar functionality. It would be great if maintainers could signal which approach they prefer so the community can consolidate effort. |
|
Thanks for the detailed support @Zjianru — great to see someone else hitting the same wall. Your event framework sounds like exactly the kind of thing that should be built-in rather than bolted on. On the performance concern: the current code already uses fire-and-forget with error isolation ( Regarding the competing PRs (#7545, #9387) — here is how they compare:
This PR fires at the dispatch level ( Would love to see maintainers weigh in on which direction they prefer so effort can consolidate. |
Technical suggestions for the two missing capabilities / 关于两个缺失能力的技术建议Hi @Drickon, thanks again for the thorough comparison table — it really clarifies the landscape. I noticed two gaps in this PR compared to the other approaches, and wanted to offer some concrete suggestions. 感谢你提供的详细对比表,非常清晰。我注意到相比其他方案,这个 PR 还有两个能力缺口,想提供一些具体的技术建议。 1.
|
|
Thanks for the suggestions! Added Agreed that interception/middleware should be a separate PR. Your multi-agent use case would give you better perspective on the design decisions there. If you're interested in taking that on, go for it. Just drop a comment here or open an issue so we can coordinate. Appreciate the input! |
74ccda2 to
9a1eada
Compare
Implements the message lifecycle hooks documented as 'Future Events' in the hooks documentation. These events enable extensions to observe inbound and outbound messages for logging, analytics, memory systems, and archival. New events: - message:received: triggered when an inbound message is processed - message:sent: triggered after an outbound message is successfully delivered Both events include rich context (text, channel, chatType, sender info, media URLs) and are fired asynchronously so they never block message processing. Includes: - 6 test cases covering event triggering, general/specific handler routing, media fields, error isolation, and type validation - Documentation update moving these from 'Future Events' to documented events Combines approaches from prior community PRs openclaw#6384, openclaw#6797, and openclaw#9859. Resolves openclaw#12775, openclaw#8807, openclaw#13004 Related: openclaw#5354, openclaw#12867, openclaw#12914
|
👋 Just wanted to add a +1 for this PR — we have a real use case waiting on it. Use case: Auto-transcription of inbound audio messages via a workspace hook. Currently audio files arrive as attachments and can get lost in context compaction before processing. The The implementation looks solid (fire-and-forget, channel-agnostic, good test coverage). Would love to see this merged! 🙏 |
|
this is exactly what we proposed in #14539. been running all of these events (received + preprocessed + sent) in production for 8+ days — 550+ events, 2 bots, 4 groups, zero crashes. a few things from production experience:
production hooks built on this:
reference hook implementation: https://github.com/matskevich/openclaw-infra/tree/main/hooks/memory-logger happy to help test or review. |
bfc1ccb to
f92900f
Compare
…l hooks Adds two new internal hook events that fire after media/link processing: - message:transcribed: fires when audio has been transcribed, providing the transcript text alongside the original body and media metadata. Useful for logging, analytics, or routing based on spoken content. - message:preprocessed: fires for every message after all media + link understanding completes. Gives hooks access to the fully enriched body (transcripts, image descriptions, link summaries) before the agent sees it. Both hooks are added in get-reply.ts, after applyMediaUnderstanding and applyLinkUnderstanding. message:received and message:sent are already in upstream (f07bb8e) and are not duplicated here. Typed contexts (MessageTranscribedHookContext, MessagePreprocessedHookContext) and type guards (isMessageTranscribedEvent, isMessagePreprocessedEvent) added to internal-hooks.ts alongside the existing received/sent types. Test coverage in src/hooks/message-hooks.test.ts.
744a8c1 to
913c370
Compare
Arrow function passed to registerInternalHook was implicitly returning the number from Array.push(), which is not assignable to void | Promise<void>. Use block body to discard the return value.
Summary
Adds two new internal hook events that fire after media and link understanding:
message:transcribedFires when audio has been fully transcribed. Provides the transcript text alongside the original body and media metadata. Useful for logging, analytics, or routing based on spoken content.
message:preprocessedFires for every message after all media + link understanding completes — giving hooks access to the fully enriched body (transcripts, image descriptions, link summaries) before the agent sees it.
Context
message:receivedandmessage:sentwere backported by the upstream team in f07bb8e (merged Feb 17). This PR adds the two remaining hooks from the original set.Community
message:transcribedfor a voice-note logging integrationmessage:preprocessedin production on a fork and reports it stableChanges
src/auto-reply/reply/get-reply.ts— triggersmessage:transcribed(audio only) andmessage:preprocessed(all messages) afterapplyMediaUnderstanding+applyLinkUnderstandingsrc/hooks/internal-hooks.ts— addsMessageTranscribedHookContext,MessagePreprocessedHookContext, and corresponding event types + type guardssrc/hooks/message-hooks.test.ts— new test file covering all four message event typesdocs/automation/hooks.md— documents all four message events and their contextssrc/config/types.hooks.ts— updates comment to include new event keys