Problem
Plugin hooks that perform I/O (cloud API calls, database queries, file reads) currently block the turn pipeline. If any hook is slow or the external service is degraded, the entire agent turn waits — the user sees no typing indicator and no response until the hook completes or fails.
Real-world example: A memory recall plugin that calls an external cloud API adds 500ms–30s of blocking latency before the LLM even starts. If the API is degraded, every single message is delayed by the full timeout (typically 30s+).
The current plugin SDK does not provide a built-in way to:
- Run a hook with a max timeout and fall through on timeout
- Run a hook asynchronously and inject its result into context if it completes before a deadline
- Distinguish between hooks that MUST complete before the turn (synchronous/critical) vs hooks that should be best-effort
Proposed Solution
1. Hook timeout config per plugin entry
{
plugins: {
entries: {
"my-memory-plugin": {
enabled: true,
hooks: {
before_turn: {
timeoutMs: 2000, // abort if hook takes > 2s
fallback: "skip" // "skip" | "error" | "empty"
}
}
}
}
}
}
2. Async context injection pattern
For hooks that inject context (system prompt fragments), allow a "race" pattern:
// Plugin SDK addition
api.registerContextProvider({
timeoutMs: 2000,
async provide(ctx) {
const data = await fetchExternalData(ctx);
return { inject: `## Memory\n${data}` };
}
// If times out: inject nothing, log warning, continue turn
});
3. Hook classification
Add a critical flag to hooks that must block (e.g. security/guard hooks):
api.on('before_turn', handler, { critical: true, timeoutMs: 500 });
// critical: true → blocks turn, errors on timeout
// critical: false (default) → best-effort, skips on timeout
Why This Matters
- Interactive responsiveness: Users sending short messages ("yes", "ok", "go") experience the same 30s+ pre-processing as complex research queries. With async hooks, non-critical plugins don't gate the response.
- Resilience: External API degradation (memory clouds, vector DBs) currently cascades into agent unresponsiveness. Timeout+fallback isolates failures.
- Separation of concerns: Plugin authors shouldn't need to implement their own timeout/retry logic. The SDK should handle this.
Implementation Notes
- The before_turn hook chain already executes sequentially. Adding
Promise.race([hook(), timeout(ms)]) per non-critical hook is the minimal change.
- For context injection specifically: collect all provider promises with a deadline, inject whatever resolves in time, skip the rest.
- Add a
plugin:hook:timeout event for observability (visible in logs/diagnostics).
Expected Impact
For agents with multiple context-provider plugins: 40-60s pre-turn latency → <5s when external services are slow. When all services are healthy, no change in behavior.
Problem
Plugin hooks that perform I/O (cloud API calls, database queries, file reads) currently block the turn pipeline. If any hook is slow or the external service is degraded, the entire agent turn waits — the user sees no typing indicator and no response until the hook completes or fails.
Real-world example: A memory recall plugin that calls an external cloud API adds 500ms–30s of blocking latency before the LLM even starts. If the API is degraded, every single message is delayed by the full timeout (typically 30s+).
The current plugin SDK does not provide a built-in way to:
Proposed Solution
1. Hook timeout config per plugin entry
2. Async context injection pattern
For hooks that inject context (system prompt fragments), allow a "race" pattern:
3. Hook classification
Add a
criticalflag to hooks that must block (e.g. security/guard hooks):Why This Matters
Implementation Notes
Promise.race([hook(), timeout(ms)])per non-critical hook is the minimal change.plugin:hook:timeoutevent for observability (visible in logs/diagnostics).Expected Impact
For agents with multiple context-provider plugins: 40-60s pre-turn latency → <5s when external services are slow. When all services are healthy, no change in behavior.