Skip to content

feat(plugins): Non-blocking async plugin hooks with configurable timeout + fallback #62702

Description

@issaba1

Problem

Plugin hooks that perform I/O (cloud API calls, database queries, file reads) currently block the turn pipeline. If any hook is slow or the external service is degraded, the entire agent turn waits — the user sees no typing indicator and no response until the hook completes or fails.

Real-world example: A memory recall plugin that calls an external cloud API adds 500ms–30s of blocking latency before the LLM even starts. If the API is degraded, every single message is delayed by the full timeout (typically 30s+).

The current plugin SDK does not provide a built-in way to:

  • Run a hook with a max timeout and fall through on timeout
  • Run a hook asynchronously and inject its result into context if it completes before a deadline
  • Distinguish between hooks that MUST complete before the turn (synchronous/critical) vs hooks that should be best-effort

Proposed Solution

1. Hook timeout config per plugin entry

{
  plugins: {
    entries: {
      "my-memory-plugin": {
        enabled: true,
        hooks: {
          before_turn: {
            timeoutMs: 2000,     // abort if hook takes > 2s
            fallback: "skip"     // "skip" | "error" | "empty"  
          }
        }
      }
    }
  }
}

2. Async context injection pattern

For hooks that inject context (system prompt fragments), allow a "race" pattern:

// Plugin SDK addition
api.registerContextProvider({
  timeoutMs: 2000,
  async provide(ctx) {
    const data = await fetchExternalData(ctx);
    return { inject: `## Memory\n${data}` };
  }
  // If times out: inject nothing, log warning, continue turn
});

3. Hook classification

Add a critical flag to hooks that must block (e.g. security/guard hooks):

api.on('before_turn', handler, { critical: true, timeoutMs: 500 });
// critical: true → blocks turn, errors on timeout
// critical: false (default) → best-effort, skips on timeout

Why This Matters

  • Interactive responsiveness: Users sending short messages ("yes", "ok", "go") experience the same 30s+ pre-processing as complex research queries. With async hooks, non-critical plugins don't gate the response.
  • Resilience: External API degradation (memory clouds, vector DBs) currently cascades into agent unresponsiveness. Timeout+fallback isolates failures.
  • Separation of concerns: Plugin authors shouldn't need to implement their own timeout/retry logic. The SDK should handle this.

Implementation Notes

  • The before_turn hook chain already executes sequentially. Adding Promise.race([hook(), timeout(ms)]) per non-critical hook is the minimal change.
  • For context injection specifically: collect all provider promises with a deadline, inject whatever resolves in time, skip the rest.
  • Add a plugin:hook:timeout event for observability (visible in logs/diagnostics).

Expected Impact

For agents with multiple context-provider plugins: 40-60s pre-turn latency → <5s when external services are slow. When all services are healthy, no change in behavior.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    Priority

    None yet

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions