memory-lancedb: add configurable timeout/retry for embedding calls#56532
memory-lancedb: add configurable timeout/retry for embedding calls#56532amittell wants to merge 4 commits into
Conversation
Greptile SummaryThis PR adds configurable Issues found:
Other notes:
Confidence Score: 4/5Safe to merge once the stale test assertion is removed — it will cause CI failures as-is. One P1 finding: the old toHaveBeenCalledWith assertion in index.test.ts was not removed and will fail now that embeddingsCreate receives a second argument. All other changes are straightforward and correct. extensions/memory-lancedb/index.test.ts — the stale assertion at lines 288–292 must be removed before merging.
|
| Filename | Overview |
|---|---|
| extensions/memory-lancedb/config.ts | Adds optional timeoutMs and maxRetries fields to the embedding config type, schema validation, and UI field descriptors. Changes look correct. |
| extensions/memory-lancedb/index.ts | Adds DEFAULT_EMBEDDING_TIMEOUT_MS (10 s) and DEFAULT_EMBEDDING_MAX_RETRIES (1) constants; passes them into the OpenAI client constructor and also overrides per-request via the options arg to embeddings.create. Logic is sound. |
| extensions/memory-lancedb/index.test.ts | Adds a new toHaveBeenCalledWith assertion that includes the timeout option, but the old assertion (without the timeout arg) was not removed — it will now fail because embeddingsCreate is called with two arguments. |
| src/daemon/launchd-plist.ts | Bumps LAUNCH_AGENT_THROTTLE_INTERVAL_SECONDS from 1 to 10 to prevent rapid respawn storms; comment updated to reflect the new tradeoff. |
Comments Outside Diff (1)
-
extensions/memory-lancedb/index.test.ts, line 288-292 (link)Stale assertion will fail after this change
The original assertion at lines 288–292 checks that
embeddingsCreatewas called with only the params object and no second argument. However, after this PR the call site becomesthis.client.embeddings.create(params, { timeout: this.timeoutMs }), so the mock is now invoked with two arguments. Vitest'stoHaveBeenCalledWithperforms an exact argument match — a call offn(a, b)does not satisfyexpect(fn).toHaveBeenCalledWith(a).The old assertion should be removed (or replaced by the new one below it), otherwise this test will fail:
Prompt To Fix With AI
This is a comment left during a code review. Path: extensions/memory-lancedb/index.test.ts Line: 288-292 Comment: **Stale assertion will fail after this change** The original assertion at lines 288–292 checks that `embeddingsCreate` was called with only the params object and no second argument. However, after this PR the call site becomes `this.client.embeddings.create(params, { timeout: this.timeoutMs })`, so the mock is now invoked with two arguments. Vitest's `toHaveBeenCalledWith` performs an exact argument match — a call of `fn(a, b)` does **not** satisfy `expect(fn).toHaveBeenCalledWith(a)`. The old assertion should be removed (or replaced by the new one below it), otherwise this test will fail: How can I resolve this? If you propose a fix, please make it concise.
Prompt To Fix All With AI
This is a comment left during a code review.
Path: extensions/memory-lancedb/index.test.ts
Line: 288-292
Comment:
**Stale assertion will fail after this change**
The original assertion at lines 288–292 checks that `embeddingsCreate` was called with only the params object and no second argument. However, after this PR the call site becomes `this.client.embeddings.create(params, { timeout: this.timeoutMs })`, so the mock is now invoked with two arguments. Vitest's `toHaveBeenCalledWith` performs an exact argument match — a call of `fn(a, b)` does **not** satisfy `expect(fn).toHaveBeenCalledWith(a)`.
The old assertion should be removed (or replaced by the new one below it), otherwise this test will fail:
```suggestion
expect(embeddingsCreate).toHaveBeenCalledWith(
{
model: "text-embedding-3-small",
input: "hello dimensions",
dimensions: 1024,
},
{ timeout: 10_000 },
);
```
How can I resolve this? If you propose a fix, please make it concise.Reviews (1): Last reviewed commit: "memory-lancedb: add configurable timeout..." | Re-trigger Greptile
There was a problem hiding this comment.
Pull request overview
Adds configurability to the memory-lancedb plugin’s embedding request behavior (timeout + retries) to reduce cascading failures during provider rate-limit/connectivity events, and increases launchd relaunch throttling to avoid respawn storms.
Changes:
- Add
embedding.timeoutMs(default 10s) andembedding.maxRetries(default 1) tomemory-lancedbconfig + UI hints. - Apply the configured timeout/retry settings to the OpenAI embeddings client and per-request call.
- Increase LaunchAgent
ThrottleIntervalfrom 1s to 10s.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| src/daemon/launchd-plist.ts | Increase ThrottleInterval constant to reduce launchd respawn storms. |
| extensions/memory-lancedb/index.ts | Thread embedding timeout/retries into the OpenAI client and embedding calls. |
| extensions/memory-lancedb/index.test.ts | Update test expectations for the new timeout request option. |
| extensions/memory-lancedb/config.ts | Extend plugin config parsing + UI hints to support timeout/retry settings. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 149c4d2eb5
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
149c4d2 to
871a86e
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 871a86eed7
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
871a86e to
55f94c5
Compare
🔒 Aisle Security AnalysisWe found 1 potential security issue(s) in this PR:
1. 🟠 SSRF and environment secret exfiltration via configurable OpenAI baseUrl with ${ENV} expansion
Description
If an attacker can influence the plugin configuration (e.g., via a shared config file checked into a repo, or any other untrusted config channel), this enables:
Vulnerable code (env var expansion and use as base URL): function resolveEnvVars(value: string): string {
return value.replace(/\$\{([^}]+)\}/g, (_, envVar) => {
const envValue = process.env[envVar];
if (!envValue) {
throw new Error(`Environment variable ${envVar} is not set`);
}
return envValue;
});
}
baseUrl:
typeof embedding.baseUrl === "string" ? resolveEnvVars(embedding.baseUrl) : undefined,Request sink: this.client = new OpenAI({
apiKey,
baseURL: baseUrl,
timeout: timeoutMs ?? DEFAULT_EMBEDDING_TIMEOUT_MS,
maxRetries: maxRetries ?? DEFAULT_EMBEDDING_MAX_RETRIES,
});RecommendationTreat
Example hardening: function validateBaseUrl(raw: string): string {
const url = new URL(raw);
if (url.protocol !== "https:") {
throw new Error("embedding.baseUrl must use https");
}
// Option A: allowlist known hosts
const allowedHosts = new Set(["api.openai.com", "api.groq.com"]);
if (!allowedHosts.has(url.hostname)) {
throw new Error("embedding.baseUrl host is not allowed");
}
return url.toString();
}
const baseUrl = typeof embedding.baseUrl === "string"
? validateBaseUrl(embedding.baseUrl) // no resolveEnvVars here
: undefined;Additionally, consider keeping Analyzed PR: #56532 at commit Last updated on: 2026-04-28T01:48:05Z |
57e4bec to
63207dd
Compare
3a879f8 to
7450fa6
Compare
|
Codex review: found issues before merge. Reviewed May 30, 2026, 11:12 AM ET / 15:12 UTC. Summary PR surface: Source +87, Tests +127, Docs +1. Total +215 across 5 files. Reproducibility: yes. source-reproducible. Current main and the PR diff show auto-recall passes a hardcoded request timeout, and the OpenAI SDK uses request timeout before client timeout, so the new config cannot bound that agent-turn path. Review metrics: 1 noteworthy metric.
Merge readiness Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch. Rank-up moves:
Risk before merge
Maintainer options:
Next step before merge
Security Review findings
Review detailsBest possible solution: Land a narrow memory-lancedb change where timeout/retry settings are applied to every advertised direct embedding path, unsupported provider-adapter modes are clearly scoped or rejected, and release notes stay in the PR body or landing commit instead of Do we have a high-confidence way to reproduce the issue? Yes, source-reproducible. Current main and the PR diff show auto-recall passes a hardcoded request timeout, and the OpenAI SDK uses request timeout before client timeout, so the new config cannot bound that agent-turn path. Is this the best way to solve the issue? No. Wiring the values into the direct OpenAI constructor is useful, but it is not the complete fix while auto-recall overrides that constructor budget and provider-backed modes accept settings they cannot honor. Full review comments:
Overall correctness: patch is incorrect AGENTS.md: found and applied where relevant. Codex review notes: model gpt-5.5, reasoning high; reviewed against 8539e0283a55. Label changesLabel changes:
Label justifications:
Evidence reviewedPR surface: Source +87, Tests +127, Docs +1. Total +215 across 5 files. View PR surface stats
Acceptance criteria:
What I checked:
Likely related people:
What the crustacean ranks mean
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics. How this review workflow works
|
7450fa6 to
f328aad
Compare
40e0ed6 to
d7d898b
Compare
d7d898b to
c96e66c
Compare
|
@clawsweeper re-review |
|
🦞🧹 I asked ClawSweeper to review this item again. Re-review progress:
|
813f761 to
f25ea71
Compare
f25ea71 to
f19a655
Compare
f19a655 to
c68cbfa
Compare
c68cbfa to
ea3f219
Compare
ea3f219 to
b1e2e23
Compare
Add timeoutMs and maxRetries config fields to the memory-lancedb embedding pipeline. Both fields are validated and clamped to safe ranges; out-of-bounds values silently fall back to undefined (SDK defaults). Wires the values into OpenAiCompatibleEmbeddings constructor options and exposes them in plugin manifest configSchema and uiHints.
b1e2e23 to
de58b85
Compare
|
Thanks @amittell. I landed the no-new-config version of this reliability fix on main: The landed shape adds internal 15s deadlines and 60s degraded cooldowns for the memory recall paths, forwards AbortSignal through provider-backed memory-lancedb embeddings, and documents the existing provider-selection escape hatch for unreachable OpenAI embeddings. We are intentionally not adding new public timeout or retry config knobs here; the supported operator action is to use the existing memorySearch.provider / model / remote provider selection when OpenAI embeddings are unreachable. Closing this PR as superseded/not planned for the config-surface part. |
Replaces #56517 (closed, could not reopen).
Real behavior proof
memory-lancedb's OpenAI-SDK embedding client had no operator-tunable timeout/retry budget. A hung or 5xx-storming embedding backend blocks on the SDK default (600000ms) and stalls every agent turn that triggers auto-recall. This PR lets operators bound it viaembedding.timeoutMs/embedding.maxRetries. Per ClawSweeper review, the bounds now apply ONLY when explicitly configured; unset installs keep the SDK defaults (no upgrade behavior change).openclawcheckout (upgrade onto current upstream/main) with the bundledopenaiSDK the plugin loads at runtime, exercised against a blackhole TCP endpoint that accepts the connection but never responds (simulating a hung embedding backend).nodeagainst the bundledopenaiSDK in~/.openclaw/local checkout:APIConnectionTimeoutErrorafter ~4.5s total — instead of hanging on the SDK default 600000ms. The deployed wiring inextensions/memory-lancedb/index.tsnow spreadstimeout/maxRetriesinto thenew OpenAI({...})constructor only when set (...(timeoutMs !== undefined ? { timeout: timeoutMs } : {})), so unconfigured installs retain SDK defaults.embedding.timeoutMsset, a hung endpoint fails fast at the configured bound. With it unset, the OpenAI SDK default (600000ms / 2 retries) is preserved unchanged — no regression for installs with slow-but-working endpoints (the upgrade risk ClawSweeper flagged). Config-layer clamping still rejects out-of-range values (timeoutMsto [1000, 60000],maxRetriesto [0, 5]).