Skip to content

fix(memory): respect QMD timeout for memory_search#95757

Merged
steipete merged 10 commits into
openclaw:mainfrom
bdjben:julian/configurable-memory-search-timeout
Jul 13, 2026
Merged

fix(memory): respect QMD timeout for memory_search#95757
steipete merged 10 commits into
openclaw:mainfrom
bdjben:julian/configurable-memory-search-timeout

Conversation

@bdjben

@bdjben bdjben commented Jun 22, 2026

Copy link
Copy Markdown
Contributor

What Problem This Solves

Fixes #91947.

QMD-backed memory_search can legitimately take longer than the historical fixed 15-second whole-tool deadline, even though QMD already exposes memory.qmd.limits.timeoutMs. As a result, configured longer QMD searches were aborted before their QMD-owned budget elapsed.

Why This Change Was Made

The final implementation assigns deadlines at their owner boundaries:

  • memory.qmd.limits.timeoutMs governs only the actual QMD query / search / vsearch subprocess or mcporter call.
  • The existing 15-second default remains active for manager setup, QMD pre/post-command maintenance, builtin search, wiki/supplement work, zero-hit sync wait, visibility filtering, and transient cleanup.
  • QMD-to-builtin fallback receives a fresh bounded 15-second setup/search budget, and failed QMD retirement no longer delays it.
  • Caller cancellation propagates through the tool wrapper and active search subprocess while preserving the original reason and avoiding cooldown.
  • Deadline timers/listeners and late one-shot managers are cleaned up.

This PR began with @bdjben's focused fix and local 23.9-second QMD proof. The maintainer repair preserves that credit while removing the broader whole-manager exemption and QMD close override from the earlier patch. The final test mock forwards the private symbol-keyed deadline option without importing the production deadline module, keeping current-main debug typing at its owner and avoiding a recurring merge conflict.

User Impact

Users who intentionally raise memory.qmd.limits.timeoutMs can now complete slow QMD-backed agent memory searches beyond 15 seconds. Non-QMD work and QMD-to-builtin fallback remain bounded by the existing default deadline, so raising the QMD timeout does not extend setup, wiki, supplement, builtin, or cleanup work.

No config, schema, storage, protocol, plugin SDK, or dependency surface is added.

Evidence

  • Verified current-main PR head: dcd0e5040012cc67fee63204a287c56edb00ad8a, including base ad1b4d9731709430757c8776ff8e99602589d8ec.
  • Exact-head Testbox tbx_01kxd3jvggptc47g9yanc6772p, Actions run 29229938011: 230/230 passed across deadline, tools, search manager, QMD manager, and host-process tests.
  • Earlier synthetic current-main Testbox tbx_01kxd0x0ggc0029as1e8vwwmyt, Actions run 29227656913: 230/230 passed across the same owner/dependency surfaces.
  • Exact-head hosted CI run 29230522584 passed.
  • Earlier Blacksmith Testbox proof passed changed extension tsgo, lint, guards, import cycles, 244 focused tests, and the full memory-core suite at 66 files / 1,020 tests.
  • AWS Crabbox real QMD 1.1.6 + mcporter 0.12.3 proof through createMemorySearchTool completed with backend qmd, five hits, 28.356 seconds wall time / 27.081 seconds search time under a configured 300-second QMD budget.
  • Fresh final test-seam autoreview reported no accepted/actionable findings at 0.99 confidence.
  • Docs formatting and git diff --check passed.

No CHANGELOG.md edit; release generation owns changelog entries.

@openclaw-barnacle openclaw-barnacle Bot added docs Improvements or additions to documentation extensions: memory-core Extension: memory-core size: S triage: needs-pr-context Candidate: external PR body lacks required problem context or evidence. triage: refactor-only Candidate: refactor/cleanup-only PR without maintainer context. labels Jun 22, 2026
@bdjben
bdjben force-pushed the julian/configurable-memory-search-timeout branch from e8e3e59 to 7362978 Compare June 22, 2026 12:03
@bdjben

bdjben commented Jun 22, 2026

Copy link
Copy Markdown
Contributor Author

Local before/after behavior proof

Redaction note: this is from a local QMD-backed OpenClaw install. Local user, agent, room, host, and filesystem identifiers are omitted or redacted.

Before, pre-patch

  • 2026-06-22 14:58:40 IDT: memory_search was invoked with corpus: "memory" and maxResults: 5 while investigating the configurable-timeout PR/manual smoke-test path.
  • 2026-06-22 14:58:55 IDT: the tool returned the old fixed-deadline unavailable payload:
{
  "results": [],
  "disabled": true,
  "unavailable": true,
  "error": "memory_search timed out after 15s",
  "warning": "Memory search is unavailable due to an embedding/provider error.",
  "action": "Check embedding provider configuration and retry memory_search.",
  "debug": {
    "error": "memory_search timed out after 15s"
  }
}

The same pre-patch runtime also showed the hardcoded deadline feeding runMemorySearchToolWithDeadline:

const MEMORY_SEARCH_TOOL_TIMEOUT_MS = 15e3;
timeoutMs: MEMORY_SEARCH_TOOL_TIMEOUT_MS

After, patched local runtime

  • Applied this PR's behavior to the same local install and restarted the gateway.
  • Effective local config for the after test: memory.qmd.limits.timeoutMs = 45000.
  • 2026-06-23 00:53:56 IDT: memory_search was invoked against the larger sessions corpus with maxResults: 10.
  • 2026-06-23 00:54:20 IDT: the tool returned successfully from QMD instead of hitting the old 15s tool deadline:
{
  "provider": "qmd",
  "model": "qmd",
  "mode": "vsearch",
  "debug": {
    "backend": "qmd",
    "configuredMode": "vsearch",
    "effectiveMode": "vsearch",
    "searchMs": 23922,
    "hits": 8
  }
}

searchMs: 23922 is materially above the old hardcoded 15000 ms deadline, so this exercises the behavior changed by the PR. A post-patch gateway log check found no actual runtime memory_search timed out after 15s entries in the search window; the only remaining text match was a prior human-written summary sentence, not a tool timeout.

@clawsweeper

clawsweeper Bot commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

Codex review: stale review; fresh review needed.

Summary
The latest durable ClawSweeper review was for head 770f19aa60b7ed95e98f0aac60d95e05fa67a820, but the PR head is now dcd0e5040012cc67fee63204a287c56edb00ad8a. Its old verdict and PR readiness labels are no longer current.

Next step
Run or wait for a fresh ClawSweeper review on the current PR head.

Review history (6 earlier review cycles)
  • reviewed 2026-07-01T11:06:16.388Z sha cb42cc5 :: needs maintainer review before merge. :: none
  • reviewed 2026-07-01T21:42:12.675Z sha cb42cc5 :: needs maintainer review before merge. :: none
  • reviewed 2026-07-12T01:52:14.896Z sha 756639b :: needs maintainer review before merge. :: none
  • reviewed 2026-07-12T02:55:27.369Z sha 91fd886 :: needs maintainer review before merge. :: none
  • reviewed 2026-07-12T04:19:49.075Z sha 43f6624 :: needs maintainer review before merge. :: none
  • reviewed 2026-07-13T06:49:16.387Z sha 770f19a :: needs maintainer review before merge. :: none

@clawsweeper clawsweeper Bot added proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. P2 Normal backlog priority with limited blast radius. merge-risk: 🚨 compatibility 🚨 May break existing users, config, migrations, defaults, or upgrade paths. merge-risk: 🚨 availability 🚨 May cause crashes, hangs, restart loops, stalls, or process outages. labels Jun 23, 2026
@bdjben
bdjben marked this pull request as ready for review June 23, 2026 12:45
@clawsweeper clawsweeper Bot added rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. and removed rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. labels Jun 23, 2026
@openclaw-barnacle openclaw-barnacle Bot added size: M and removed size: S triage: needs-pr-context Candidate: external PR body lacks required problem context or evidence. labels Jun 29, 2026
@bdjben

bdjben commented Jun 29, 2026

Copy link
Copy Markdown
Contributor Author

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented Jun 29, 2026

Copy link
Copy Markdown
Contributor

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

@bdjben

bdjben commented Jun 30, 2026

Copy link
Copy Markdown
Contributor Author

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented Jun 30, 2026

Copy link
Copy Markdown
Contributor

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

@bdjben

bdjben commented Jun 30, 2026

Copy link
Copy Markdown
Contributor Author

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented Jun 30, 2026

Copy link
Copy Markdown
Contributor

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

@clawsweeper clawsweeper Bot removed the rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. label Jun 30, 2026
@steipete

steipete commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

Maintainer repair is pushed in cb42cc5993. Thanks @bdjben for the original fix and local >15-second QMD evidence; the final commit keeps you as co-author.

What changed:

  • QMD's configured timeout now applies only while the actual QMD search command is running.
  • Setup and QMD maintenance keep the remaining 15-second default budget.
  • Same-call builtin fallback gets a fresh bounded 15-second budget.
  • Wiki/supplement, builtin, zero-hit sync wait, visibility filtering, and one-shot cleanup remain bounded by the default deadline.
  • Caller abort, stable timeout precedence, timer/listener cleanup, late transient-manager cleanup, grouped/direct/mcporter QMD paths, and missing-collection repair are covered.

Exact proof on this head:

  • Focused local: node scripts/run-vitest.mjs extensions/memory-core/src/memory/search-deadline.test.ts extensions/memory-core/src/tools.test.ts extensions/memory-core/src/memory/search-manager.test.ts extensions/memory-core/src/memory/qmd-manager.test.ts packages/memory-host-sdk/src/host/qmd-process.test.ts — 244 tests passed.
  • Blacksmith Testbox through Crabbox tbx_01kwef6xcy84pa8h6b7xhfwgth:
    • changed extension production/test tsgo, lint, guards, and import cycles passed;
    • focused 244 tests passed;
    • pnpm test extensions/memory-core — 66 files / 1,020 tests passed.
  • AWS Crabbox cbx_93a759a4ae3f, run run_6b7232fee439: real QMD 1.1.6 + mcporter 0.12.3 through the actual createMemorySearchTool path completed with backend qmd, five hits, elapsedMs=28356, searchMs=27081, under memory.qmd.limits.timeoutMs=300000.
  • node scripts/format-docs.mjs --check and git diff --check passed.
  • Fresh autoreview with gpt-5.6-sol, xhigh: no accepted/actionable findings.
  • Repo review artifacts validate with scripts/pr review-validate-artifacts 95757 and recommend READY FOR /prepare-pr with zero findings.

Known proof gaps: none for the changed memory-core behavior. CI is running on the pushed SHA; final preparation/merge remains intentionally serialized.

Preserve the phase-scoped QMD timeout semantics on current main.

Co-authored-by: Peter Steinberger <[email protected]>
@bdjben
bdjben force-pushed the julian/configurable-memory-search-timeout branch from cb42cc5 to 756639b Compare July 12, 2026 01:41
@bdjben

bdjben commented Jul 12, 2026

Copy link
Copy Markdown
Contributor Author

Refreshed this branch onto current main (7a456e362d0b) and preserved the phase-aware QMD deadline design from Peter’s repair.

What remains intentional:

  • memory.qmd.limits.timeoutMs governs only the actual QMD / mcporter search subprocesses; the ordinary memory-search deadline pauses only during those commands.
  • Setup, maintenance/sync, filtering, cleanup, corpus supplements, and QMD-to-builtin fallback remain under the ordinary 15s budget. The builtin fallback receives a fresh ordinary deadline through handoff.
  • Current-main local-service acquisition and the long-lived-QMD no-hot-path-forced-sync behavior are retained; the one-shot CLI retry stays covered separately.

Verification on the refreshed head (756639b027c):

  • search-deadline.test.ts + search-manager.test.ts: 47/47 passing.
  • Relevant direct-QMD, mcporter, and retry-phase tests: 3/3 passing.
  • tools.test.ts: 33/33 passing with one known baseline fractional-input timeout excluded; that timeout reproduces unchanged on unmodified main.
  • qmd-manager.test.ts: 146 passing; its three session-export failures reproduce unchanged on unmodified main and are outside this diff.
  • git diff --check, formatting, and extension lint pass.

The branch was independently reviewed after the refresh; no blocker was found.

@bdjben

bdjben commented Jul 12, 2026

Copy link
Copy Markdown
Contributor Author

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented Jul 12, 2026

Copy link
Copy Markdown
Contributor

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

Re-review progress:

@clawsweeper clawsweeper Bot removed proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🦞 diamond lobster Very strong PR readiness with only minor maintainer review expected. merge-risk: 🚨 compatibility 🚨 May break existing users, config, migrations, defaults, or upgrade paths. merge-risk: 🚨 availability 🚨 May cause crashes, hangs, restart loops, stalls, or process outages. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. labels Jul 13, 2026
@steipete

Copy link
Copy Markdown
Contributor

Land-ready at dcd0e5040012cc67fee63204a287c56edb00ad8a, updated with current base ad1b4d9731709430757c8776ff8e99602589d8ec.

Maintainer reconciliation keeps the 15-second default around setup, maintenance, builtin, supplement, and cleanup phases; only the QMD query subprocess uses memory.qmd.limits.timeoutMs. QMD failure hands builtin fallback a fresh bounded budget, caller abort terminates the active host process, and the final symbol-agnostic test seam stays formatter-clean and conflict-free with current-main debug typing.

Proof:

  • Exact-head Testbox 29229938011: 230/230 focused tests passed after updating with current main.
  • Earlier synthetic current-main Testbox 29227656913: 230/230 focused tests passed.
  • Exact-head hosted CI 29230522584: passed.
  • Earlier full memory-core proof: 66 files / 1,020 tests passed.
  • Real QMD 1.1.6 proof through createMemorySearchTool: 28.356 seconds wall time with five QMD hits under the configured QMD-owned budget.
  • Fresh final autoreview: clean, 0.99 confidence.

Known proof gap: the live QMD scenario predates the final current-main merge; no PR runtime path changed after that proof.

No contributor changelog edit; release-note context is preserved in the PR body.

@steipete
steipete merged commit 2725742 into openclaw:main Jul 13, 2026
113 of 124 checks passed
@steipete

Copy link
Copy Markdown
Contributor

Merged via squash.

wm0018 pushed a commit to wm0018/openclaw that referenced this pull request Jul 14, 2026
* fix(memory): scope qmd search deadline

Preserve the phase-scoped QMD timeout semantics on current main.

Co-authored-by: Peter Steinberger <[email protected]>

* docs: refresh generated docs map

* test(memory): reconcile manager mock with current main

* test(memory): isolate deadline mock import

* style(memory): format search test mocks

* test(memory): decouple mock from deadline symbol

* chore(memory): refresh reviewed pull request head

---------

Co-authored-by: Benjamin Badejo <[email protected]>
Co-authored-by: Peter Steinberger <[email protected]>
Co-authored-by: Peter Steinberger <[email protected]>
Co-authored-by: Peter Steinberger <[email protected]>
github-actions Bot pushed a commit to Desicool/openclaw that referenced this pull request Jul 14, 2026
* fix(memory): scope qmd search deadline

Preserve the phase-scoped QMD timeout semantics on current main.

Co-authored-by: Peter Steinberger <[email protected]>

* docs: refresh generated docs map

* test(memory): reconcile manager mock with current main

* test(memory): isolate deadline mock import

* style(memory): format search test mocks

* test(memory): decouple mock from deadline symbol

* chore(memory): refresh reviewed pull request head

---------

Co-authored-by: Benjamin Badejo <[email protected]>
Co-authored-by: Peter Steinberger <[email protected]>
Co-authored-by: Peter Steinberger <[email protected]>
Co-authored-by: Peter Steinberger <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs Improvements or additions to documentation extensions: memory-core Extension: memory-core P2 Normal backlog priority with limited blast radius. size: XL triage: refactor-only Candidate: refactor/cleanup-only PR without maintainer context.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

memory_search 工具硬编码 15s timeout 不够 qmd query 跑完,建议可配置

2 participants