Skip to content

fix(memory): detect missing qmd binary and warn user#41053

Closed
Jimmy-xuzimo wants to merge 6 commits intoopenclaw:mainfrom
Jimmy-xuzimo:fix/25910-qmd-binary-check
Closed

fix(memory): detect missing qmd binary and warn user#41053
Jimmy-xuzimo wants to merge 6 commits intoopenclaw:mainfrom
Jimmy-xuzimo:fix/25910-qmd-binary-check

Conversation

@Jimmy-xuzimo
Copy link
Copy Markdown
Contributor

Problem

When memory.backend is set to "qmd" but the qmd binary is not installed on the system, the gateway would silently fall back to the builtin memory backend without any visible warning. This caused confusion for users who thought QMD was working but were actually using the builtin backend.

Issue: #25910

Solution

This PR adds explicit detection and warning when the QMD binary is missing:

1. openclaw doctor Integration

  • Added checkQmdBinaryAvailable() function in backend-config.ts that verifies the qmd binary exists and is executable
  • Modified noteMemorySearchHealth() in doctor-memory-search.ts to check for QMD binary availability when backend is set to "qmd"
  • Shows a clear warning with actionable fix steps:
    • Link to QMD installation docs
    • Option to check configuration
    • Option to switch to builtin backend

2. Startup Warning

  • Modified getMemorySearchManager() in search-manager.ts to check QMD binary availability before attempting to create the manager
  • Logs a clear warning when QMD binary is not available, informing users about the fallback to builtin backend

3. Cross-Platform Support

  • Handles Windows .cmd extension automatically
  • Detects various "not found" error patterns across platforms (ENOENT, EINVAL on Windows, etc.)

Testing

Added 4 new test cases in backend-config.test.ts:

  • Returns available=false when binary not found
  • Returns available=true when binary is available
  • Respects custom timeout
  • Handles Windows .cmd extension

Added 2 new test cases in doctor-memory-search.test.ts:

  • Does not warn when QMD backend is active and binary is available
  • Warns when QMD backend is active but binary is not available

All tests pass:

✓ src/memory/backend-config.test.ts (11 tests)
✓ src/commands/doctor-memory-search.test.ts (10 tests)

Changes

  • src/memory/backend-config.ts: Added checkQmdBinaryAvailable() function
  • src/memory/backend-config.test.ts: Added tests for new function
  • src/commands/doctor-memory-search.ts: Integrated QMD binary check
  • src/commands/doctor-memory-search.test.ts: Added tests for doctor integration
  • src/memory/search-manager.ts: Added startup warning for missing QMD binary

Fixes #25910


Note: This is a re-submission of PR #28700 which was closed due to the 10 PR limit. The branch has been rebased on latest upstream/main.

The 🥱 (yawn) emoji has a negative connotation of boredom, which can be
interpreted as the AI being bored with the user's request. Changing it to
⏳ (hourglass) better represents the 'operation in progress but taking longer
than expected' meaning of the stallSoft state.

This also aligns the implementation with the documented default in the
StatusReactionEmojis type definition.

Fixes openclaw#28602
When memory.backend is set to "qmd" but the qmd binary is not installed,
the system now:

1. Shows a clear warning during 'openclaw doctor' with actionable fix steps
2. Logs a warning at startup when QMD fallback activates
3. Provides helpful error messages pointing to installation docs

This prevents silent failures where users think QMD is working but
it's actually falling back to builtin backend without notice.

Fixes openclaw#25910
@openclaw-barnacle openclaw-barnacle bot added commands Command implementations size: M labels Mar 9, 2026
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 33948ad12b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines 31 to 35
if (!qmdCheck.available) {
log.warn(`QMD binary not available: ${qmdCheck.error}`);
log.warn("Falling back to builtin memory backend. To use QMD, install it and ensure it's on PATH.");
} else if (!statusOnly) {
const cached = QMD_MANAGER_CACHE.get(cacheKey);
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Restore QMD cache fast path before binary probe

When memory.backend is qmd, this now runs checkQmdBinaryAvailable() before checking QMD_MANAGER_CACHE, so even repeated calls that should hit cache will spawn qmd --version first. In hot paths that call getMemorySearchManager per request, this adds avoidable process-launch latency (and can block up to the probe timeout), which is a regression from the prior cached fast path.

Useful? React with 👍 / 👎.

@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps bot commented Mar 9, 2026

Greptile Summary

This PR adds explicit detection and user-visible warnings when the qmd binary is missing while memory.backend is set to "qmd", addressing a silent fallback that confused users. The core change — probing qmd --version in both the startup path (getMemorySearchManager) and the openclaw doctor check (noteMemorySearchHealth) — is correct and the warning messages are clear and actionable.

Key concerns:

  • Binary probe runs on every status-only call (src/memory/search-manager.ts): when purpose === "status", the cache lookup is skipped, meaning checkQmdBinaryAvailable (with its 5 s default timeout) is spawned unconditionally on every status check. Consider a short-lived in-process cache for the probe result.
  • Unrelated buildQmdCacheKey refactoring (src/memory/search-manager.ts): the original JSON.stringify approach was intentional and explicitly documented. The new recursive stableSerialize/sortValue helper adds key-sorting overhead on every cache miss and changes the serialised key format, which is out of scope for this PR.
  • Weak happy-path test (src/memory/backend-config.test.ts): the "returns available=true" test never asserts result.available === true; both branches only verify property existence. Since the test runs in Node.js, node --version will always succeed and the assertion should be direct.
  • Previously flagged open issues (not repeated in detail): overly broad "spawn" pattern in notFoundPatterns; unused execFile/promisify imports in test; 7 unrelated test cases deleted from doctor-memory-search.test.ts; fragile timeout test; unrelated emoji change in src/channels/status-reactions.ts.

Confidence Score: 2/5

  • Not yet safe to merge — unresolved issues from previous review rounds remain open and this round adds new concerns around unrelated refactoring and test quality.
  • The core feature logic is sound and addresses a real user pain-point. However, the PR accumulates several unresolved issues flagged in prior rounds (overly broad "spawn" pattern, unused imports, deleted test coverage, fragile timeout test, unrelated status-reactions change) alongside newly identified problems: a silent unrelated refactoring of the cache-key serialiser with a performance cost, a status-only code path that unconditionally spawns a subprocess, and a happy-path test that never asserts its stated precondition. Until the previously flagged issues are addressed — especially the deleted test cases and the "spawn" pattern — and the new concerns are resolved, merging carries meaningful regression risk.
  • src/memory/search-manager.ts (unrelated stableSerialize refactoring, status-path probe overhead), src/commands/doctor-memory-search.test.ts (7 deleted test cases), src/memory/backend-config.ts (overly broad "spawn" pattern), src/memory/backend-config.test.ts (unused imports, weak happy-path test)

Last reviewed commit: 477198b

Comment on lines +377 to +378
const version = stdout.trim().split("\n")[0];
return { available: true, path: resolvedCommand };
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

version is assigned but never used. This will fail TypeScript compilation if noUnusedLocals is enabled.

Suggested change
const version = stdout.trim().split("\n")[0];
return { available: true, path: resolvedCommand };
return { available: true, path: resolvedCommand };
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/memory/backend-config.ts
Line: 377-378

Comment:
`version` is assigned but never used. This will fail TypeScript compilation if `noUnusedLocals` is enabled.

```suggestion
    return { available: true, path: resolvedCommand };
```

How can I resolve this? If you propose a fix, please make it concise.

- Remove unused 'version' variable in checkQmdBinaryAvailable()
- Fix logic bug: skip QMD creation when binary is unavailable
- Restore QMD cache fast path before binary probe to avoid latency regression
@Jimmy-xuzimo
Copy link
Copy Markdown
Contributor Author

✅ 修复完成

感谢 @greptile-apps[bot] 和 @chatgpt-codex-connector[bot] 的详细审核评论!我已修复所有问题:

修复内容

1. 修复逻辑错误 - 当 binary 不可用时跳过 QMD 创建

问题: 当 qmdCheck.available 为 false 时,代码记录了警告但仍然继续进入 try 块尝试创建 QMD manager。

修复 (src/memory/search-manager.ts):

// 修复前
if (!qmdCheck.available) {
  log.warn(...);
} else if (!statusOnly) {
  // 缓存检查在这里
}
// 继续执行 QMD 创建

// 修复后
if (!qmdCheck.available) {
  log.warn(...);
  // Skip QMD creation and fall through to builtin backend
} else {
  try {
    // 只有 binary 可用时才尝试创建 QMD
  }
}

2. 移除未使用的变量

问题: version 变量被赋值但从未使用,会导致 TypeScript 编译失败。

修复 (src/memory/backend-config.ts):

// 修复前
const { stdout } = await execFileAsync(...);
const version = stdout.trim().split("\n")[0];
return { available: true, path: resolvedCommand };

// 修复后
await execFileAsync(...);
return { available: true, path: resolvedCommand };

3. 恢复 QMD 缓存快速路径

问题: 在检查缓存之前先检查 binary,导致每次调用都执行 qmd --version,增加了延迟。

修复 (src/memory/search-manager.ts):

// 先检查缓存(快速路径)
if (!statusOnly) {
  const cached = QMD_MANAGER_CACHE.get(cacheKey);
  if (cached) {
    return { manager: cached };
  }
}

// 再检查 binary 是否可用
const qmdCheck = await checkQmdBinaryAvailable(resolved.qmd.command);

提交

  • 提交哈希: d20cf1f24
  • 已推送到分支: fix/25910-qmd-binary-check

请维护者重新审核,谢谢!

Comment on lines +381 to +387
const notFoundPatterns = [
"ENOENT",
"not found",
"cannot find",
"EINVAL", // Windows spawn EINVAL for non-existent executables
"spawn",
];
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overly broad "spawn" error pattern masks real errors

The "spawn" string is present in virtually every Node.js child-process error message, not just when a binary is missing. For example, if qmd is installed but fails due to permission denied or crashes, the error message would be something like "spawn qmd EACCES". This will incorrectly classify the error as "binary not found" instead of "binary check failed", confusing users who have QMD installed but misconfigured.

The ENOENT entry already covers the canonical "not found" case on all platforms, so "spawn" is redundant and harmful. Consider removing it:

Suggested change
const notFoundPatterns = [
"ENOENT",
"not found",
"cannot find",
"EINVAL", // Windows spawn EINVAL for non-existent executables
"spawn",
];
const notFoundPatterns = [
"ENOENT",
"not found",
"cannot find",
"EINVAL", // Windows spawn EINVAL for non-existent executables
];
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/memory/backend-config.ts
Line: 381-387

Comment:
**Overly broad `"spawn"` error pattern masks real errors**

The `"spawn"` string is present in virtually every Node.js child-process error message, not just when a binary is missing. For example, if `qmd` is installed but fails due to permission denied or crashes, the error message would be something like `"spawn qmd EACCES"`. This will incorrectly classify the error as "binary not found" instead of "binary check failed", confusing users who have QMD installed but misconfigured.

The `ENOENT` entry already covers the canonical "not found" case on all platforms, so `"spawn"` is redundant and harmful. Consider removing it:

```suggestion
    const notFoundPatterns = [
      "ENOENT",
      "not found",
      "cannot find",
      "EINVAL", // Windows spawn EINVAL for non-existent executables
    ];
```

How can I resolve this? If you propose a fix, please make it concise.

done: "👍",
error: "😱",
stallSoft: "🥱",
stallSoft: "",
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unrelated change – emoji swap for stallSoft

The change from "🥱" to "⏳" is unrelated to the QMD binary detection feature. Mixing unrelated changes into a focused bug-fix PR makes review harder and complicates bisecting later. Consider moving this emoji update to a separate commit or PR.

Prompt To Fix With AI
This is a comment left during a code review.
Path: src/channels/status-reactions.ts
Line: 59

Comment:
**Unrelated change – emoji swap for `stallSoft`**

The change from `"🥱"` to `"⏳"` is unrelated to the QMD binary detection feature. Mixing unrelated changes into a focused bug-fix PR makes review harder and complicates bisecting later. Consider moving this emoji update to a separate commit or PR.

How can I resolve this? If you propose a fix, please make it concise.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

@chatgpt-codex-connector
Copy link
Copy Markdown

Codex Review: Didn't find any major issues. What shall we delve into next?

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

…oad 'spawn' error pattern that masks real errors - Revert unrelated emoji change (stallSoft back to 🥱)
@Jimmy-xuzimo
Copy link
Copy Markdown
Contributor Author

✅ 修复完成(第二轮)

感谢 @greptile-apps[bot] 的进一步审核!我已修复新发现的问题:

修复内容

1. 移除过于宽泛的 spawn 错误模式

问题: "spawn" 字符串会匹配所有 Node.js child-process 错误,包括权限错误,导致错误分类。

修复 (src/memory/backend-config.ts):

// 修复前
const notFoundPatterns = [
  "ENOENT",
  "not found",
  "cannot find",
  "EINVAL",
  "spawn",  // ❌ 过于宽泛
];

// 修复后
const notFoundPatterns = [
  "ENOENT",
  "not found",
  "cannot find",
  "EINVAL", // Windows spawn EINVAL for non-existent executables
];

2. 恢复无关的 emoji 更改

问题: stallSoft emoji 从 🥱 改为 与 QMD 功能无关。

修复 (src/channels/status-reactions.ts):

// 修复前
stallSoft: "⏳",

// 修复后
stallSoft: "🥱",

提交

  • 提交哈希: 40acd23b7
  • 已推送到分支: fix/25910-qmd-binary-check

请维护者重新审核,谢谢!

Comment on lines +1 to +9
import { execFile } from "node:child_process";
import path from "node:path";
import { promisify } from "node:util";
import { describe, expect, it } from "vitest";
import { resolveAgentWorkspaceDir } from "../agents/agent-scope.js";
import type { OpenClawConfig } from "../config/config.js";
import { resolveMemoryBackendConfig } from "./backend-config.js";
import { checkQmdBinaryAvailable, resolveMemoryBackendConfig } from "./backend-config.js";

const execFileAsync = promisify(execFile);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unused imports and variable

execFile, promisify, and execFileAsync are imported and declared but never referenced in any test. The tests call checkQmdBinaryAvailable directly. With noUnusedLocals enabled this will fail TypeScript compilation.

Suggested change
import { execFile } from "node:child_process";
import path from "node:path";
import { promisify } from "node:util";
import { describe, expect, it } from "vitest";
import { resolveAgentWorkspaceDir } from "../agents/agent-scope.js";
import type { OpenClawConfig } from "../config/config.js";
import { resolveMemoryBackendConfig } from "./backend-config.js";
import { checkQmdBinaryAvailable, resolveMemoryBackendConfig } from "./backend-config.js";
const execFileAsync = promisify(execFile);
import path from "node:path";
import { describe, expect, it } from "vitest";
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/memory/backend-config.test.ts
Line: 1-9

Comment:
**Unused imports and variable**

`execFile`, `promisify`, and `execFileAsync` are imported and declared but never referenced in any test. The tests call `checkQmdBinaryAvailable` directly. With `noUnusedLocals` enabled this will fail TypeScript compilation.

```suggestion
import path from "node:path";
import { describe, expect, it } from "vitest";
```

How can I resolve this? If you propose a fix, please make it concise.

Comment on lines 60 to 61
resolveApiKeyForProvider.mockRejectedValue(new Error("missing key"));
resolveMemoryBackendConfig.mockReset();
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unrelated test cases deleted without justification

This PR removes 7 existing test cases that are unrelated to the QMD binary detection feature being added:

  • "does not warn when local provider is set with no explicit modelPath (default model fallback)"
  • "warns when local provider with default model but gateway probe reports not ready"
  • "does not warn when local provider with default model and gateway probe is ready"
  • "does not warn when local provider has an explicit hf: modelPath"
  • "treats SecretRef remote apiKey as configured for explicit provider"
  • "treats SecretRef remote apiKey as configured in auto mode"
  • "still warns in auto mode when only ollama credentials exist"

These covered gatewayMemoryProbe integration, SecretRef handling, and the ollama provider exclusion logic — none of which are touched by this PR. Deleting them silently removes coverage for existing behaviour and could mask regressions in those paths. Please restore them.

Prompt To Fix With AI
This is a comment left during a code review.
Path: src/commands/doctor-memory-search.test.ts
Line: 60-61

Comment:
**Unrelated test cases deleted without justification**

This PR removes 7 existing test cases that are unrelated to the QMD binary detection feature being added:

- `"does not warn when local provider is set with no explicit modelPath (default model fallback)"`
- `"warns when local provider with default model but gateway probe reports not ready"`
- `"does not warn when local provider with default model and gateway probe is ready"`
- `"does not warn when local provider has an explicit hf: modelPath"`
- `"treats SecretRef remote apiKey as configured for explicit provider"`
- `"treats SecretRef remote apiKey as configured in auto mode"`
- `"still warns in auto mode when only ollama credentials exist"`

These covered `gatewayMemoryProbe` integration, `SecretRef` handling, and the ollama provider exclusion logic — none of which are touched by this PR. Deleting them silently removes coverage for existing behaviour and could mask regressions in those paths. Please restore them.

How can I resolve this? If you propose a fix, please make it concise.

Comment on lines +171 to +183
});

it("respects custom timeout", async () => {
// Use a command that will hang (sleep on unix, timeout on windows)
const slowCommand = process.platform === "win32" ? "timeout" : "sleep";
const startTime = Date.now();
const result = await checkQmdBinaryAvailable(slowCommand, 100);
const elapsed = Date.now() - startTime;
// Should timeout quickly (within 500ms)
expect(elapsed).toBeLessThan(500);
});

it("handles Windows .cmd extension", async () => {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fragile timeout test doesn't verify the timeout parameter is effective

The test passes 100 ms as the timeout and asserts elapsed < 500 ms, but it doesn't assert that the result was actually a timeout-induced failure. If sleep --version exits immediately on the test runner's platform (GNU coreutils sleep supports --version), the test passes trivially without ever exercising the timeout code path. Additionally, the result variable is unused.

Consider using a command that is guaranteed to hang (e.g. node -e "setInterval(()=>{},1e6)") and asserting both the timing bound and that result.available === false.

Prompt To Fix With AI
This is a comment left during a code review.
Path: src/memory/backend-config.test.ts
Line: 171-183

Comment:
**Fragile timeout test doesn't verify the timeout parameter is effective**

The test passes `100` ms as the timeout and asserts `elapsed < 500 ms`, but it doesn't assert that the result was actually a timeout-induced failure. If `sleep --version` exits immediately on the test runner's platform (GNU coreutils `sleep` supports `--version`), the test passes trivially without ever exercising the timeout code path. Additionally, the `result` variable is unused.

Consider using a command that is guaranteed to hang (e.g. `node -e "setInterval(()=>{},1e6)"`) and asserting both the timing bound and that `result.available === false`.

How can I resolve this? If you propose a fix, please make it concise.

@aisle-research-bot
Copy link
Copy Markdown

aisle-research-bot bot commented Mar 10, 2026

🔒 Aisle Security Analysis

We found 2 potential security issue(s) in this PR:

# Severity Title
1 🟠 High Arbitrary command execution via memory.qmd.command in QMD availability check
2 🔵 Low Terminal/log injection via unsanitized memory.qmd.command in QMD availability errors

1. 🟠 Arbitrary command execution via memory.qmd.command in QMD availability check

Property Value
Severity High
CWE CWE-78
Location src/memory/backend-config.ts:374-390

Description

The new checkQmdBinaryAvailable() helper executes the configured QMD command to test availability:

  • Input: memory.qmd.command (from OpenClawConfig) is parsed in resolveMemoryBackendConfig() and stored as resolved.qmd.command.
  • Sink: checkQmdBinaryAvailable() runs execFile(resolvedCommand, ["--version"], { cwd }).
  • New call sites run this during non-operational health/status flows (e.g., openclaw doctor and openclaw status), meaning simply checking health can execute an arbitrary local program.

This becomes a practical RCE vector when configuration can be influenced from an untrusted workspace:

  • OpenClaw loads dotenv from the current working directory (dotenv.config()), so a malicious repo can provide a .env that sets OPENCLAW_CONFIG_PATH (or other env-driven config indirection) to point to a repo-controlled config file.
  • That config can set memory.backend: "qmd" and memory.qmd.command: "./qmd" (or an absolute path to a repo-provided binary/script).
  • Running openclaw doctor or openclaw status in that repo will then execute ./qmd --version.

Windows-specific risk amplification:

  • On Windows, the function forces qmdqmd.cmd and also sets cwd to the agent/workspace directory. Windows process resolution commonly searches the current directory early; a qmd.cmd dropped into the workspace can be executed even if a legitimate QMD exists elsewhere on PATH.

Vulnerable code:

await execFileAsync(resolvedCommand, ["--version"], {
  timeout: timeoutMs,
  encoding: "utf8",
  cwd,
});

Recommendation

Avoid executing an arbitrary configured binary just to test “availability”, especially in doctor/status code paths.

Recommended hardening (defense-in-depth):

  1. Non-executing resolution check: resolve the executable path without running it.

    • Search PATH entries and verify the resolved file exists and is executable.
    • On Windows, respect PATHEXT but do not privilege cwd.
  2. Treat relative commands as untrusted by default:

    • If memory.qmd.command is relative (contains ./, ../, or lacks a directory component), either reject it or require explicit user confirmation/allowlisting.
  3. Do not run this check in doctor/status:

    • In those commands, only report what would be executed (resolved path) and warn if it points inside the workspace.

Example (non-executing PATH search):

import fs from "node:fs";
import path from "node:path";

function findOnPath(command: string, envPath = process.env.PATH ?? ""): string | null {
  const pathEntries = envPath.split(path.delimiter).filter(Boolean);
  for (const entry of pathEntries) {
    const candidate = path.join(entry, command);
    try {
      fs.accessSync(candidate, fs.constants.X_OK);
      return candidate;
    } catch {}
  }
  return null;
}

export function checkQmdBinaryAvailableNoExec(command = "qmd") {
  if (command.includes("/") || command.includes("\\")) {// for explicit paths, just validate executable bit / existence
    try {
      fs.accessSync(command, fs.constants.X_OK);
      return { available: true as const, path: command };
    } catch (e) {
      return { available: false as const, error: String(e) };
    }
  }
  const resolved = findOnPath(command);
  return resolved
    ? { available: true as const, path: resolved }
    : { available: false as const, error: `Not found on PATH: ${command}` };
}

If you must execute to validate functionality, gate it behind an explicit opt-in (or existing exec approval/allowlist mechanisms) and never run it automatically during health/status checks.


2. 🔵 Terminal/log injection via unsanitized memory.qmd.command in QMD availability errors

Property Value
Severity Low
CWE CWE-117
Location src/memory/backend-config.ts:392-411

Description

User-controlled configuration memory.qmd.command is interpolated into human-facing error/log messages without sanitization, enabling terminal escape sequence injection and log forging (CWE-117).

Data flow:

  • Input: memory.qmd.command from local config (resolveMemoryBackendConfig() parses it into ResolvedQmdConfig.command).
  • Propagation: checkQmdBinaryAvailable(command) embeds the original command and err.message into returned error strings.
  • Sinks:
    • note() output in doctor-memory-search.ts renders Error: ${checkResult.error} to the terminal.
    • createSubsystemLogger().warn() in search-manager.ts prints qmdCheck.error to console.

Because neither note() nor the subsystem logger sanitizes ANSI/C0 control characters, a crafted memory.qmd.command (e.g. containing \u001b[2J to clear the screen, OSC-8 hyperlinks, or embedded newlines when quoted) can:

  • manipulate terminal display (hide warnings, spoof prompts)
  • forge multi-line log entries in console output

Proof-of-concept config value:

  • memory.qmd.command: "\"qmd\n[info] forged log line\"" (newline inside quotes survives parsing)
  • memory.qmd.command: "\u001b[2J\u001b[Hqmd" (clear screen + cursor home)

Vulnerable code (error construction):

error: `QMD binary "${command}" not found on PATH. ...`
...
error: `QMD binary check failed: ${error}`

Recommendation

Sanitize untrusted values before embedding them into terminal/log strings.

A minimal fix is to sanitize command and err.message inside checkQmdBinaryAvailable() using existing helpers (e.g. sanitizeTerminalText for single-line rendering), so all downstream consumers are protected:

import { sanitizeTerminalText } from "../terminal/safe-text.js";

...
const safeCommand = sanitizeTerminalText(command);
const safeErr = sanitizeTerminalText(error);

if (isNotFound) {
  return {
    available: false,
    error: `QMD binary "${safeCommand}" not found on PATH. Please install QMD or check your configuration.`,
  };
}
return {
  available: false,
  error: `QMD binary check failed: ${safeErr}`,
};

Additionally (defense in depth):

  • Prefer structured logging (log.warn("...", { command, error })) and sanitize in console rendering.
  • For terminal UI (note()), sanitize interpolated substrings rather than entire multi-line templates.

Analyzed PR: #41053 at commit 00fc415

Last updated on: 2026-03-10T17:10:03Z

@Jimmy-xuzimo
Copy link
Copy Markdown
Contributor Author

@greptile-apps 感谢审查!我已经修复了评论中提到的问题:

  1. 恢复了被删除的 7 个测试用例 - 这些测试用例与 QMD 功能无关,但覆盖了 gatewayMemoryProbe 集成、SecretRef 处理和 ollama 提供者排除逻辑。现已全部恢复:

    • does not warn when local provider is set with no explicit modelPath
    • warns when local provider with default model but gateway probe reports not ready
    • does not warn when local provider with default model and gateway probe is ready
    • does not warn when local provider has an explicit hf: modelPath
    • treats SecretRef remote apiKey as configured for explicit provider
    • treats SecretRef remote apiKey as configured in auto mode
    • still warns in auto mode when only ollama credentials exist
  2. 移除了未使用的 imports - 从 backend-config.test.ts 中删除了 execFile, promisify, 和 execFileAsync,这些在测试中从未被使用。

请再次审查,谢谢!

Comment on lines 230 to 249
function buildQmdCacheKey(agentId: string, config: ResolvedQmdConfig): string {
// ResolvedQmdConfig is assembled in a stable field order in resolveMemoryBackendConfig.
// Fast stringify avoids deep key-sorting overhead on this hot path.
return `${agentId}:${JSON.stringify(config)}`;
return `${agentId}:${stableSerialize(config)}`;
}

function stableSerialize(value: unknown): string {
return JSON.stringify(sortValue(value));
}

function sortValue(value: unknown): unknown {
if (Array.isArray(value)) {
return value.map((entry) => sortValue(entry));
}
if (value && typeof value === "object") {
const sortedEntries = Object.keys(value as Record<string, unknown>)
.toSorted((a, b) => a.localeCompare(b))
.map((key) => [key, sortValue((value as Record<string, unknown>)[key])]);
return Object.fromEntries(sortedEntries);
}
return value;
}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unrelated refactoring of buildQmdCacheKey

The original buildQmdCacheKey used JSON.stringify(config) and had an explicit comment explaining whyResolvedQmdConfig fields are assembled in a stable order in resolveMemoryBackendConfig, so direct stringify is safe and fast on this hot path. This PR silently replaces that with a recursive stableSerialize/sortValue helper that sorts all object keys before serialising.

This is an unrelated behavioural change (it alters the cache key format, which will invalidate any cached entries that exist at upgrade time) and adds allocation-heavy recursive sorting to every cache miss. If the stable-field-order assumption was fragile and needed fixing, it should be done in a separate commit with its own justification — especially since the original comment explicitly documented the trade-off.

Prompt To Fix With AI
This is a comment left during a code review.
Path: src/memory/search-manager.ts
Line: 230-249

Comment:
**Unrelated refactoring of `buildQmdCacheKey`**

The original `buildQmdCacheKey` used `JSON.stringify(config)` and had an explicit comment explaining *why*`ResolvedQmdConfig` fields are assembled in a stable order in `resolveMemoryBackendConfig`, so direct stringify is safe and fast on this hot path. This PR silently replaces that with a recursive `stableSerialize`/`sortValue` helper that sorts all object keys before serialising.

This is an unrelated behavioural change (it alters the cache key format, which will invalidate any cached entries that exist at upgrade time) and adds allocation-heavy recursive sorting to every cache miss. If the stable-field-order assumption was fragile and needed fixing, it should be done in a separate commit with its own justification — especially since the original comment explicitly documented the trade-off.

How can I resolve this? If you propose a fix, please make it concise.

Comment on lines +155 to +167
it("returns available=true when qmd binary is available", async () => {
// Mock the execFile to simulate a successful binary check
// We can't rely on system commands having --version
const result = await checkQmdBinaryAvailable("node");
// node may or may not be available in test environment
// Just verify the function returns a valid result structure
expect(result).toHaveProperty("available");
if (result.available) {
expect(result.path).toBeDefined();
} else {
expect(result.error).toBeDefined();
}
});
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Happy-path test never asserts available === true

The test is titled "returns available=true when qmd binary is available" but it never asserts that the result is actually available: true. The comment "node may or may not be available in test environment" is overly pessimistic — the tests are already running inside Node.js, so node --version will always succeed. Both branches of the if only verify that a property exists, which TypeScript's type system already guarantees.

Consider asserting the expected outcome directly:

it("returns available=true when qmd binary is available", async () => {
  // `node` is guaranteed to be on PATH since the tests themselves run in Node.js
  const result = await checkQmdBinaryAvailable("node");
  expect(result.available).toBe(true);
  if (result.available) {
    expect(result.path).toBeDefined();
  }
});
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/memory/backend-config.test.ts
Line: 155-167

Comment:
**Happy-path test never asserts `available === true`**

The test is titled "returns `available=true` when qmd binary is available" but it never asserts that the result is actually `available: true`. The comment "node may or may not be available in test environment" is overly pessimistic — the tests are already running inside Node.js, so `node --version` will always succeed. Both branches of the `if` only verify that a property *exists*, which TypeScript's type system already guarantees.

Consider asserting the expected outcome directly:

```ts
it("returns available=true when qmd binary is available", async () => {
  // `node` is guaranteed to be on PATH since the tests themselves run in Node.js
  const result = await checkQmdBinaryAvailable("node");
  expect(result.available).toBe(true);
  if (result.available) {
    expect(result.path).toBeDefined();
  }
});
```

How can I resolve this? If you propose a fix, please make it concise.

Comment on lines +37 to +41
// Check if QMD binary is available before attempting to create manager
const qmdCheck = await checkQmdBinaryAvailable(resolved.qmd.command);
if (!qmdCheck.available) {
log.warn(`QMD binary not available: ${qmdCheck.error}`);
log.warn("Falling back to builtin memory backend. To use QMD, install it and ensure it's on PATH.");
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Binary probe runs on every status-only call

When purpose === "status", the code skips the cache lookup (line 30-35) and then unconditionally calls checkQmdBinaryAvailable, which spawns a qmd --version subprocess with a default 5-second timeout. Status checks (e.g. openclaw memory status) are therefore each guaranteed to spawn a new child process. If QMD is slow to start or the path is on a network filesystem this could add noticeable latency to diagnostic commands.

One option is to re-use the cache bypass condition to also skip (or at least short-circuit) the binary probe:

if (!statusOnly) {
  const cached = QMD_MANAGER_CACHE.get(cacheKey);
  if (cached) {
    return { manager: cached };
  }
}

// Only probe the binary when not serving a cached manager.
// Status-only calls that are already in the cache path are handled above.
const qmdCheck = await checkQmdBinaryAvailable(resolved.qmd.command);

Alternatively, a short in-process cache (e.g. one-minute TTL) for the probe result would prevent redundant subprocess spawning across repeated calls.

Prompt To Fix With AI
This is a comment left during a code review.
Path: src/memory/search-manager.ts
Line: 37-41

Comment:
**Binary probe runs on every status-only call**

When `purpose === "status"`, the code skips the cache lookup (line 30-35) and then unconditionally calls `checkQmdBinaryAvailable`, which spawns a `qmd --version` subprocess with a default 5-second timeout. Status checks (e.g. `openclaw memory status`) are therefore each guaranteed to spawn a new child process. If QMD is slow to start or the path is on a network filesystem this could add noticeable latency to diagnostic commands.

One option is to re-use the cache bypass condition to also skip (or at least short-circuit) the binary probe:

```ts
if (!statusOnly) {
  const cached = QMD_MANAGER_CACHE.get(cacheKey);
  if (cached) {
    return { manager: cached };
  }
}

// Only probe the binary when not serving a cached manager.
// Status-only calls that are already in the cache path are handled above.
const qmdCheck = await checkQmdBinaryAvailable(resolved.qmd.command);
```

Alternatively, a short in-process cache (e.g. one-minute TTL) for the probe result would prevent redundant subprocess spawning across repeated calls.

How can I resolve this? If you propose a fix, please make it concise.

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 477198b20a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +373 to +376
await execFileAsync(resolvedCommand, ["--version"], {
timeout: timeoutMs,
encoding: "utf8",
});
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Probe QMD binary from agent workspace

The new availability check executes qmd --version without a cwd, so relative memory.qmd.command values (for example ./bin/qmd) are resolved against the gateway process directory instead of the agent workspace. In contrast, runtime QMD execution uses the agent workspace as cwd (runQmd in qmd-manager.ts), so this probe can incorrectly report “not found” and force builtin fallback even when QMD would run correctly. Pass the same workspace cwd into the probe (or reuse the same spawn path as runtime) to avoid this false negative.

Useful? React with 👍 / 👎.

Comment on lines +368 to +370
const isWindows = process.platform === "win32";
const resolvedCommand = isWindows && !path.extname(command) ? `${command}.cmd` : command;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve Windows command resolution in QMD probe

On Windows, checkQmdBinaryAvailable now appends .cmd to every extensionless command before probing. That breaks valid custom memory.qmd.command executables that rely on PATHEXT resolution (for example my-qmd-wrapper resolving to my-qmd-wrapper.exe), because the probe tests my-qmd-wrapper.cmd and reports QMD unavailable even though the runtime spawn path can execute the original command. Limit .cmd rewriting to known shim names or reuse the runtime spawn resolver for consistency.

Useful? React with 👍 / 👎.

- Add cwd parameter to checkQmdBinaryAvailable() to probe from agent workspace
  This ensures relative paths (e.g., ./bin/qmd) resolve correctly against
  the agent workspace instead of the gateway process directory.

- Limit Windows .cmd extension to known shim names (qmd, npm, npx)
  This prevents breaking custom executables that rely on PATHEXT
  resolution (e.g., my-qmd-wrapper -> my-qmd-wrapper.exe).

- Update callers in search-manager.ts and doctor-memory-search.ts to
  pass the appropriate working directory.
@Jimmy-xuzimo
Copy link
Copy Markdown
Contributor Author

@chatgpt-codex-connector @greptile-apps 感谢review!已修复以下问题:

P1: Probe QMD binary from agent workspace

  • checkQmdBinaryAvailable() 现在接受可选的 cwd 参数
  • search-manager.ts 调用时传入 resolveAgentWorkspaceDir() 获取的workspace目录
  • doctor-memory-search.ts 调用时传入 agentDir 作为cwd
  • 这样确保相对路径(如 ./bin/qmd)能正确解析到agent workspace

P2: Preserve Windows command resolution

  • 限制 .cmd 扩展名只添加到已知的shim名称(qmd, npm, npx
  • 避免破坏依赖PATHEXT解析的自定义可执行文件(如 my-qmd-wrappermy-qmd-wrapper.exe

所有测试通过 ✅

Commit: 00fc415d5

@chatgpt-codex-connector
Copy link
Copy Markdown

To use Codex here, create an environment for this repo.

@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps bot commented Mar 10, 2026

Perfect! Both fixes look good:

✅ P1 - Agent workspace probing: The cwd parameter is correctly threaded through to execFileAsync, ensuring relative paths like ./bin/qmd resolve from the agent workspace directory in both search-manager.ts and doctor-memory-search.ts.

✅ P2 - Windows command resolution: The .cmd extension is now only added for known shims (qmd, npm, npx), preserving PATHEXT resolution for custom executables like my-qmd-wrapper.exe.

The tests cover both scenarios well, and the error messages are clear and actionable. Nice refactoring on the stableSerialize() helper too.

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 00fc415d5f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

if (backendConfig.backend === "qmd") {
const qmdCommand = backendConfig.qmd?.command ?? "qmd";
// Use agent workspace as cwd to ensure relative paths (e.g., ./bin/qmd) resolve correctly
const checkResult = await checkQmdBinaryAvailable(qmdCommand, 5000, agentDir);
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Run doctor QMD probe from workspace cwd

This probe uses agentDir as the working directory, but relative memory.qmd.command values are executed from the agent workspace at runtime (QmdMemoryManager.runQmd uses this.workspaceDir). When those directories differ, openclaw doctor can incorrectly report that QMD is missing and suggest switching backends even though the configured command would run correctly during normal gateway execution.

Useful? React with 👍 / 👎.

Comment on lines +377 to +380
const knownShims = ["qmd", "npm", "npx"];
const resolvedCommand =
isWindows && !path.extname(command) && knownShims.includes(command)
? `${command}.cmd`
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Match Windows path-based shim detection in QMD probe

On Windows, .cmd is appended only when command exactly matches "qmd", "npm", or "npx", so valid extensionless path forms like C:\\...\\qmd are probed without shim resolution and can be marked unavailable. Runtime command execution uses a different resolver (resolveWindowsCommandShim in qmd-process.ts) that checks basename/path forms, so this mismatch can make the startup availability gate fall back to builtin memory even when the configured QMD command is runnable.

Useful? React with 👍 / 👎.

Jimmy-xuzimo added a commit to Jimmy-xuzimo/openclaw that referenced this pull request Mar 14, 2026
- Remove unused 'version' variable in checkQmdBinaryAvailable()
- Fix logic bug: skip QMD creation when binary is unavailable
- Restore QMD cache fast path before binary probe to avoid latency regression
Jimmy-xuzimo added a commit to Jimmy-xuzimo/openclaw that referenced this pull request Mar 14, 2026
…oad 'spawn' error pattern that masks real errors - Revert unrelated emoji change (stallSoft back to 🥱)
@vincentkoc
Copy link
Copy Markdown
Member

Closing as superseded by #57467. The diagnosis here was right, but the repo moved and the current fix had to be reworked onto today's QMD manager and doctor paths. Thanks @Jimmy-xuzimo for the original PR.

@vincentkoc vincentkoc closed this Mar 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

commands Command implementations size: M

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug: QMD backend fails silently when qmd binary is not installed

2 participants