Skip to content

memory-core: mergeHybridResults ignores textScore when keyword/vector chunk IDs don't overlap #92337

Description

@rrriiiccckkk

Issue: mergeHybridResults ignores textScore when keyword and vector results have non-overlapping chunk IDs

Summary

In hybrid memory search, mergeHybridResults only updates textScore when keyword and vector results share the same chunk ID. When they don't overlap (common with CJK/trigram queries), the keyword search results are effectively discarded — textScore stays 0 even though searchKeyword returned valid results.

Environment

  • OpenClaw version: 2026.6.5
  • FTS tokenizer: trigram
  • Embedding provider: Ollama (qwen3-embedding:4b)

Root Cause

In extensions/memory-core/src/host/search/manager.ts, mergeHybridResults merges keyword and vector results by chunk ID:

for (const r of params.vector) byId.set(r.id, {
    ...r,
    vectorScore: r.score,
    textScore: 0  // Initialize to 0
});
for (const r of params.keyword) {
    const existing = byId.get(r.id);
    if (existing) {
        existing.textScore = r.textScore;  // Only updates if ID matches!
    } else {
        byId.set(r.id, { ...r, vectorScore: 0, textScore: r.textScore });
    }
}

When keyword results have different chunk IDs than vector results:

  1. Vector results keep textScore: 0
  2. Keyword-only results are added with vectorScore: 0
  3. Final score = vectorWeight * 0 + textWeight * textScore → very low, often below minScore

Reproduction

  1. Configure agents.defaults.memorySearch.store.fts.tokenizer: "trigram"
  2. Index memory files containing Chinese text
  3. Run memory_search with a multi-character CJK query (e.g., "飞书插件配置")
  4. Observe: searchKeyword returns rows with usedMatch: true, but final results show textScore: 0

Logs confirm keyword search works:

[DEBUG] searchKeyword rows: 2 usedMatch: true matchQuery: "飞书插件配置" substringTerms: 6

But the final output:

{ "textScore": 0, "vectorScore": 0.597 }

Expected Behavior

Keyword search results should contribute to textScore even when chunk IDs don't perfectly overlap with vector results. Possible approaches:

  1. Union merge: Add keyword-only results as separate entries (already partially done, but they get filtered by minScore)
  2. Relaxed matching: Match on path + startLine range instead of exact chunk ID
  3. Score blending: When no ID overlap, blend keyword score into textScore based on content similarity

Impact

  • CJK queries with trigram tokenizer always have textScore: 0 in hybrid mode
  • English queries may also be affected when FTS5 and vector indexes produce different chunk boundaries
  • Vector search still works (vectorScore is correct), but the BM25 component contributes nothing to the final score

Metadata

Metadata

Assignees

No one assigned

    Labels

    P2Normal backlog priority with limited blast radius.clawsweeper:linked-pr-openClawSweeper found an open linked pull request for this issue.clawsweeper:no-new-fix-prClawSweeper does not recommend queueing a new automated fix PR for this issue.clawsweeper:source-reproClawSweeper found a high-confidence source-level issue reproduction.clownfishTracked by Clownfish automationimpact:session-stateSession, memory, transcript, context, or agent state can drift or corrupt.issue-rating: 🦞 diamond lobsterVery strong issue quality with high-confidence source-level or clear reproduction.

    Type

    No type

    Fields

    Priority

    None yet

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions