-
Notifications
You must be signed in to change notification settings - Fork 2
feat(skills): skill disambiguation false positive — process-management injected for memory-update queries #2513
Description
Description
During CI-351 live testing, skill disambiguation incorrectly selects the process-management skill with confidence 0.90 for queries like "Update my memory: project name is Orion and deadline is May 1st".
Root cause: The word "memory" in user queries matches the process-management skill description ("...CPU and memory consumption with ps, top, htop..."). The LLM disambiguation call reinforces this false semantic match.
Reproduction
Input: "Update my memory: the project name is Orion and deadline is May 1st"
Config: disambiguation_threshold = 0.05 (testing.toml)
Log output:
INFO zeph_core::agent::context::assembly: disambiguation selected skill skill=process-management confidence=0.8999999761581421
The agent then calls memory_save (correct tool) but the process-management skill is injected into the system prompt, adding irrelevant context and wasting tokens.
Related
- CI-350 anomaly bug(skills): rust-agent-handoff skill injected at low confidence (0.14) hijacks general memory/update tasks #2512:
rust-agent-handoffat 0.141 for similar generic memory update prompts. This is a separate but related false-positive where disambiguation picks the wrong skill.
Impact
The skill disambiguation selects semantically irrelevant skills due to keyword overlap between user input terms and skill descriptions. The agent still completes the task correctly (uses memory_save) but injects irrelevant skill context.
Proposed fix
- The
process-managementskill description should avoid generic terms like "memory consumption" that overlap with domain-specific memory management concepts. - The disambiguation LLM prompt should include the user's actual intent context more explicitly.
- Consider increasing
disambiguation_thresholddefault to reduce unnecessary disambiguation calls.