Skip to content

feat(skills): skill disambiguation false positive — process-management injected for memory-update queries #2513

@bug-ops

Description

@bug-ops

Description

During CI-351 live testing, skill disambiguation incorrectly selects the process-management skill with confidence 0.90 for queries like "Update my memory: project name is Orion and deadline is May 1st".

Root cause: The word "memory" in user queries matches the process-management skill description ("...CPU and memory consumption with ps, top, htop..."). The LLM disambiguation call reinforces this false semantic match.

Reproduction

Input: "Update my memory: the project name is Orion and deadline is May 1st"
Config: disambiguation_threshold = 0.05 (testing.toml)

Log output:

INFO zeph_core::agent::context::assembly: disambiguation selected skill skill=process-management confidence=0.8999999761581421

The agent then calls memory_save (correct tool) but the process-management skill is injected into the system prompt, adding irrelevant context and wasting tokens.

Related

Impact

The skill disambiguation selects semantically irrelevant skills due to keyword overlap between user input terms and skill descriptions. The agent still completes the task correctly (uses memory_save) but injects irrelevant skill context.

Proposed fix

  1. The process-management skill description should avoid generic terms like "memory consumption" that overlap with domain-specific memory management concepts.
  2. The disambiguation LLM prompt should include the user's actual intent context more explicitly.
  3. Consider increasing disambiguation_threshold default to reduce unnecessary disambiguation calls.

Metadata

Metadata

Assignees

Labels

P3Research — medium-high complexitybugSomething isn't workingmemoryzeph-memory crate (SQLite)skillszeph-skills crate

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions