Skip to content

fix(kernel): unify auto_evolve and skill_workshop into single pending-draft pipeline #5798

Description

@DaBlitzStein

Problem

There are two independent skill auto-creation systems that confuse users and produce poor results separately:

  1. auto_evolve / background_skill_review — LLM reviewer runs after each turn. Generates good semantic content BUT:

    • 30s timeout too short for local models (Ollama/qwen) — most reviews time out
    • Review prompt is too conservative — almost always returns skip
    • Creates skills directly without user approval
  2. skill_workshop (PR feat(skill-workshop): closes #3328 — passive after-turn capture pipeline #4741 / Skill workshop: capture and refine reusable skills from successful agent interactions #3328) — Heuristic scanner detects repeated tool patterns, user corrections, explicit instructions. Has approval flow (pending drafts) BUT:

    • RepeatedToolPattern scanner generates useless drafts like tool_sequence_file_read_x5 with just tool names listed
    • No semantic understanding of what the user was doing

From a user perspective, these should be one system: the agent learns from interactions and proposes skills for human review.

Proposed fix

  1. Increase background_skill_review timeout from 30s to 120s (local models need more time)
  2. Lower the review threshold — add criteria like "3+ different tools in sequence" and "multi-step procedure that could be reused"
  3. Route create actions through skill_workshop::storage::save_candidate instead of evolution::create_skill — so LLM-proposed skills land in the Pending tab for human approval instead of being created directly
  4. Keep update/patch actions as direct operations (modifying skills the user already approved)

Result

  • One unified flow: agent turn → LLM review → pending draft with semantic content → human approval
  • No more useless tool_sequence_* drafts — the LLM generates meaningful names, descriptions, and procedures
  • Users see proposed skills in the Pending tab and decide what to keep

Refs: #3328

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/ciCI/CD and build toolingarea/kernelCore kernel (scheduling, RBAC, workflows)area/skillsSkill system and FangHub marketplacebugSomething isn't workinghas-prA pull request has been linked to this issueneeds-infoIssue lacks reproduction steps, logs, or version info

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions