Skip to content

Skill workshop: capture and refine reusable skills from successful agent interactions #3328

Description

@houko

Problem

Today librefang-skills only loads static, author-time-defined skills (TOML manifest + Python / JS / Prompt body). There is no path for an agent to learn a new skill from a successful interaction — when a user spends 20 turns teaching the agent a multi-step workflow, that workflow is lost the moment the conversation ends. The next time the user wants the same thing, they teach it again.

Hands give us a packaging primitive (multi-agent, role-keyed, persisted overrides), but a hand is author-time too. There is no in-loop capture / refinement / promotion path.

Reference: openclaw

extensions/skill-workshop captures repeatable workflows as workspace skills with:

  • autoCapture: detect user corrections ("no, do it like X") and reusable workflow instructions ("from now on, always Y") after agent turns; surface as a candidate skill draft.
  • approvalPolicy: store learned skill changes as pending (default) or write them automatically. Pending suggestions show up in a workshop UI for the user to approve / edit / reject.
  • reviewMode: heuristic (cheap pattern match), threshold-LLM (call a cheap model to confirm), both (heuristic gate → LLM review), none (manual capture only).
  • maxPending: cap on pending suggestions per workspace to avoid drift.
  • Safe writes: every promoted skill goes through the same SHA256 / prompt-injection scan as marketplace skills before being loaded.
  • Prompt refresh: when a skill is promoted, the system prompt is rebuilt so the agent picks it up next turn — no restart.

Proposed approach

This is a needs-rfc size feature. Outline:

  1. New crate-internal module librefang-skills::workshop:

    • CandidateSkill struct (draft TOML + extracted body + provenance: agent ID, session ID, turn range).
    • CaptureSource enum: UserCorrection, ExplicitInstruction, RepeatedToolPattern.
  2. Capture pipeline runs after_turn (paired with the before_prompt_build hook from Plugin-style before_prompt_build hook for system context injection #3326):

  3. Storage: ~/.librefang/skills/pending/<workspace>/<uuid>.toml with a status = "pending" field. The existing skill loader skips pending skills.

  4. Approval surface:

    • Dashboard page listing pending skills, diff against existing skills, approve / reject / edit.
    • CLI: librefang skill pending list / approve <id> / reject <id>.
  5. Approval promotes: validated TOML → moves to ~/.librefang/skills/active/, kernel reloads skill registry, prompt rebuilds.

  6. Per-agent agent.toml:

    [skill_workshop]
    enabled = false                     # default off — opt in
    auto_capture = true
    approval_policy = "pending"         # pending | auto
    review_mode = "heuristic"           # heuristic | threshold_llm | both | none
    max_pending = 20

Acceptance criteria

  • RFC document committed under docs/rfcs/skill-workshop.md covering capture heuristics, security model (prompt-injection scan path), and rollback.
  • Capture pipeline implemented behind [skill_workshop] enabled = true, default off.
  • At least the heuristic review mode lands; LLM review is a follow-up.
  • CLI librefang skill pending subcommands.
  • Dashboard pending-skill review page (can ship in a follow-up PR; gate the feature on CLI alone first).
  • Integration test: simulated 3-turn conversation where user says "from now on always run cargo fmt before commit"; verify a candidate appears in ~/.librefang/skills/pending/.
  • Security test: candidate body containing prompt-injection markers (Ignore previous instructions ...) is rejected by the same scanner used for marketplace skills.

Depends on auxiliary LLM client (#3314) for the LLM review mode and on the prompt-build hook (#3326) for the post-turn trigger.

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/runtimeAgent loop, LLM drivers, WASM sandboxarea/skillsSkill system and FangHub marketplacedifficulty/hard1 day+, requires understanding multiple cratesenhancementNew feature or requestfeature-parityPorted from OpenFanghas-prA pull request has been linked to this issueneeds-attentionReporter/contributor replied — needs maintainer actionneeds-rfcRequires an RFC before implementationseverity/mediumBug or limitation with workaround / partial impact

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions