Skip to content

[bug] procedural learning captures nothing: every session entry is written with an empty toolsUsed #591

Closed

Description

@cdeust

What happened

Procedural learning has never produced a single skill. procedural_skills holds 0 rows in the maintainer's PostgreSQL store (checked 2026-09-16), and recall_skills returns {"skills": [], "count": 0} both cwd-scoped and unscoped. The same measurement was taken on 2026-08-25 during a harness comparison, so this is not a recent regression.

The chain stops at its first link. handlers/record_session_end.py:369 mines from load_session_log()["sessions"], and hooks/session_lifecycle.py:184-185 builds each of those entries as:

"turnCount": event.get("turn_count", 0),
"toolsUsed": event.get("tools_used") or [],

The SessionEnd payload carries session_id, transcript_path and cwd, not turn_count or tools_used. Measured on the live log (~/.claude/methodology/session-log.json): 413 sessions, every one with toolsUsed: [] and turnCount: 0. procedural_skill_writer._session_log_to_mining_input skips an entry with no tools, so the mining input is always empty, maybe_mine_skills returns skipped / insufficient_history, and nothing is ever written.

The transcript the event does carry has the evidence. Measured over 60 randomly sampled transcripts under ~/.claude/projects/ on 2026-09-16: 35 of 60 carry at least 2 tool uses (the MIN_SKILL_LEN floor), the median non-empty one carries 34, the largest carries 539, and extracting the sequence costs about 6 ms per transcript (0.4 s for all 60; a 17 MB transcript parses in 0.04 s).

A second defect sits one step further: infrastructure/scanner_parse.py:150 records "toolsUsed": list(stats["tools_used"]) where stats["tools_used"] is a set, so the import path produces an unordered, deduplicated list. core/procedural_memory.mine_skills mines contiguous subsequences, so an unordered list yields sequences that were never performed.

Expected behavior

A session entry carries the ordered tool sequence actually performed, with repetitions, derived from the transcript when the event does not supply it, and a turn count. Mining then has real input, and recall_skills can return something.

Evidence required to close

A test deriving the sequence from a transcript fixture (order and repetition preserved), a test that the session entry is filled when the event omits the fields, and an end-to-end measurement showing mine_skills producing at least one skill from real session entries.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions