Skip to content

Commit a981f93

Browse files
cdeustclaude
andauthored
fix(hooks): fill a session entry's tool sequence from its transcript (#591) (#592)
* fix(hooks): fill a session entry's tool sequence from its transcript (#591) Every session entry was written with toolsUsed: [] and turnCount: 0, because the SessionEnd payload carries neither and _build_session_entry read them from it. Measured on the live log: 413 of 413 entries empty, procedural_skills 0 rows, recall_skills always empty. infrastructure/transcript_activity.py reads the transcript the event does carry and returns the ordered tool sequence, repetitions included, plus the assistant turn count. The entry is filled from it when the event does not supply the fields; an event that carries either keeps its own value (ADR-1072). Order matters because mine_skills mines contiguous subsequences, so scanner_parse now accumulates a list rather than a set, and the scanner takes the sequence from the whole file instead of the head-and-tail sample it uses for metadata. Measured over the 374 transcripts under ~/.claude/projects: 206 carry a sequence, mine_skills returns 796 skills in 0.02 s, and a session end reads one transcript in about 4.5 ms. Proficiency stays at 0.5 for all of them: the session log records no outcome yet. Full suite: 8724 passed, 8 skipped. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01Cz7A6goxVzfKL9NT7k4zDH * fix(hooks): skip a transcript line that is not a JSON object (#591) The review of PR #592 showed transcript_activity crashing with AttributeError on a line that parses as valid JSON but is not an object, which a whole-file JSON array produces. _build_session_entry runs inside process_event with no try/except, so that would have cost the session its log entry and its profile update, worse than the empty entry it replaces. The record is now skipped like an unparseable line, and the docstring says so. Two tests cover it: a whole-file JSON array and a JSONL holding a bare string and a bare array; both raise on the previous code. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01Cz7A6goxVzfKL9NT7k4zDH * fix(hooks): a transcript record's message field can hold anything (#591) The previous commit guarded the record and left the field below it unguarded: `(record.get("message") or {}).get("content")` falls back only when message is falsy, so a truthy non-object (a string, a list) reached .get() and raised AttributeError in the same unguarded SessionEnd hook path. Reproduced on the branch with {"type": "assistant", "message": "not a dict"} before fixing. _message_content reads the content only when message is an object, and _tool_names already ignores a content that is not a list of blocks. Four parametrized cases (string, list, number, bool) cover it; each raises on the previous code, and the record still counts as a turn. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01Cz7A6goxVzfKL9NT7k4zDH --------- Co-authored-by: Claude Opus 5 (1M context) <[email protected]>
1 parent d548fc1 commit a981f93

11 files changed

Lines changed: 428 additions & 10 deletions

‎CHANGELOG.md‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,19 @@ adheres to [Semantic Versioning](https://semver.org/).
2020

2121
### Fixed
2222

23+
- **Procedural learning finally has an input (#591).** Every session entry was
24+
written with `toolsUsed: []` and `turnCount: 0`, because the SessionEnd
25+
payload carries neither and `hooks/session_lifecycle.py` read them from it:
26+
413 of 413 entries in the live log, so `procedural_skills` held 0 rows and
27+
`recall_skills` always returned nothing. The entry is now filled from the
28+
session's own transcript, in call order with repetitions, which is what
29+
`mine_skills` reads (`infrastructure/transcript_activity.py`, ADR-1072). The
30+
scanner's import path kept tool names in a set built from a head-and-tail
31+
sample; it now takes the ordered sequence from the whole file. Measured over
32+
374 transcripts: 206 carry a sequence, mining returns 796 skills in 0.02 s,
33+
and a session end reads one transcript in about 4.5 ms. Proficiency stays at
34+
0.5 until sessions record an outcome.
35+
2336
- **The launcher installs from an interpreter that has no pip (#582).**
2437
Every install ran `sys.executable -m pip`, so from a `uv venv` (no pip module
2538
by design) the plugin's private `deps/` directory could not be filled at all:
Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
<!-- Generated by Cortex wiki_reindex from wiki/adr/cortex/1072-a-session-entry-carries-the-tool-sequence-read-from-its-transcript.md; ADR-1072. Read-only mirror: edit the canonical wiki page. -->
2+
3+
---
4+
created: 2026-09-16T11:12:38Z
5+
kind: adr
6+
number: 1072
7+
status: accepted
8+
tags: [procedural-memory, hooks, session-log, issue-591]
9+
title: A session entry carries the tool sequence read from its transcript
10+
---
11+
# ADR-1072: A session entry carries the tool sequence read from its transcript
12+
13+
## Status
14+
15+
accepted
16+
17+
## Context
18+
19+
Procedural learning had never produced a skill: `procedural_skills` held 0 rows in the maintainer's store on 2026-09-16, and `recall_skills` returned an empty list both cwd-scoped and unscoped, matching the same measurement taken on 2026-08-25.
20+
21+
The chain stopped at its first link. `hooks/session_lifecycle.py::_build_session_entry` wrote `"toolsUsed": event.get("tools_used") or []` and `"turnCount": event.get("turn_count", 0)`, but the SessionEnd payload carries `session_id`, `transcript_path` and `cwd`, not those two fields. Measured on the live `session-log.json`: 413 sessions, every one with `toolsUsed: []` and `turnCount: 0`. `procedural_skill_writer._session_log_to_mining_input` drops an entry with no tools, so `maybe_mine_skills` always returned `skipped / insufficient_history` (issue #591).
22+
23+
The transcript the event does carry holds the evidence. Measured 2026-09-16 over the 374 transcripts under `~/.claude/projects`: 206 carry at least one tool call, the longest sequence is 971 calls, the median non-empty one 19, and reading one costs about 4.5 ms.
24+
25+
`core/procedural_memory.mine_skills` mines *contiguous subsequences*, so order and repetition are the evidence. Two readers destroyed them: `scanner_parse.extract_message_stats` accumulated tool names into a set, and `scanner._parse_conversation_file` fed it from `read_head_tail`, a head-and-tail sample that would splice two halves of a session into one apparent sequence.
26+
27+
## Decision
28+
29+
`infrastructure/transcript_activity.py` reads a transcript as a stream and returns the ordered tool sequence, repetitions included, with the assistant turn count. `_build_session_entry` fills `toolsUsed` and `turnCount` from it when the event does not supply them; an event that carries either keeps its own value, and a missing or unreadable transcript leaves the entry as it was.
30+
31+
The sequence is capped at `MAX_TOOL_SEQUENCE = 2000` so one long session cannot dominate the session log, a cap the measured maximum of 971 sits under.
32+
33+
`scanner_parse` accumulates the sequence in a list rather than a set, and `scanner._parse_conversation_file` takes the sequence from the whole file instead of the head-and-tail sample it uses for metadata.
34+
35+
## Consequences
36+
37+
Easier: mining has an input. Measured on the same 374 transcripts, `mine_skills` returns 796 skills over the 206 sessions that carry a sequence, in 0.02 s; over a 120-transcript sample, the strongest were `Bash>ToolSearch` (32 sessions), `Agent>Bash` (27) and `Bash>Agent` (25). A session end now reads one transcript, about 4.5 ms.
38+
39+
`tests_py/infrastructure/test_transcript_activity.py` covers order and repetition, a session with turns but no tools, a malformed line, a missing transcript and the cap. `tests_py/hooks/test_session_lifecycle.py::TestSessionEntryActivity` covers the three entry cases. `tests_py/infrastructure/test_scanner.py` now expects `["Read", "Read"]` where it expected `["Read"]`: the repetition the set used to swallow.
40+
41+
Harder: every proficiency is 0.5 today. The session log records no outcome, so `mine_skills` treats every session as neutral and the smoothed success rate sits at its prior. Skills are recurrence without judgment until an outcome is recorded; that is the next step, not this one.
42+
43+
Unchanged: the 413 entries already in the log stay empty; they carry no transcript path to recover from.

‎mcp_server/hooks/session_lifecycle.py‎

Lines changed: 11 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -52,6 +52,7 @@
5252
load_session_log,
5353
save_session_log,
5454
)
55+
from mcp_server.infrastructure.transcript_activity import transcript_activity
5556
from mcp_server.shared.categorizer import categorize
5657
from mcp_server.shared.project_ids import (
5758
cwd_to_project_id,
@@ -173,6 +174,14 @@ def _build_session_entry(event: dict[str, Any], domain_id: str) -> dict[str, Any
173174
174175
source: ADR-0497"""
175176
keywords = event.get("keywords") or []
177+
tools = event.get("tools_used") or []
178+
turns = event.get("turn_count") or 0
179+
if not tools or not turns:
180+
# The SessionEnd payload carries neither; its transcript does
181+
# (ADR-1072).
182+
activity = transcript_activity(event.get("transcript_path"))
183+
tools = tools or activity["tool_sequence"]
184+
turns = turns or activity["turn_count"]
176185
return {
177186
"sessionId": session_id_from_transcript(event.get("transcript_path"))
178187
or event["session_id"],
@@ -181,8 +190,8 @@ def _build_session_entry(event: dict[str, Any], domain_id: str) -> dict[str, Any
181190
"project": event.get("project") or cwd_to_project_id(event.get("cwd")),
182191
"cwd": event.get("cwd"),
183192
"duration": event.get("duration"),
184-
"turnCount": event.get("turn_count", 0),
185-
"toolsUsed": event.get("tools_used") or [],
193+
"turnCount": turns,
194+
"toolsUsed": tools,
186195
"category": categorize(" ".join(keywords)) if keywords else "general",
187196
"entryKeywords": keywords,
188197
}

‎mcp_server/infrastructure/scanner.py‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@
1515

1616
from mcp_server.infrastructure.config import CLAUDE_DIR
1717
from mcp_server.infrastructure.file_io import list_dir, read_text_file, stat_file
18+
from mcp_server.infrastructure.transcript_activity import transcript_activity
1819
from mcp_server.infrastructure.scanner_parse import (
1920
build_conversation_record,
2021
extract_message_stats,
@@ -202,6 +203,10 @@ def _parse_conversation_file(
202203
if stats["user_count"] + stats["assistant_count"] == 0:
203204
return None
204205

206+
# head+tail sampling would splice two halves of the session into one
207+
# sequence; the tool order comes from the whole file (ADR-1072).
208+
stats["tools_used"] = transcript_activity(file_path)["tool_sequence"]
209+
205210
return build_conversation_record(meta, stats, file_path, project_name)
206211

207212

‎mcp_server/infrastructure/scanner_parse.py‎

Lines changed: 11 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -55,16 +55,20 @@ def extract_metadata_fields(raw_records: list[dict]) -> dict[str, Any]:
5555
}
5656

5757

58-
def _extract_tools_from_content(content: Any, tools_used: set[str]) -> None:
59-
"""Extract tool names from assistant message content blocks."""
58+
def _extract_tools_from_content(content: Any, tool_sequence: list[str]) -> None:
59+
"""Append the tool names of one assistant message, in call order.
60+
61+
Order and repetition are the evidence procedural mining reads
62+
(ADR-1072); a set would describe a sequence nobody performed.
63+
"""
6064
if isinstance(content, list):
6165
for block in content:
6266
if (
6367
isinstance(block, dict)
6468
and block.get("type") == "tool_use"
6569
and block.get("name")
6670
):
67-
tools_used.add(block["name"])
71+
tool_sequence.append(block["name"])
6872

6973

7074
# source: ADR-0596
@@ -75,7 +79,7 @@ def extract_message_stats(raw_records: list[dict]) -> dict[str, Any]:
7579
"""Extract per-message statistics: counts, tools, text content."""
7680
user_count = 0
7781
assistant_count = 0
78-
tools_used: set[str] = set()
82+
tool_sequence: list[str] = []
7983
first_message = None
8084
all_text_parts: list[str] = []
8185
all_text_len = 0
@@ -98,13 +102,13 @@ def extract_message_stats(raw_records: list[dict]) -> dict[str, Any]:
98102
if rec.get("type") == "assistant":
99103
assistant_count += 1
100104
_extract_tools_from_content(
101-
(rec.get("message") or {}).get("content"), tools_used
105+
(rec.get("message") or {}).get("content"), tool_sequence
102106
)
103107

104108
return {
105109
"user_count": user_count,
106110
"assistant_count": assistant_count,
107-
"tools_used": tools_used,
111+
"tools_used": tool_sequence,
108112
"first_message": first_message,
109113
"all_text": " ".join(all_text_parts) or None,
110114
}
@@ -147,7 +151,7 @@ def build_conversation_record(
147151
"userCount": stats["user_count"],
148152
"assistantCount": stats["assistant_count"],
149153
"turnCount": stats["assistant_count"],
150-
"toolsUsed": list(stats["tools_used"]),
154+
"toolsUsed": list(stats["tools_used"]), # ordered, ADR-1072
151155
"duration": compute_duration(meta["first_timestamp"], meta["last_timestamp"]),
152156
"fileSize": st.st_size if st else None,
153157
}
Lines changed: 82 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,82 @@
1+
"""What a session actually did, read from its transcript.
2+
3+
The SessionEnd payload carries `session_id`, `transcript_path` and `cwd`; it
4+
carries neither the tools the session used nor its turn count, so
5+
`hooks/session_lifecycle.py` wrote every session entry with `toolsUsed: []`
6+
and `turnCount: 0` and procedural mining never had an input (issue #591).
7+
8+
The transcript has both. This reads it as a stream, keeping the tool calls in
9+
the order they were made, repetitions included, because `core.procedural_memory`
10+
mines contiguous subsequences: a deduplicated set would describe a sequence
11+
nobody performed.
12+
13+
source: ADR-1072"""
14+
15+
from __future__ import annotations
16+
17+
import json
18+
from pathlib import Path
19+
from typing import Any
20+
21+
# A session's tool calls, capped so one long session cannot dominate the
22+
# session log.
23+
# source: measured 2026-09-16 over the 374 transcripts under
24+
# ~/.claude/projects, where the largest carried 539 tool calls and the
25+
# median non-empty one 34; the decision is recorded in ADR number 1072.
26+
MAX_TOOL_SEQUENCE = 2000
27+
28+
29+
def _message_content(record: dict) -> Any:
30+
"""The record's message content, or None when the record is not shaped
31+
the way a transcript record is. Nothing here trusts a field's type:
32+
every level of a hand-written or truncated transcript can hold anything.
33+
"""
34+
message = record.get("message")
35+
return message.get("content") if isinstance(message, dict) else None
36+
37+
38+
def _tool_names(content: Any) -> list[str]:
39+
if not isinstance(content, list):
40+
return []
41+
return [
42+
block["name"]
43+
for block in content
44+
if isinstance(block, dict)
45+
and block.get("type") == "tool_use"
46+
and block.get("name")
47+
]
48+
49+
50+
def transcript_activity(transcript_path: str | Path | None) -> dict[str, Any]:
51+
"""Return {"tool_sequence": [...], "turn_count": n} for a transcript.
52+
53+
An unreadable, missing or malformed transcript yields an empty sequence
54+
and a zero count: the caller keeps whatever the event supplied. A line
55+
that does not parse as JSON, or parses as anything other than an object,
56+
is skipped and the rest of the file is still read — this runs inside the
57+
SessionEnd hook, where an exception would cost the session its log entry
58+
and its profile update.
59+
"""
60+
empty: dict[str, Any] = {"tool_sequence": [], "turn_count": 0}
61+
if not transcript_path:
62+
return empty
63+
path = Path(transcript_path)
64+
sequence: list[str] = []
65+
turns = 0
66+
try:
67+
with path.open(encoding="utf-8", errors="replace") as stream:
68+
for line in stream:
69+
if '"assistant"' not in line:
70+
continue
71+
try:
72+
record = json.loads(line)
73+
except ValueError:
74+
continue
75+
if not isinstance(record, dict) or record.get("type") != "assistant":
76+
continue
77+
turns += 1
78+
if len(sequence) < MAX_TOOL_SEQUENCE:
79+
sequence.extend(_tool_names(_message_content(record)))
80+
except OSError:
81+
return empty
82+
return {"tool_sequence": sequence[:MAX_TOOL_SEQUENCE], "turn_count": turns}

‎tests_py/hooks/test_session_lifecycle.py‎

Lines changed: 72 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -333,3 +333,75 @@ def test_tombstone_runs_even_on_tty_with_no_event(
333333
main()
334334
mock_tomb.assert_called_once_with()
335335
mock_pe.assert_not_called()
336+
337+
338+
class TestSessionEntryActivity:
339+
"""Issue #591: the payload carries no tools; its transcript does."""
340+
341+
def _transcript(self, tmp_path):
342+
records = [
343+
{"type": "user", "message": {"content": "fix the bug"}},
344+
{
345+
"type": "assistant",
346+
"message": {
347+
"content": [
348+
{"type": "tool_use", "name": "Read", "input": {}},
349+
{"type": "tool_use", "name": "Edit", "input": {}},
350+
]
351+
},
352+
},
353+
{
354+
"type": "assistant",
355+
"message": {
356+
"content": [{"type": "tool_use", "name": "Bash", "input": {}}]
357+
},
358+
},
359+
]
360+
path = tmp_path / "abc123.jsonl"
361+
path.write_text(
362+
"\n".join(json.dumps(record) for record in records) + "\n",
363+
encoding="utf-8",
364+
)
365+
return str(path)
366+
367+
def test_entry_is_filled_from_the_transcript(self, tmp_path):
368+
from mcp_server.hooks.session_lifecycle import _build_session_entry
369+
370+
entry = _build_session_entry(
371+
{
372+
"session_id": "abc123",
373+
"transcript_path": self._transcript(tmp_path),
374+
"cwd": "/tmp/project",
375+
},
376+
"my-project",
377+
)
378+
379+
assert entry["toolsUsed"] == ["Read", "Edit", "Bash"]
380+
assert entry["turnCount"] == 2
381+
382+
def test_the_event_wins_when_it_carries_the_fields(self, tmp_path):
383+
from mcp_server.hooks.session_lifecycle import _build_session_entry
384+
385+
entry = _build_session_entry(
386+
{
387+
"session_id": "abc123",
388+
"transcript_path": self._transcript(tmp_path),
389+
"cwd": "/tmp/project",
390+
"tools_used": ["Write"],
391+
"turn_count": 9,
392+
},
393+
"my-project",
394+
)
395+
396+
assert entry["toolsUsed"] == ["Write"]
397+
assert entry["turnCount"] == 9
398+
399+
def test_no_transcript_leaves_the_entry_empty(self):
400+
from mcp_server.hooks.session_lifecycle import _build_session_entry
401+
402+
entry = _build_session_entry(
403+
{"session_id": "abc123", "cwd": "/tmp/project"}, "my-project"
404+
)
405+
406+
assert entry["toolsUsed"] == []
407+
assert entry["turnCount"] == 0

‎tests_py/infrastructure/test_scanner.py‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -161,7 +161,8 @@ def test_scopes_to_requested_projects_only(self, tmp_path, monkeypatch):
161161
result = discover_conversations_for_projects(["proj-a"], limit=20)
162162

163163
assert len(result) == 1
164-
assert result[0]["toolsUsed"] == ["Read"]
164+
# order and repetition preserved: two assistant turns, two calls
165+
assert result[0]["toolsUsed"] == ["Read", "Read"]
165166

166167
def test_never_touches_unrequested_project_dirs(self, tmp_path, monkeypatch):
167168
monkeypatch.setattr("mcp_server.infrastructure.scanner.CLAUDE_DIR", tmp_path)

0 commit comments

Comments
 (0)