Skip to content

Commit f4d136f

Browse files
udayvarmora07claude
andcommitted
Estimate Codex's skill-listing budget and flag skills OpenCode truncates
Overnight round 2 (docs/44). The hygiene report gains a read-only codex_listing section that applies Codex's own listing rule (2% of the context window, 1,024-character description cap, ceil(bytes/4) tokens per line; openai/codex#19679) to ~/.agents/skills, and an opencode-output-truncation warning for enabled skills over OpenCode's 2,000-line / 50 KiB tool-output head (anomalyco/opencode#62718). Near-limit documents are read through the bounded regular-file reader, so a FIFO cannot block the report. No command, route, Store method or schema change. Co-Authored-By: Claude Opus 5.5 <[email protected]>
1 parent d189f66 commit f4d136f

9 files changed

Lines changed: 433 additions & 12 deletions

‎docs/02-modules.md‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -259,6 +259,23 @@ report adds a home-redacted `codex_config` section from
259259
warning when any entry matches no skill; other scopes report it
260260
`unavailable`. Nothing is written.
261261

262+
**[SPEC] `codex_listing` (2026-11-13, round 2).** Estimates Codex's skill
263+
listing from enabled, readable `agents`-scope instances (Codex's documented
264+
USER root), excluding `SKILL.md.disabled` and `[[skills.config]]`-disabled
265+
skills. Each line is `- name: description (file: path)` with the description
266+
cut at 1,024 characters and costed `ceil(utf-8 bytes / 4)` tokens; the budget
267+
is 2% of the report window, replaced by an assumed `gpt-5.6` window (and
268+
said so) when the window is not a GPT model. Project, admin and bundled roots
269+
and a configured token budget are unread, so `confidence` is `partial`. One
270+
`codex-listing-budget` info finding when over budget (codex#19679,
271+
codex-rs/ext/skills/src/render.rs).
272+
273+
**[SPEC] `opencode-output-truncation` (2026-11-13, round 2).** An enabled
274+
instance in a scope OpenCode reads (`opencode`, `agents`, `claude-code`)
275+
whose `SKILL.md` exceeds 2,000 lines or 50 KiB gets one warning: OpenCode
276+
keeps only the head of a tool result (opencode#62718). Only documents whose
277+
body is near either limit are opened (bounded at 8 MiB); nothing is written.
278+
262279
## `key_support.py`
263280

264281
Pure, stdlib-only table of the frontmatter keys each consumer documents, copied

‎docs/03-cli-surface.md‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -220,6 +220,14 @@ The report's `codex_config` section (`--scope codex` or `all`) names Codex
220220
instead of its `SKILL.md`, a missing file, or another file — and the human
221221
output prints each with the `SKILL.md` path to use. Nothing is changed.
222222

223+
The report's `codex_listing` section estimates Codex's skill listing from
224+
`~/.agents/skills` against its 2% context-window budget (codex#19679) and
225+
prints one line, for example `Codex listing: ~10,600 tokens vs 8,000 budget
226+
(2% rule, estimate); 40 skills from ~/.agents/skills`. An
227+
`opencode-output-truncation` warning names enabled skills OpenCode reads
228+
whose `SKILL.md` exceeds 2,000 lines or 50 KiB, the head OpenCode keeps of a
229+
tool result (opencode#62718). Both are read-only estimates.
230+
223231
The report's `agents_bridge` section (docs/29 H2, explain-only) names enabled,
224232
readable skills that exist in `~/.agents/skills` but have no copy or link in
225233
`~/.claude/skills`. Claude Code's documented roots do not include

‎docs/06-progress-log.md‎

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,40 @@ dated 2026-09-23 and later; everything older lives verbatim in
88
@docs/archive/README.md.** Read the digest below first — it is the part a session
99
actually needs. Facts flagged stale here are corrected in the owning doc.
1010

11+
## 2026-11-13 — Overnight round 2: Codex listing budget, OpenCode truncation
12+
13+
**[SPEC]** Two read-only hygiene additions from
14+
@docs/44-overnight-user-problems-2026-11-13.md, no new command, route, Store
15+
method, schema or dependency:
16+
17+
- **`codex_listing`** (openai/codex#19679). Codex's renderer was read in
18+
`codex-rs/ext/skills/src/render.rs`: 2% of the window in tokens, lines
19+
`- name: description (file: path)`, descriptions cut at 1,024 characters,
20+
`ceil(bytes / 4)` tokens per line. Applied to enabled `~/.agents/skills`
21+
instances (file- and config-disabled excluded); one `codex-listing-budget`
22+
info finding when over. A non-GPT report window becomes an assumed `gpt-5.6`
23+
window and the basis says so. CLI prints one summary line.
24+
- **`opencode-output-truncation`** (sst/opencode#62718). OpenCode keeps only
25+
the first 2,000 lines / 50 KiB of a tool result. Enabled instances in
26+
`opencode`, `agents` or `claude-code` scopes over either limit get one
27+
warning. Only near-limit documents are opened, through
28+
`file_reads.read_regular_file` (8 MiB bound), so a FIFO named `SKILL.md`
29+
cannot block the report; self-review caught the first draft using plain
30+
`open()`.
31+
32+
**[NOTE]** Red-first: 11 new tests, 10 fail on `d189f66` (the other is a
33+
guard). Real data, read-only: 153 skills ≈ 14,640 Codex tokens (within the
34+
assumed 21,000; over 8,000 at a 400k window); one 57 KiB skill flagged for
35+
OpenCode. **Not verified:** Codex or OpenCode reading the fixtures; Codex's
36+
project/admin/bundled roots and a configured token budget are unread, so the
37+
estimate is `partial`; the v2 Quality page was not re-rendered (the findings
38+
use its generic finding layout).
39+
40+
**[NOTE] Verification on this tree:** 1,943 tests on each Python 3.11, 3.12,
41+
3.13 and 3.14 (two optional oracle skips each); both smokes, `check_docs`,
42+
`check_complexity` (1,277 functions, nothing raised), `check_design_tokens`,
43+
frontend syntax, Ruff 0.16.8 (F821/F822/F823) and Bandit 1.9.4 (0 results).
44+
1145
## 2026-11-13 — Overnight round 1: problems users hit that a local check can name
1246

1347
**[NOTE]** An unattended round researched skill-management problems reported

‎docs/08-web-ui.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -398,7 +398,7 @@ stated beside the setting and in every result.
398398
| GET | `/api/search?q=term[&scope=SCOPE]` | search over name/description/body (scope-aware); bounded wildcard failures return the standard JSON `StoreError` 400 |
399399
| GET | `/api/stats` | Store.stats plus scope/token enrichment: `scopes`, `all_total`, `all_tokens`, `all_avg_tokens`, `window`, `window_tokens`, `all_pct_window`, `largest`, `has_tiktoken`. When an enrichment step fails, the documented keys are still filled with safe fallbacks and the step is named in a `degraded` array (e.g. `"largest"`, `"scopes"`) |
400400
| GET | `/api/doctor[?scope=all]` | Store.doctor (global), including filesystem/index drift and transaction-artifact diagnostics; `?scope=all` adds `scopes` + `duplicates` (`scopes.find_duplicates()`), and on an enrichment failure both are returned as `[]` with an entry appended to `degraded` (`[{"section", "reason"}]`), so a client can tell "no duplicates" from "the scan crashed" |
401-
| GET | `/api/doctor?scope=SCOPE&hygiene=1` | Adds the bounded, deterministic, read-only Skill Hygiene Report under `hygiene`; uses only configured observed scope records, preserves ordinary Doctor output, rejects unknown scopes and `explain`, and returns degraded/unavailable evidence without a new endpoint or mutation. The report includes an advisory `ignored-frontmatter` finding for a documented behavioural key (`disable-model-invocation`, `allowed-tools`, …) that the reading consumer does not document; scopes without a documented key list (`global`, `commandcode`, `cursor`, project scopes) add a `frontmatter-key-support` unavailable signal instead. The report's `lockfiles` section reads the `skills` CLI's global `.skill-lock.json` and (when a project root is supplied) `skills-lock.json`, comparing declared copies read-only; no project root is supplied over HTTP, so that lock is reported unavailable. The report's `claude_listing` section estimates the Claude Code skill listing from the `claude-code` scope: entries honour `skillOverrides` (`off` excluded, `name-only` counted as its name), the per-entry cap is `skillListingMaxDescChars` (default 1,536) over `description` plus `when_to_use`, and the character budget is `SLASH_COMMAND_TOOL_CHAR_BUDGET` or `skillListingBudgetFraction` times the token window with the character figure marked as an estimate. `status` is `within_budget`, `over_budget_estimate`, or `unknown` when a settings file exists but could not be parsed, and `name_only_candidates` are ordered by description size only because Claude Code's invocation-count order is not observable. No project root is supplied over HTTP, so project `.claude/skills` instances are reported unavailable. An `agent-parse-risk` warning names instances whose frontmatter a stricter agent parser drops (empty block, unquoted `: `, byte-order mark before `---`). With `scope=codex` or `scope=all`, the read-only `codex_config` section and a `codex-config-dead-entry` warning list Codex `[[skills.config]]` path entries that name a folder, a missing file or a non-`SKILL.md` file and so never apply; v2 Quality lists each with the `SKILL.md` path to use. With `scope=all`, the read-only `agents_bridge` section and an `agents-bridge-gap` info finding name enabled `~/.agents/skills` skills that Claude Code cannot see, with the existing `sync` copy as the remedy (other scopes report it unavailable). |
401+
| GET | `/api/doctor?scope=SCOPE&hygiene=1` | Adds the bounded, deterministic, read-only Skill Hygiene Report under `hygiene`; uses only configured observed scope records, preserves ordinary Doctor output, rejects unknown scopes and `explain`, and returns degraded/unavailable evidence without a new endpoint or mutation. The report includes an advisory `ignored-frontmatter` finding for a documented behavioural key (`disable-model-invocation`, `allowed-tools`, …) that the reading consumer does not document; scopes without a documented key list (`global`, `commandcode`, `cursor`, project scopes) add a `frontmatter-key-support` unavailable signal instead. The report's `lockfiles` section reads the `skills` CLI's global `.skill-lock.json` and (when a project root is supplied) `skills-lock.json`, comparing declared copies read-only; no project root is supplied over HTTP, so that lock is reported unavailable. The report's `claude_listing` section estimates the Claude Code skill listing from the `claude-code` scope: entries honour `skillOverrides` (`off` excluded, `name-only` counted as its name), the per-entry cap is `skillListingMaxDescChars` (default 1,536) over `description` plus `when_to_use`, and the character budget is `SLASH_COMMAND_TOOL_CHAR_BUDGET` or `skillListingBudgetFraction` times the token window with the character figure marked as an estimate. `status` is `within_budget`, `over_budget_estimate`, or `unknown` when a settings file exists but could not be parsed, and `name_only_candidates` are ordered by description size only because Claude Code's invocation-count order is not observable. No project root is supplied over HTTP, so project `.claude/skills` instances are reported unavailable. An `agent-parse-risk` warning names instances whose frontmatter a stricter agent parser drops (empty block, unquoted `: `, byte-order mark before `---`). With `scope=codex` or `scope=all`, the read-only `codex_config` section and a `codex-config-dead-entry` warning list Codex `[[skills.config]]` path entries that name a folder, a missing file or a non-`SKILL.md` file and so never apply; v2 Quality lists each with the `SKILL.md` path to use. The read-only `codex_listing` section estimates Codex's `~/.agents/skills` listing in tokens against 2% of the context window and adds a `codex-listing-budget` info finding when over it; an `opencode-output-truncation` warning names enabled skills in OpenCode-read scopes whose `SKILL.md` exceeds 2,000 lines or 50 KiB. With `scope=all`, the read-only `agents_bridge` section and an `agents-bridge-gap` info finding name enabled `~/.agents/skills` skills that Claude Code cannot see, with the existing `sync` copy as the remedy (other scopes report it unavailable). |
402402
| GET | `/api/doctor?explain=CONSUMER[&project=DIR][&skill=NAME]` | adds the read-only effective-resolution report (issue #12, `effective.explain`) under `explain`; derived at read time, writes nothing, and reports `unknown-consumer`/`missing-project` as explicit results. **The report is confined to the manager's data directory** plus any extra root the operator passed at server construction; a request cannot widen that boundary (SEC-2/SEC-3): every path outside it is replaced by `<redacted>` with `paths_redacted: true`, and a `project` outside it returns resolution `project-outside-managed-roots` **without being walked** (no nested-root search, no instance read). Consumers include `commandcode`, `codex`, `claude-code`, `gemini`, the corrected `opencode`, `cline`, `kiro`, `amp`, `roo`, `zed`, plus `cursor`, `copilot` and `devin` with `undocumented-precedence`; a documented `home`-relative root is redacted exactly like an absolute `path` root. The CLI passes no boundary and stays fully transparent |
403403
| GET | `/api/tokens?window=&name=&scope=&text=` | token/context estimate: with `text=` it estimates that text, with `name=` it estimates one skill's document (scope-qualified when `scope=` is a scope id, otherwise the first match across scopes), and with neither it aggregates over `scope` (default `all`). Aggregate responses add `window`, `window_tokens` and `pct_window`; `window` selects the context window (claude, claude-haiku, gpt-5.6, gpt-5, gpt-4o, gemini, gemini-2m) |
404404
| GET | `/api/history?name=&limit=` | Store.history (name optional) |

‎docs/44-overnight-user-problems-2026-11-13.md‎

Lines changed: 41 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,8 @@
11
# Overnight User Problems — 2026-11-13
22

3-
**Version 0.1.0**
3+
**Version 0.2.0**
44

5-
**AI manifest**: Ranked research record from one unattended improvement round on 2026-11-13: which real skill-management problems reported since 2026-10-01 this tool can detect locally, which three were fixed, and what is left for later rounds. Evidence is GitHub issues read on 2026-11-13; reaction counts on recent issues are small, so the ranking is judgment, not votes.
5+
**AI manifest**: Ranked research record from two unattended improvement rounds on 2026-11-13: which real skill-management problems reported since 2026-10-01 this tool can detect locally, which three were fixed, and what is left for later rounds. Evidence is GitHub issues read on 2026-11-13; reaction counts on recent issues are small, so the ranking is judgment, not votes.
66

77
## 1. Method
88

@@ -21,14 +21,14 @@ deterministic check can catch it before the agent does.
2121

2222
| # | Problem | Evidence | Locally detectable? |
2323
|---|---|---|---|
24-
| 1 | Listing budgets silently truncate descriptions (Claude Code and Codex; Codex uses 2% of the window, fallback 8,000 characters) | claude-code #107255; codex #19679 (40 reactions) | Yes; Claude Code already estimated (`claude_listing`), Codex not yet |
24+
| 1 | Listing budgets silently truncate descriptions (Claude Code and Codex; Codex uses 2% of the window, fallback 8,000 characters) | claude-code #107255; codex #19679 (40 reactions) | Yes; Claude Code already estimated (`claude_listing`); Codex **fixed in round 2** |
2525
| 2 | A UTF-8 byte-order mark before `---` makes Codex skip the skill ("missing YAML frontmatter delimited by ---") | codex #13918 (updated 2026-10-28) | Yes — **fixed this round** |
2626
| 3 | Frontmatter quirks drop skills in strict parsers | gemini-cli #25693 | Partly — the existing lint now **reaches Doctor and Quality** this round |
2727
| 4 | A Codex `[[skills.config]]` `path` entry naming the skill **folder** silently disables nothing; it must name `SKILL.md` | codex #51044 (2026-10-05), codex #60951 | Yes — **fixed this round** |
2828
| 5 | `disable-model-invocation` ignored by some agents | opencode #53172, claude-code #98898 | Partly; the ignored-frontmatter finding covers the documented part |
2929
| 6 | Skills under `~/.agents/skills` invisible to some agents | claude-code #31005 (619 reactions), opencode #41213 | Partly; Claude Code covered by `agents_bridge` |
3030
| 7 | Duplicate copies across agent folders break `npx skills update` | vercel-labs/skills #2406, #2574 | Mostly existing (duplicates, lockfile evidence) |
31-
| 8 | Very large `SKILL.md` tails truncated by OpenCode at 2,000 lines / 50 KiB | opencode #62718 | Yes; not done yet |
31+
| 8 | Very large `SKILL.md` tails truncated by OpenCode at 2,000 lines / 50 KiB | opencode #62718 | Yes — **fixed in round 2** |
3232

3333
## 3. What was fixed, and why these three
3434

@@ -61,11 +61,44 @@ pinned by tests that fail on the previous commit.
6161
cached tokens; its example implies it, and the caps rely on that. Not observed
6262
against a live provider.
6363

64-
## 4. Candidates for later rounds
64+
## 4. Round 2 (same day)
65+
66+
**[SPEC]** Two read-only hygiene additions, both pinned by tests that fail on
67+
`d189f66`:
68+
69+
1. **Codex listing budget (problem 1).** Codex's renderer
70+
(`codex-rs/ext/skills/src/render.rs`, read 2026-11-13) reserves 2% of the
71+
model's context window in tokens (8,000 characters when the window is
72+
unknown), renders each skill as `- name: description (file: path)`, caps a
73+
description at 1,024 characters and costs a line `ceil(bytes / 4)` tokens.
74+
The new `codex_listing` section applies that rule to enabled
75+
`~/.agents/skills` instances, excluding file- and config-disabled skills,
76+
and adds one `codex-listing-budget` info finding when over budget. A
77+
non-GPT report window is replaced by an assumed `gpt-5.6` window, stated in
78+
the budget basis.
79+
2. **OpenCode tool-output truncation (problem 8).** OpenCode keeps the first
80+
2,000 lines / 50 KiB of a tool result, so the tail of a larger `SKILL.md`
81+
never reaches the model. An `opencode-output-truncation` warning names
82+
enabled instances in the scopes OpenCode reads (`opencode`, `agents`,
83+
`claude-code`) over either limit.
84+
85+
**[?]** Codex's project `.agents/skills`, `/etc/codex/skills`, bundled system
86+
skills and a configured token budget are not read, so the Codex estimate is
87+
`partial`. Whether OpenCode counts 50 KiB in bytes is inferred from the issue's
88+
"KiB"; bytes are used.
89+
90+
**[NOTE]** On the maintainer's machine (read-only): 153 listed
91+
`~/.agents/skills` skills cost about 14,640 Codex tokens, within the assumed
92+
`gpt-5.6` budget of 21,000 but nearly twice the 8,000 a 400k-token `gpt-5`
93+
window allows; one skill (`do-and-judge`, 57,505 bytes) is over OpenCode's
94+
50 KiB head.
95+
96+
## 5. Candidates for later rounds
6597

66-
- Codex listing-budget estimate (2% of the window, 8,000-character fallback)
67-
beside the existing Claude Code estimate (problem 1).
68-
- OpenCode oversized-document warning at 2,000 lines / 50 KiB (problem 8).
6998
- A6 standalone demote-only explainer for risk findings on installed skills.
7099
- A per-agent visibility table generalising `agents_bridge` (problem 6).
71100
- Skill-directory plugin layout check before sync (vercel-labs/skills #2609).
101+
- OpenCode `metadata.opencode/autoinvoke: false` as the working equivalent of
102+
`disable-model-invocation` in the ignored-frontmatter finding (opencode #53172).
103+
- Lockfile `skillPath` ambiguity and nested skills shadowed by a root
104+
`SKILL.md` (vercel-labs/skills #2406, #2574).

‎skillsmgr/cli_handlers.py‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -742,6 +742,16 @@ def _render_codex_config_text(section: dict | None) -> None:
742742
print(f" {_display(str(entry.get('problem', '')))}: {_display(str(entry.get('path', '')))}{hint}")
743743

744744

745+
def _render_codex_listing_text(section: dict | None) -> None:
746+
"""One summary line for the Codex listing-budget estimate (codex#19679)."""
747+
if not section or section.get("status") == "no_instances":
748+
return
749+
budget = int((section.get("budget") or {}).get("budget_tokens") or 0)
750+
used = int(section.get("listing_tokens_estimate", 0) or 0)
751+
print(f" Codex listing: ~{used:,} tokens vs {budget:,} budget (2% rule, estimate); "
752+
f"{int(section.get('listed_skills', 0) or 0)} skills from ~/.agents/skills")
753+
754+
745755
def _render_hygiene_text(report: dict) -> None:
746756
"""Render report evidence through the terminal sanitization seam."""
747757
summary = report.get("summary", {})
@@ -759,6 +769,7 @@ def _render_hygiene_text(report: dict) -> None:
759769
_render_claude_listing_text(report.get("claude_listing"))
760770
_render_agents_bridge_text(report.get("agents_bridge"))
761771
_render_codex_config_text(report.get("codex_config"))
772+
_render_codex_listing_text(report.get("codex_listing"))
762773
if summary.get("findings_truncated"):
763774
print(" findings returned are bounded; inspect the JSON limits/degraded evidence")
764775
for finding in report.get("findings", []):

0 commit comments

Comments
 (0)