feat(skills): compact skill paths with ~ to reduce prompt tokens#14776
feat(skills): compact skill paths with ~ to reduce prompt tokens#14776steipete merged 2 commits intoopenclaw:mainfrom
Conversation
|
CI failing (check): oxfmt formatting issue in src/agents/skills/workspace.ts. Fix: run ERR_PNPM_NO_IMPORTER_MANIFEST_FOUND No package.json (or package.yaml, or package.json5) was found in "/Users/raysopenclaw/.openclaw/workspace-dev-deploy". (oxfmt --write) and commit/push. Then CI should go green. |
|
CI failing due to oxfmt formatting (real, not flaky): Patch (oxfmt-only) that makes diff --git a/src/agents/skills/workspace.ts b/src/agents/skills/workspace.ts
index 24d543a..05afcaf 100644
--- a/src/agents/skills/workspace.ts
+++ b/src/agents/skills/workspace.ts
@@ -287,7 +287,10 @@ export function buildWorkspaceSkillsPrompt(
(entry) => entry.invocation?.disableModelInvocation !== true,
);
const remoteNote = opts?.eligibility?.remote?.note?.trim();
- return [remoteNote, formatSkillsForPrompt(compactSkillPaths(promptEntries.map((entry) => entry.skill)))]
+ return [
+ remoteNote,
+ formatSkillsForPrompt(compactSkillPaths(promptEntries.map((entry) => entry.skill))),
+ ]
.filter(Boolean)
.join("\n");
}Fix locally with: |
|
CI failure is from formatting:
Suggested fix: run |
| import os from "node:os"; | ||
| import path from "node:path"; | ||
| import { describe, expect, it } from "vitest"; | ||
| import { buildWorkspaceSkillsPrompt } from "./skills.js"; |
There was a problem hiding this comment.
Broken test import path
buildWorkspaceSkillsPrompt is imported from ./skills.js, but under src/agents/ the entrypoint is skills.ts (and this repo doesn’t appear to have a skills.js alongside the test). This will fail module resolution in Vitest/TS (can’t run the new test). Update the import to the correct module (e.g. the existing src/agents/skills.ts entrypoint).
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/agents/skills.compact-skill-paths.test.ts
Line: 5:5
Comment:
**Broken test import path**
`buildWorkspaceSkillsPrompt` is imported from `./skills.js`, but under `src/agents/` the entrypoint is `skills.ts` (and this repo doesn’t appear to have a `skills.js` alongside the test). This will fail module resolution in Vitest/TS (can’t run the new test). Update the import to the correct module (e.g. the existing `src/agents/skills.ts` entrypoint).
How can I resolve this? If you propose a fix, please make it concise.| it("preserves paths outside home directory", async () => { | ||
| // Skills outside ~ should keep their absolute paths | ||
| const workspaceDir = await fs.mkdtemp(path.join(os.tmpdir(), "openclaw-compact-")); | ||
| const skillDir = path.join(workspaceDir, "skills", "ext-skill"); |
There was a problem hiding this comment.
Test doesn’t match intent
This test claims it “preserves paths outside home directory”, but it creates the workspace under os.tmpdir() (which may or may not be under os.homedir() depending on platform) and never asserts that the <location> path stayed absolute vs compacted. As written it only checks that a <location>...SKILL.md</location> tag exists, so it won’t catch regressions in the compaction/preservation behavior.
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/agents/skills.compact-skill-paths.test.ts
Line: 59:62
Comment:
**Test doesn’t match intent**
This test claims it “preserves paths outside home directory”, but it creates the workspace under `os.tmpdir()` (which may or may not be under `os.homedir()` depending on platform) and never asserts that the `<location>` path stayed absolute vs compacted. As written it only checks that a `<location>...SKILL.md</location>` tag exists, so it won’t catch regressions in the compaction/preservation behavior.
How can I resolve this? If you propose a fix, please make it concise.bfc1ccb to
f92900f
Compare
Replace absolute home directory prefix with ~ in skill <location> tags injected into the system prompt. Models understand ~ expansion and the read tool resolves it, so this is a safe, backward-compatible change. Saves ~5-6 tokens per skill path. For a workspace with 90+ skills, this reduces system prompt size by ~400-600 tokens. Changes: - Add compactSkillPaths() helper in workspace.ts - Apply in buildWorkspaceSkillSnapshot and buildWorkspaceSkillsPrompt - Add test for path compaction behavior Before: /Users/alice/.bun/install/global/node_modules/openclaw/skills/github/SKILL.md After: ~/.bun/install/global/node_modules/openclaw/skills/github/SKILL.md
2eb081d to
97444f7
Compare
|
That's still a whole lot of tokens for those paths. Wouldn't it make sense to (on systems that support it*) add a symlink and *) macOS, Linux, Windows w/ admin |
(cherry picked from commit 7694900)
Problem
Skill
<location>tags in the system prompt use absolute paths like:Each path costs ~24 tokens. For a workspace with 90+ skills, this adds up to a significant chunk of the system prompt budget.
Solution
Add a
compactSkillPaths()helper that replaces the home directory prefix with~:This is applied in
buildWorkspaceSkillSnapshotandbuildWorkspaceSkillsPromptbefore passing skills toformatSkillsForPrompt.Why this is safe
~— Claude, GPT-4, Gemini all resolve~/to the home directory~— OpenClaw's file read tool already handles tilde expansionImpact
/Users/username→~)Test
Added
skills.compact-skill-paths.test.tscovering:~/...Greptile Overview
Greptile Summary
This PR introduces
compactSkillPaths()insrc/agents/skills/workspace.tsand applies it when building the skills prompt/snapshot so skill<location>tags replace the user’s home directory prefix with~/to save system-prompt tokens.The core runtime change is isolated to prompt formatting (the
Skillobjects used for loading/frontmatter parsing remain untouched), but the newly added test file has issues that will prevent it from reliably validating the behavior across environments.Confidence Score: 3/5
(2/5) Greptile learns from your feedback when you react with thumbs up/down!