[bug] githubSkills fails a whole repository on one binary file or one skill's frontmatter #2512

Filed by Claude Opus 5.5 on behalf of Davies (@daviesayo).

Version. Reproduced against packages/apps on v2 at 2f4c398 (apps 0.0.1-beta.122). First seen on a deployment built from 2.0.0-beta.32.

What happens. githubSkills returns no skills at all when one file or one skill in the repository is unusable. Two real public repositories fail this way:

Narrowing path to a folder without the bad skill loads fine (mattpocock/skills at skills/productivity: 7 skills; anthropics/skills at skills/pdf: 1 skill). The error does not name the file or skill, so an author cannot tell which one to exclude.

Repro (run from packages/apps with Bun):

import { githubSkills, fileSkill } from "./src/skills.ts";

for (const [repo, ref] of [
  ["mattpocock/skills", "4ae2317ddb22b046e26455e4f9c9b3aaaf62212d"],
  ["anthropics/skills", "2c5958ecac7cee2b583967a2c4981d0f4a75c579"],
]) {
  try {
    console.log(repo, (await githubSkills({ repo, ref, fetch })).length);
  } catch (e: any) {
    console.log(repo, e._tag, e.reason, e.message);
  }
}

const doc = (fm: string) => [
  { path: "SKILL.md", content: `---\nname: demo\ndescription: Demo.\n${fm}---\n\nBody\n` },
];
for (const fm of [
  "allowed-tools: Read Grep\n",
  "allowed-tools:\n  - Read\n  - Grep\n",
  "metadata:\n  author: someone\n",
  "metadata:\n  credits:\n    author: someone\n",
]) await fileSkill(doc(fm)).then(() => console.log("ok"), (e) => console.log(e._tag, e.reason));

Output:

mattpocock/skills SkillLoadFailed document A skill from GitHub is not a valid skill document.
anthropics/skills SkillLoadFailed encoding A skill file from GitHub is not valid UTF-8 text.
ok
SkillDefinitionInvalid metadata
ok
SkillDefinitionInvalid metadata

Cause.

• Every file in a skill directory is read as fatal UTF-8 text (skills.ts L183-L191, L239-L243), and githubCatalog reads all skills in one Effect.forEach, so the first failure fails the catalog (L548-L583).
• AppSkillMetadata accepts metadata only as string-to-string and allowed-tools only as a string (contracts/skills.ts L52-L53). Any other shape fails skillFromFiles (skill-files.ts L45-L47), which githubCatalog turns into a catalog-wide document failure.

What the spec says. The Agent Skills specification defines metadata as "a map from string keys to string values" and allowed-tools as "a space-separated string" (experimental). So Executor's metadata schema matches the spec, and the two frontmatter shapes above are non-conforming. The same spec lists images and data files under assets/ and says a skill directory "may contain any files", so binary files are conforming.

Suggestion. Isolate failures per skill and per file:

• Skip a non-UTF-8 file, or every file outside SKILL.md that fails to decode, and list it as skipped. The skill stays usable from its text files.
• Skip a skill whose frontmatter fails, and report its directory and reason beside the catalog, instead of failing the read. Alternatively, keep unknown metadata shapes as text in SKILL.md and drop only the field, as other unknown keys are already ignored.
• Name the file or skill directory in SkillLoadFailed when one causes it.

Today an app has to reimplement the GitHub loader to serve these repositories.