fix(quality): score non-mapping metadata instead of crashing - #226

Summary

A SKILL.md whose metadata is not a mapping crashes the quality check, and the whole Tier 1 run with it:

---
name: odd-metadata
description: A focused skill. Use when testing malformed contributor metadata.
metadata: [author, tags]
---

$ skillevaluator validate ./odd-metadata --tiers 1
...
  File ".../skillevaluator/validators/quality_score.py", line 889, in _check_spec_fields
    if field_name not in metadata or metadata[field_name] is None:
TypeError: list indices must be integers or slices, not str

metadata: true, metadata: 1.5, an unquoted date, and any list or string containing author or tags fail the same way. validate, tier1 and quality-check stop with a traceback and write no report (a catalog run lists the skill as an unexpected error), so the user never sees the schema finding that already describes the problem: Field 'metadata': Input should be a valid dictionary or instance of SkillMetadata.

Cause. _check_spec_fields replaced only falsy values with {} (frontmatter.get("metadata") or {}), then ran field_name in metadata and metadata[field_name] on whatever was left. No other Tier 1 check that reads frontmatter crashes on these values.

Fix. A metadata value that is not a mapping is scored as absent metadata: metadata.author and metadata.tags count as missing, as they already did for null, empty and other non-mapping values such as metadata: Jane Doe. Mapping values score exactly as before.

Verification

• I am familiar with the Contributing Guidelines
• Added or updated focused tests
• Updated documentation for user-visible changes (changelog)
• Ran make lint
• Ran make test
• Ran make build
• Did not add credentials, private datasets, or proprietary benchmark content

New tests in tests/validators/test_quality_score.py: one runs the quality check on list, string, boolean, float and date metadata values, and one runs run_validation(checks="schema,quality") and checks that the run completes and reports the schema finding. All six raise TypeError without the fix and pass with it.

make test on this branch, run with a 60 s per-test timeout because a few process-cleanup tests hang in my sandbox: 14,441 passed, 25 skipped, 107 failed. The failures are environmental. 106 of them fail the same way on unpatched main (the Seatbelt and Docker process tests my sandboxed machine can't run, parallel catalog workers, sockets and writes under /tmp, a Python 3.9 python3 on PATH, and a pip TLS error in the packaging test). The last one is the OSS boundary scan hitting the 60 s limit under load; it passes on its own. make lint passes, and python -m build builds the sdist and wheel (with --installer uv, for the same pip TLS reason). The quality tests also pass on Python 3.13 with only the dev extra.

Release Impact

• No user-visible release note needed
• Updated CHANGELOG.md

🤖 Generated with Claude Code