Repository navigation
[Extension]: Evaluator Contract #4403
Description
Extension Submission
Extension Metadata
- ID: evaluator
- Name: Evaluator Contract
- Version: 1.0.0
- Description: Standard evaluator result contract for evidence, provenance, uncertainty, and recovery — a provider-neutral protocol for extensions that evaluate artifact quality between Spec-Driven Development phases
- Author: ElectroHire
- License: MIT
Links
- Repository: https://github.com/electrohire/spec-kit-evaluator
- Download URL: https://github.com/electrohire/spec-kit-evaluator/archive/refs/tags/v1.0.0.zip
- Documentation: https://github.com/electrohire/spec-kit-evaluator#readme
- Changelog: https://github.com/electrohire/spec-kit-evaluator/blob/main/CHANGELOG.md
Requirements
- Spec Kit Version: >=1.0.0
- Tools: None required
Provides
- Commands: 4 (speckit.evaluator.run, .compose, .report, .route)
- Hooks: 4 (after_specify, after_plan, after_tasks, after_implement)
Tags
evaluator, evidence, provenance, quality, governance, compliance, workflow, model-routing, portfolio
Key Features
- JSON Schema for evaluator results (6 outcomes, 14 finding kinds, 5 evidence kinds)
- Deterministic composition with contradiction detection (strict/majority/optimistic)
- Model routing: recommends budget/standard/premium tier per phase
- All 5 report formats: terminal, markdown, JSON, CI annotations, gate
- 3 parity scripts: Python, Bash, PowerShell
- Quick-start demo:
python examples/demo.py - 70 tests with 0 regressions against spec-kit test suite
- Token-economic benchmark suite with Monte Carlo simulation
Testing Confirmation
- Tested locally with
uv run specify --help - Ran existing tests with
uv sync && uv run pytest - Tested with a sample project (install → verify → uninstall)
Activity
github-actions commentedon Sep 2, 2026
cc @github/spec-kit-maintainers — new catalog submission for review.
[-]Extension Submission: Evaluator Contract[/-][+][Extension]: Evaluator Contract[/+]github-actions commentedon Sep 2, 2026
Validation failed for this community extension submission:
Proposed Catalog Entryfield is missing from the issue body. Add the complete proposed catalog JSON entry.Testing Checklistsection is missing. The body includes a differently namedTesting Confirmationchecklist, so the required checklist cannot be confirmed.Submission Requirementssection and its checkboxes are missing.The extension ID (
evaluator), version (1.0.0), repository and required repository files (extension.yml,README.md,LICENSE), release tag (v1.0.0), and download URL pattern were otherwise validated successfully. Please update the issue with the missing required fields and checklists, then submit it again for validation.fix: detect non-form-format issue submissions in community catalog wo…
tbitcs commentedon Sep 3, 2026
Superseded by #4414, which was resubmitted through the current Extension Submission form with the required field headings, Testing Checklist, Submission Requirements, and proposed catalog JSON. Please use #4414 for review.
AI-assistance disclosure: This administrative comment was drafted with ChatGPT assistance and reviewed by me.
fix: address all 12 Copilot review findings for evaluator extension