Skip to content

About

Spec Kit extension: checks whether merged sibling specs' invariants still hold after a feature's change, and whether the amendment was written where it belongs

Resources

Stars

0 stars

Watchers

0 watching

Forks

Repository files navigation

Spec Kit Cross-Spec Coherence

A Spec Kit extension that checks whether the promises of already-merged sibling specs still hold after the feature you are working on — and whether the amendment was written in the sibling, where it belongs.

The gap this fills

/speckit-analyze reads exactly one feature directory: its spec.md, plan.md, tasks.md, and the constitution. It is thorough about that feature and structurally blind to the failure below.

A merged spec makes a closed per-intent allowlist its central safety control on machine-proposed selectors. Sound — while the intent set is closed and code-declared. A later spec makes the intent set page-derived and unbounded. The first spec is not wrong; an assumption underneath it moved, and nobody wrote the amendment. Both specs passed their own gate. Nothing diffed them against each other.

In the project this came from, that hole survived three review cycles, 61 findings and six review lenses — every one of them pointed at a single document, or at document-versus-code. None was pointed at document-versus-document. It surfaced only when someone asked a concrete behavioural question and walked straight into it.

This extension is pointed there.

What it is not

  • Not spec-versus-code. That is reconciliation, and other tools do it. Code is cited here only as evidence that an assumption moved.
  • Not an architecture review. Coupling inside the running system belongs to a different lens. The test: if a finding would still stand with all the code deleted, it belongs here.
  • Not a grade on the sibling's design. The sibling was usually well-reasoned. The finding is that the pair was never reconciled — and only that framing is actionable.

Two halves, on purpose

The two prior community attempts at cross-spec analysis were pure prompt markdown with no executable logic; both were abandoned within days of publication. The plausible failure mode is that model-only cross-spec analysis produces unusable noise at scale. So the mechanical half here is code with tests, and the judgment half is deliberately narrow.

1. Deterministic — scripts/enumerate-siblings.js (zero-dep Node, read-only, no model)

Finds the siblings this feature has a declared link to — cited by number in its prose, named in an # Implements: header of a file the branch changed, or standing behind a contract that declares itself a delta. "Specs that feel related" is explicitly not in scope; that is how this degenerates into a second architecture review.

It reports two defects that need no judgment at all:

Defect Meaning
CITED_BUT_ABSENT the feature depends on a spec nobody can open — the dependency is real even when the document is not
DELTA_WITHOUT_BASE a contract calls itself a delta over a spec with no contract to be a delta of, so nothing is authoritative for that shape

Citations come in two tiers, because a spec number inside a code span is usually an example, not a dependency. One real spec quotes `# Encodes: … | spec 018 FR-001` as a sample citation format; gating on that reds CI over somebody's example string, which teaches people to stop writing examples. So a number found only inside a code span or fence is reported as an ILLUSTRATIVE_CITATION observation — surfaced, never gated.

The tempting fix — mask every code span — is worse. Genuine references are routinely written as `docs/specs/183-foo/spec.md` because they are paths, and dropping those trades a false positive for a false negative, which is the more expensive error for a tool whose job is not missing things. Both tiers are kept; only what each may escalate to differs.

And it collects the raw material for the judgment pass: per merged sibling, the paragraphs whose wording promises a closed world (allowlist, taxonomy, exhaustive, must be one of…), and for the feature under review, the paragraphs promising an unbounded one (page-derived, mined, operator-editable…). It deliberately does not pair them — that call is semantic.

"Merged" is decided from git (git ls-tree <base-ref>), not from prose. A spec directory that exists only in your working tree is a peer under construction, not a promise already made, and is reported separately.

2. Judgment — commands/cross-spec.md

Runs after /speckit-analyze, asks the assumption question of each merged sibling invariant, and classifies: HOLDS · SILENTLY BROKEN · AMENDED · CITED_BUT_ABSENT. AMENDED requires the amendment to be in the sibling's artefact — an amendment living only in the changing document has not been made.

Every finding names who inherits the contradiction, and says plainly where the fix is governance, not refactoring: a cross-spec amendment changes another spec's promises and needs that spec's owner, not a code edit.

Install

specify extension add --dev /path/to/spec-kit-coherence     # local checkout
specify extension add --from <release-url>                  # from a release

Then copy coherence-config.template.yml to coherence-config.yml in the installed extension directory and edit. Every key is optional; an unparseable file is ignored rather than fatal.

Use

Automatic: the manifest registers a non-optional after_analyze hook, so the pass runs as the second half of /speckit-analyze without anyone having to remember it. That is deliberate — an optional: true hook only prints To execute: … and moves on, so the judgment half would run about as often as somebody thinks to type it. Set enabled: false on the hook entry in .specify/extensions.yml if you would rather invoke it by hand.

Auto-running the hook is not enforcement, and should not be mistaken for it. Spec Kit hooks are prompt-level and fail silently on unparseable YAML, which is why the two mechanical defects are gated in CI instead (see below).

Manual:

/speckit.coherence.cross-spec
/speckit.coherence.cross-spec docs/specs/192-intent-mining

Deterministic half alone, which is how CI should invoke it:

node scripts/enumerate-siblings.js \
  --feature-dir docs/specs/192-intent-mining \
  --specs-root docs/specs --base-ref origin/main \
  --format markdown --fail-on-defect

--fail-on-defect exits 1 when a mechanical defect is found. Without it the script always exits 0 — fail-open, so the SDLC is never blocked by this check.

Noise control, which matters more than it sounds: a mature spec says "closed" or "allowlist" dozens of times in passing. --max-hits-per-sibling (default 5) and --load-bearing-only narrow the output; load-bearing lines sort first and the suppressed count is always printed, so it narrows without hiding. Measured on a real 8-sibling feature: 72 closure lines → 15, with the invariant that motivated the whole extension still the top hit for its sibling.

Optional: draft the remediation spec

Some findings are closed by editing a sibling. Others reveal that the reconciliation is new work: closure was the sibling's safety mechanism, and if the feature removes closure, something has to replace it — something that does not exist yet.

For findings at or above remediation_min_severity (default HIGH), the command offers to draft a remediation spec via /speckit-specify, carrying the finding's own evidence into the description. It only offers; it never writes without an explicit yes, and it stops after the draft — no plan, no tasks, no implementation. Decline and the finding is recorded as deferred with a named owner, because a finding that is neither fixed nor owned is one the next incident rediscovers.

Set remediation_spec_enabled: false to turn the offer off entirely.

Memory hook-in

When the memory extension is installed, the command recalls prior sessions before judging, scoped to the feature and its siblings.

It is used for exactly one thing: telling SILENTLY BROKEN from AMENDED. An amendment that was discussed and written into the sibling is AMENDED. One that was discussed and never written is SILENTLY BROKEN — and the recall is evidence the team knew, which raises the severity rather than lowering it. Without the memory extension the step is skipped, not failed.

Known limits

  • Hooks are prompt-level, not a runtime dispatcher. The vendored command markdown instructs the agent to read .specify/extensions.yml; unparseable YAML fails silently. Nothing that must not be skipped should depend on the hook — put the mechanical half in CI via --fail-on-defect.
  • Current Spec Kit templates do not sort hooks by priority, and condition causes a hook to be skipped rather than evaluated. This extension depends on neither.
  • Vocabulary matching is a filter, not a finding. An invariant can assume a closed world without using any of the listed words — the motivating case never wrote "assumes the intent set is finite." Extend the vocabulary with terms your own specs coined via closure_vocabulary_extra.
  • Only the first max_vocabulary_hits (default 25) paragraphs per document are collected.
  • The prose/code split is positional, not semantic. A number in running prose that is genuinely illustrative still gates, and a real dependency mentioned only inside a fence stays an observation. Measured over a 169-spec corpus this classified every case correctly, but the rule is a heuristic: if a citation matters, write it in prose, where it is gated.

Development

node --test test/*.test.js

The tests build real git repositories with real spec directories and a real merge base. merged is read from git, so mocking git out would leave the interesting half untested.

License

MIT

About

Spec Kit extension: checks whether merged sibling specs' invariants still hold after a feature's change, and whether the amendment was written where it belongs

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Used by

Contributors

Languages