Repository navigation
refactor(workflows): let the evaluator report its own leaves (#4274) - #4460
Conversation
…4274) _unresolvable_term answered one question -- does every operand in this condition resolve to something? -- by walking the expression itself: filters, then or/and/not, then comparisons, then list literals, down to the leaves. That walk was a second implementation of the parsing in _evaluate_simple_expression, kept in step with it by hand. Two helpers existed only to restate rules the evaluator already had. _looks_numeric mirrored the float()-only-when-a-dot-is-present rule because a bare float() accepts 1e3 and the evaluator does not. _is_literal mirrored the matching-close-is-the-final-character string test because startswith/endswith accepts 'a' 'b' and the evaluator does not. Both docstrings said "mirror the evaluator exactly", which is the tell: when the two drift nothing breaks loudly, the gate just answers wrongly, and the wrong answer is a paste-ready correction that inverts a condition. Seven of the nine findings in github#4230 were the same defect wearing different clothes -- the gate disagreeing with the evaluator about where the operands are. Each round fixed one shape. Nothing stopped a tenth. _evaluate_simple_expression has exactly one place where a substring stops being grammar and becomes a name to resolve: its final line, _resolve_dot_path. Literals return before it; operands, filter arguments and list elements all arrive there by construction. Record the leaf there, behind a ContextVar that is None outside a probe, and the gate applies namespace rules to that list instead of re-deriving it. It now contains no grammar at all. Two properties this rests on, both asserted rather than assumed: * or/and are not short-circuited -- both sides are evaluated and only then combined -- so a leaf is recorded whatever the other side is worth. If that ever changes the gate would go quietly blind, so there is a test for it. * A probe run can raise on its own placeholder values. The leaves seen before that point are real, so they are kept rather than discarded; discarding them would lose `bogus` in `inputs.tags | join(bogus)`, which an earlier round of github#4230 had to add by hand. expressions.py is 109 lines lighter and 84 heavier. All 336 existing tests pass unchanged, including the 20 cases of test_operands_must_be_literals_or_known_paths that took eight rounds to get right. test_literal_test_mirrors_the_evaluator tested the mirror, so it becomes test_literal_handling_comes_from_the_evaluator and asserts the same knowledge about 1e3 and 'a' 'b' through the gate instead. Four mutations, each killed by the tests that should kill it -- removing the leaf report alone turns 38 red. ruff 0.15.0 clean.
|
Thanks — this is the right root-cause fix: having the evaluator report its own leaves instead of |
The gate no longer restates the operator grammar, but it still restated the shape of a path segment: _PATH_SEGMENT and an inline fullmatch both described the index form that _resolve_dot_path matches with its own regex. Three copies of one rule, kept in step by hand -- the same drift this refactor set out to remove, one layer down. Name the form once as _INDEXED_SEGMENT beside _resolve_dot_path and have the gate ask it. Behaviour is unchanged: the regex is copied verbatim. What changes is that widening indexing now reaches the gate for free.
Two regression tests for the property this refactor is for, both of which a second copy of the grammar in the gate would break while every existing test stayed green: - widening _INDEXED_SEGMENT alone reaches the gate (the negative-index shape from github#4416) - when the evaluator stops treating something as a leaf, the gate stops checking it, with no gate edit (the grouped-operand shape from github#4417) Both were checked by reintroducing the drift: giving the gate its own segment regex again fails the first with the real message rather than an import error.
|
Thanks — both points addressed, and the second one turned up something I had missed in my own PR. (1) AI disclosure — added to the PR body, and it applies to this comment too. I used Claude Code as a coding agent for the code, the tests and the measurements below, reviewing and directing throughout. Sorry for the omission on the original submission; an oversight rather than an attempt to hide it. (2) The coordination question. I did not want to answer this from reading, so I applied the evaluator-side hunks only of #4416 and #4417 on top of this branch — no edit to
So the answer is yes, but only after a fix I had to add. My refactor as you first saw it did not cover the negative-index case. The gate had stopped restating the operator grammar, but it still restated the shape of a path segment — The parenthesised case needed nothing — it already worked, because once the evaluator stops treating a group as a leaf, the leaf sink stops reporting it. One correction to my own framing, since it bears on your sequencing: this refactor does not fix their bugs. On tests: Happy to rebase whenever suits the order you pick. |
|
Thanks — the disclosure's added, and I really appreciate you actually running the #4416/#4417-hunks experiment rather than just asserting compatibility. That settles the sequencing: I'll land this refactor first, then #4416/#4417 rebase down to just their evaluator hunks. No worries on the original disclosure omission. Approving the CI run; once it's green I'll review for merge. |
There was a problem hiding this comment.
🟡 Changes recommended
Probe exceptions can prematurely stop traversal and hide later unresolved leaves.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
Refactors condition validation to derive resolvable leaves directly from the expression evaluator, preventing duplicated grammar rules.
Changes:
- Adds ContextVar-based evaluator leaf collection.
- Shares indexed path-segment parsing between evaluation and validation.
- Adds regression and sink-isolation tests.
File summaries
| File | Description |
|---|---|
src/specify_cli/workflows/expressions.py |
Implements evaluator-driven leaf reporting and shared path validation. |
tests/unit/test_condition_expression_block.py |
Tests leaf collection, sink cleanup, and shared evaluator definitions. |
Review details
- Files reviewed: 2/2 changed files
- Comments generated: 1
- Review effort level: Balanced
💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
|
The re-review found a real correctness bug in the refactor: catching the first probe-value exception stops leaf collection, so later valid operands vanish — e.g. |
The refactor stopped the leaf walk at the first exception a probe value
raised, so every leaf further along the chain was lost. That is the one
thing the collection exists to report, and it was a step backwards from
the hand-written walk this PR replaces:
inputs.blob | from_json | contains(bogus)
origin/main reports 'bogus'
this PR before the fix MISSED
this PR after the fix reports 'bogus'
from_json receives the probe placeholder mapping and raises; the walk ended
there and contains(bogus) was never reached.
Carry on past a failing filter while the sink is armed. _apply_filter
evaluates a filter argument before it can raise on the value, so the failing
segment's own leaves are already recorded; a fresh placeholder goes into the
next filter, matching what the probe namespace hands out.
Scoped to the probe: the sink is armed only by _collect_leaves, and
_evaluator_rejects runs its own probe without it, so a mis-wired filter is
still rejected and a real evaluation still raises rather than quietly
returning the unfiltered value.
|
Fixed in AI disclosure, per CONTRIBUTING: I used Claude Code as a coding agent for this fix, its tests and the measurements below, reviewing and directing throughout. What it was
The hand-written walk this PR deletes read
The fixCarry on past a failing filter, but only while the sink is armed: value = _evaluate_simple_expression(head, namespace)
sink = _leaf_sink.get()
for segment in segments[1:]:
if sink is None:
value = _apply_filter(value, segment.strip(), namespace)
continue
try:
value = _apply_filter(value, segment.strip(), namespace)
except Exception: # noqa: BLE001 - probe values, not the author's text
value = _ProbeNamespace()Two things make this safe rather than a broad swallow:
A fresh TestsThree added to
Mutation, three ways:
The third one matters most: without it the fix could have been a blanket swallow that passed the first two.
Ready for re-review whenever suits you. Happy to keep this as a separate commit or squash it into the refactor, whichever you prefer for the |
There was a problem hiding this comment.
🟢 Approval recommended
The refactor preserves validation behavior while removing duplicated parsing rules and includes focused regression coverage.
Review details
- Files reviewed: 2/2 changed files
- Comments generated: 0 new
- Review effort level: Balanced
|
Thanks for your contributions here, @ntdatt812 — a quick note on review prioritization. You currently have 5 open pull requests, above the three-open-PR guideline in CONTRIBUTING. This isn't a freeze, and it doesn't hold up this PR: #4460 is in good shape — green CI and an approving review — and the (Drafted with AI assistance — GitHub Copilot.) |
|
Thank you! |
* [extension] Update Spec Kit Schedule extension to v0.7.4 (#4498) * Update Spec Kit Schedule extension to v0.7.4 Update schedule extension submitted by @jfranc38:\n- extensions/catalog.community.json (version, download_url, metadata)\n- docs/community/extensions.md community extensions table\n\nCloses #4457\n\nCo-authored-by: Copilot <[email protected]>\nAssisted-by: GitHub Copilot (model: gpt-5.2-codex, autonomous) * Apply suggestion from @KSchlobohm --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Ken Schlobohm <[email protected]> * chore: shorten stale timeline to 60 days stale, 30 days to close (#4503) Update the stale workflow so issues and PRs are marked stale after 60 days of inactivity and closed 30 days later (90 days total), down from 150/30. Messages updated to match. Assisted-by: GitHub Copilot (model: Claude Opus 4.8, autonomous) Co-authored-by: Copilot App <[email protected]> * docs(core): SPECIFY_FEATURE sets the feature label, not the feature directory (#3786) * docs(core): SPECIFY_FEATURE sets the feature label, not the feature directory docs/reference/core.md told users to set SPECIFY_FEATURE "to the feature directory name ... to work on a specific feature when not using Git branches". That does not work: SPECIFY_FEATURE only feeds get_current_branch / Get-CurrentBranch (the feature *label*). The directory comes from SPECIFY_FEATURE_DIRECTORY or .specify/feature.json. Verified on main with the real helper -- with ONLY SPECIFY_FEATURE set: $ SPECIFY_FEATURE=001-photo-albums ... get_feature_paths ERROR: Feature directory not found. Set SPECIFY_FEATURE_DIRECTORY or run the specify command to create .specify/feature.json. exit=1 $ SPECIFY_FEATURE_DIRECTORY=specs/001-photo-albums ... get_feature_paths FEATURE_DIR -> <resolved> CURRENT_BRANCH -> 001-photo-albums The code's own error message points at the other variable, and the doc's own "Two resolution axes" note directly below already says the feature is selected by SPECIFY_FEATURE_DIRECTORY / .specify/feature.json -- so the table row contradicted both the code and the paragraph under it. Describe what the variable actually does, note that /speckit.specify and the Git extension normally set it, and point at the directory axis. Docs only. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> * docs(core): describe SPECIFY_FEATURE as an explicit label override The row still misstated when and how the label is applied: * "when there is no Git branch context" — get_current_branch and Get-CurrentBranch never inspect Git at all. They return the variable verbatim when set, and otherwise fall back to the basename of the resolved feature directory. * "Normally set for you by /speckit.specify" — specify.md persists feature_directory to .specify/feature.json and never sets this variable. * The Bash and Python feature scripts can only *print* a commented export hint, because a child process cannot change its parent's environment. The PowerShell scripts do assign $env:SPECIFY_FEATURE, but only reach the caller when run inside the current session. Rewrite it as an explicit user-set label override, and distinguish the printed persistence hint from actually setting the caller's environment. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> * docs(core): attribute the label fallback to get_feature_paths, not get_current_branch The row said the label "falls back to the basename of the resolved feature directory" when SPECIFY_FEATURE is unset, and attributed that to get_current_branch / Get-CurrentBranch. Those helpers return an EMPTY string when the variable is unset — scripts/bash/common.sh:87 says so outright ("Return empty to signal 'unknown'") and scripts/python/common.py is `return os.environ.get("SPECIFY_FEATURE", "")`. The basename substitution happens later, in get_feature_paths / Get-FeaturePaths, after the feature directory has been resolved (scripts/python/common.py:168-169). Measured: get_current_branch (unset) -> [] get_current_branch (set) -> [my-label] get_feature_paths CURRENT_BRANCH -> [001-photo-albums] So a caller invoking the named helpers directly does not get the fallback. Distinguish the two behaviours. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> * docs(core): name the PowerShell helper Get-FeaturePathsEnv The doc cited `Get-FeaturePaths`, which does not exist. The PowerShell twin of `get_feature_paths` is `Get-FeaturePathsEnv` (scripts/powershell/common.ps1:152); there is no bare `Get-FeaturePaths` anywhere in the tree. Verified every function name the entry cites now resolves against the scripts: get_current_branch, Get-CurrentBranch, get_feature_paths, Get-FeaturePathsEnv. The quoted resolution error is verbatim from scripts/bash/common.sh:206. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> --------- Co-authored-by: Claude Opus 5 (1M context) <[email protected]> * fix(events): cap stdin in the generated dispatcher, not just the CLI command (#4337) * fix(events): cap stdin in the generated dispatcher, not just the CLI command The #3857 fix capped stdin at 1 MiB in `specify event run` (src/specify_cli/commands/event.py), but that command is not the code path native hooks actually invoke. Every installed integration writes a self-contained `.specify/events.py` dispatcher (the `_EVENTS_DISPATCHER_TEMPLATE` string in src/specify_cli/events.py) that native hook configs call directly, and its `main()` did: payload = sys.stdin.read() if not sys.stdin.isatty() else "{}" with no size cap at all — the exact DoS #3857 was meant to close, wide open on the primary invocation path. `specify event run` is a secondary/manual entry point; the generated dispatcher is what actually runs on every session_start/pre_tool_use/etc. hook fire in real usage. Fix: apply the same byte-capped read (from the binary buffer, so the cap counts encoded bytes rather than decoded characters — matching the just-merged fix for the CLI command) inside the dispatcher template, so every newly-installed or refreshed dispatcher enforces the limit. ## Test plan - Added 3 tests in tests/integrations/test_events.py::TestCommandRunner: an oversized payload exits 1 with the limit message instead of running unbounded, a multibyte payload (~300k emoji, ~1.14 MiB UTF-8 but only 300k characters) is still rejected by the byte-based cap, and a normal under-the-cap payload still reaches the handler script unchanged. - Verified both new failing-without-fix tests via test-the-test (stashed the src fix): the oversized-payload test failed because the dispatcher silently accepted the full payload and returned "not found" instead of exiting 1 with the limit message — reproducing the exact bug. - Ran the full tests/integrations/test_events.py suite (124/128 pass; the remaining 4 are the pre-existing Windows symlink-elevation failures unrelated to this change). Co-Authored-By: Claude Sonnet 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01PJHJ2dHP2RVCNncHqN8Qm9 * fix(events): pin utf-8 encoding on the handler subprocess in both dispatch paths Addresses Copilot review feedback on PR #4337: Both `_run_inline` (the generated dispatcher's stdlib fallback) and `resolve_and_run_event_command` (the delegated/CLI-native path) decode stdin explicitly as utf-8, then pass that string to the handler via `subprocess.run(..., text=True)` with no explicit `encoding=`. Without one, `text=True` re-encodes the payload for the child's stdin using `locale.getpreferredencoding()` — on Windows that's commonly the ANSI codepage, not UTF-8 — so a non-ASCII payload byte (e.g. "é") reaches the handler as the wrong byte, corrupting JSON for handlers that expect UTF-8. Pin `encoding="utf-8"` on both subprocess.run calls so the decode and re-encode agree. Also rewrote `test_dispatcher_underlimit_stdin_still_runs` (previously skipped entirely on Windows via a POSIX-only `sh` handler) to use a cross-platform Python handler and assert byte-for-byte fidelity of a non-ASCII payload, and added test_dispatcher_inline_fallback_preserves_non_ascii_payload, which forces the `_run_inline` fallback (never reached in a dev environment where specify_cli is importable, since the dispatcher always delegates first) so that path's fix is independently verified too. Co-Authored-By: Claude Sonnet 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01NhR6g8xT8at5pPMhkrC3e2 --------- Co-authored-by: Claude Sonnet 5 <[email protected]> * chore: release 1.0.6, begin 1.0.7.dev0 development (#4511) * chore: bump version to 1.0.6 * chore: begin 1.0.7.dev0 development --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * chore: refresh bug-assess workflow with gh-aw v0.88.7 (#4497) * chore: refresh bug-assess workflow with gh-aw v0.88.7 Regenerate bug-assess and update compiler-managed metadata. Assisted-by: GitHub Copilot (model: GPT-6 Astra, autonomous) Co-authored-by: Copilot <[email protected]> * fix: restore immutable bug-assess setup action pin Regenerate with gh-aw v0.88.7 and working GitHub authentication so setup references resolve to the release commit. Assisted-by: GitHub Copilot (model: GPT-6 Astra, autonomous) Co-authored-by: Copilot <[email protected]> * chore: remove duplicate workflow attributes rule Restore .gitattributes to its pre-PR contents while retaining the existing generated-workflow attributes. Assisted-by: GitHub Copilot (model: GPT-6 Astra, autonomous) Co-authored-by: Copilot <[email protected]> --------- Co-authored-by: Copilot <[email protected]> * docs: exempt repository maintenance workflows from PR throttle (#4499) * docs: exempt repository maintenance workflows from PR throttle Keep contributor confirmation requirements while allowing verified repository-owned gh-aw maintenance runs on behalf of CODEOWNERS to create their configured PR outputs. Assisted-by: GitHub Copilot (model: GPT-6 Astra, autonomous) Co-authored-by: Copilot <[email protected]> * docs: simplify maintenance workflow confirmation exception Limit the policy change to one sentence per document; retain existing review prioritization and author-over-cap guidance. Assisted-by: GitHub Copilot (model: GPT-6 Astra, autonomous) Co-authored-by: Copilot <[email protected]> --------- Co-authored-by: Copilot <[email protected]> * feat: add JSON output to preset and extension lists (#4218) * feat(cli): add JSON output for installed lists * fix(cli): preserve installed source provenance in JSON Preserve valid catalog provenance in installed preset and extension JSON output while retaining the local fallback for missing, legacy, unknown, and malformed records. Carry raw registry source metadata through healthy and corrupt manager records, whitelist the public kind/catalog shape in the shared adapter, and document and test the contract without changing provenance producers. * fix(cli): persist catalog provenance across install paths Propagate normalized catalog names through preset and extension install, init, bundler refresh, archive, and update paths while preserving local fallbacks and deterministic JSON ordering. * fix(cli): serialize installed-list usage errors as JSON Handle parse-time Click usage errors for preset and extension list when the raw --json flag is present, preserving stderr-only output and exit 2 in either flag order. Document and test the contract. * fix(cli): support Typer's vendored usage errors Catch parse failures from Typer's vendored Click implementation while retaining a narrow fallback for pre-vendoring Typer releases. Normalize ANSI only in human-output assertions. * fix: count extension hook events in JSON output Use one event-key count for the legacy extension list record and public JSON response. Update the multi-entry regression to preserve that contract. --------- Co-authored-by: root <[email protected]> * Add Product Definition as Code (PDaC) extension to community catalog (#4514) Add pdac extension submitted by @juangcarmona to the community catalog and documentation.\n\nCloses #4454\nCo-authored-by: Copilot <[email protected]>\nAssisted-by: GitHub Copilot (model: gpt-5.2-codex, autonomous) Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * [preset] Add Secure Development Assurance Governance preset (#4513) * Add Secure Development Assurance Governance preset to community catalog Add secure-development-assurance-governance preset submitted by @hindermath to: - presets/catalog.community.json (alphabetical order) - docs/community/presets.md community presets table Closes #4455 Assisted-by: GitHub Copilot (model: gpt-5.2-codex, autonomous) Co-authored-by: Copilot <[email protected]> * Apply suggestion from @KSchlobohm * Update Secure Development Assurance Governance entry Co-authored-by: Copilot Autofix powered by AI <[email protected]> --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Copilot <[email protected]> Co-authored-by: Ken Schlobohm <[email protected]> Co-authored-by: Copilot Autofix powered by AI <[email protected]> * refactor(workflows): let the evaluator report its own leaves (#4274) (#4460) * refactor(workflows): let the evaluator report its own leaves (#4274) _unresolvable_term answered one question -- does every operand in this condition resolve to something? -- by walking the expression itself: filters, then or/and/not, then comparisons, then list literals, down to the leaves. That walk was a second implementation of the parsing in _evaluate_simple_expression, kept in step with it by hand. Two helpers existed only to restate rules the evaluator already had. _looks_numeric mirrored the float()-only-when-a-dot-is-present rule because a bare float() accepts 1e3 and the evaluator does not. _is_literal mirrored the matching-close-is-the-final-character string test because startswith/endswith accepts 'a' 'b' and the evaluator does not. Both docstrings said "mirror the evaluator exactly", which is the tell: when the two drift nothing breaks loudly, the gate just answers wrongly, and the wrong answer is a paste-ready correction that inverts a condition. Seven of the nine findings in #4230 were the same defect wearing different clothes -- the gate disagreeing with the evaluator about where the operands are. Each round fixed one shape. Nothing stopped a tenth. _evaluate_simple_expression has exactly one place where a substring stops being grammar and becomes a name to resolve: its final line, _resolve_dot_path. Literals return before it; operands, filter arguments and list elements all arrive there by construction. Record the leaf there, behind a ContextVar that is None outside a probe, and the gate applies namespace rules to that list instead of re-deriving it. It now contains no grammar at all. Two properties this rests on, both asserted rather than assumed: * or/and are not short-circuited -- both sides are evaluated and only then combined -- so a leaf is recorded whatever the other side is worth. If that ever changes the gate would go quietly blind, so there is a test for it. * A probe run can raise on its own placeholder values. The leaves seen before that point are real, so they are kept rather than discarded; discarding them would lose `bogus` in `inputs.tags | join(bogus)`, which an earlier round of #4230 had to add by hand. expressions.py is 109 lines lighter and 84 heavier. All 336 existing tests pass unchanged, including the 20 cases of test_operands_must_be_literals_or_known_paths that took eight rounds to get right. test_literal_test_mirrors_the_evaluator tested the mirror, so it becomes test_literal_handling_comes_from_the_evaluator and asserts the same knowledge about 1e3 and 'a' 'b' through the gate instead. Four mutations, each killed by the tests that should kill it -- removing the leaf report alone turns 38 red. ruff 0.15.0 clean. * refactor(workflows): let _resolve_dot_path define the indexed segment The gate no longer restates the operator grammar, but it still restated the shape of a path segment: _PATH_SEGMENT and an inline fullmatch both described the index form that _resolve_dot_path matches with its own regex. Three copies of one rule, kept in step by hand -- the same drift this refactor set out to remove, one layer down. Name the form once as _INDEXED_SEGMENT beside _resolve_dot_path and have the gate ask it. Behaviour is unchanged: the regex is copied verbatim. What changes is that widening indexing now reaches the gate for free. * test(workflows): pin that the gate reads the evaluator's definitions Two regression tests for the property this refactor is for, both of which a second copy of the grammar in the gate would break while every existing test stayed green: - widening _INDEXED_SEGMENT alone reaches the gate (the negative-index shape from #4416) - when the evaluator stops treating something as a leaf, the gate stops checking it, with no gate edit (the grouped-operand shape from #4417) Both were checked by reintroducing the drift: giving the gate its own segment regex again fails the first with the real message rather than an import error. * fix(workflows): keep collecting leaves after a probe error The refactor stopped the leaf walk at the first exception a probe value raised, so every leaf further along the chain was lost. That is the one thing the collection exists to report, and it was a step backwards from the hand-written walk this PR replaces: inputs.blob | from_json | contains(bogus) origin/main reports 'bogus' this PR before the fix MISSED this PR after the fix reports 'bogus' from_json receives the probe placeholder mapping and raises; the walk ended there and contains(bogus) was never reached. Carry on past a failing filter while the sink is armed. _apply_filter evaluates a filter argument before it can raise on the value, so the failing segment's own leaves are already recorded; a fresh placeholder goes into the next filter, matching what the probe namespace hands out. Scoped to the probe: the sink is armed only by _collect_leaves, and _evaluator_rejects runs its own probe without it, so a mis-wired filter is still rejected and a real evaluation still raises rather than quietly returning the unfiltered value. * Fix catalog-latest-url-bypass: require tag-pinned catalog download URLs (#4194) * fix: require tag-pinned catalog download URLs (#4185) Reject floating releases/latest URLs in the community catalog agent workflows and require the URL tag to match the submitted version. Refs #4185 Assisted-by: Cursor Grok 4.6 (supervised) Signed-off-by: shaurya2k06 <[email protected]> * test: assert both catalog tag forms together Separate substring checks for vX.Y.Z and X.Y.Z were not independent. Refs #4185 Assisted-by: Cursor Grok 4.6 (supervised) Signed-off-by: shaurya2k06 <[email protected]> * fix: allow scoped catalog tags and same-repo download URLs Keep tag-pinned URLs but accept suffixes like aide-v1.0.0, require download_url to match the submitted repository, and treat sha256 as optional follow-up rather than a hard catalog gate. Refs #4185 Assisted-by: Cursor Grok 4.6 (supervised) Signed-off-by: shaurya2k06 <[email protected]> * fix: keep collecting catalog validation failures after a latest URL Skip the HTTP check for a floating releases/latest URL without aborting the rest of Step 2. Refs #4185 Assisted-by: Cursor Grok 4.6 (supervised) Signed-off-by: shaurya2k06 <[email protected]> * fix: gate catalog SHA checks on URL pinning Keep archive fetching and optional hash verification behind successful URL pinning checks. Refs #4185 Assisted-by: Codex (model: GPT-5, autonomous) Signed-off-by: shaurya2k06 <[email protected]> --------- Signed-off-by: shaurya2k06 <[email protected]> * [extension] Add GitHub Issue Triage extension to community catalog (#4539) * Add GitHub Issue Triage extension to community catalog Add gh-triage extension submitted by @arrrrny to: - extensions/catalog.community.json (alphabetical order) - docs/community/extensions.md community extensions table Closes #4339 Co-authored-by: Copilot <[email protected]> Assisted-by: GitHub Copilot (model: gpt-5.2-codex, autonomous) * Apply suggestion from @KSchlobohm --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Copilot <[email protected]> Co-authored-by: Ken Schlobohm <[email protected]> * fix(workflows): harden community submission workflows (#4510) * fix(workflows): harden community submission workflows per security review Address two hardening suggestions from a GitHub Security Lab PVR against the community catalog submission workflows (bundle, extension, preset): 1. Validate the sanitized event snapshot instead of the live issue. Step 1 now consumes ${{ steps.sanitized.outputs.text }} — the documented gh-aw sanitized full-context output — rather than instructing the agent to re-fetch the issue body, which could change between the maintainer applying the label and the agent reading it. This ties validation to the triggering submission. 2. Make threat detection block safe outputs. Add safe-outputs.threat-detection.continue-on-error: false so a detected threat fails the run instead of only warning and still producing a draft PR. Recompiled the three .lock.yml files with gh-aw v0.79.8. (Finding #2, create-pull-request allowed-files, is already implemented on all three workflows upstream, so no change was needed there.) Assisted-by: GitHub Copilot (model: Claude Opus 4.8, autonomous) Co-authored-by: Copilot App <[email protected]> * fix(workflows): preserve action pins and add fail-closed regression test Address PR review feedback: - Restore actions/checkout (v7.0.1) and actions/setup-node (v7.0.0) pins in the three regenerated lock files. A local recompile had resolved older cached pins; the lock files now differ from the base only by the intended snapshot and threat-detection changes. - Add a regression test asserting each community submission workflow enables threat detection with continue-on-error: false in source and compiles to the fail-closed detection gate, so a later regeneration cannot silently restore warning-only behavior. Assisted-by: GitHub Copilot (model: Claude Opus 4.8, autonomous) Co-authored-by: Copilot App <[email protected]> * fix(workflows): drop sanitized-snapshot input, keep fail-closed detection The regenerated sanitized snapshot (steps.sanitized.outputs.text) redacts any HTTPS host absent from the workflow's allowed-domains list. These submission workflows record off-allowlist URLs verbatim (extension homepage/documentation/ changelog, bundle required component catalogs, and the proposed catalog-entry JSON), so a snapshot input would corrupt otherwise-valid submissions. Revert Step 1 to reading the triggering issue and keep the separate maintainer PR review as the control for issue edits. The fail-closed threat-detection change (continue-on-error: false) and its regression test are retained. Recompiled the three lock files; action pins and the pin database are unchanged from the base. Assisted-by: GitHub Copilot (model: Claude Opus 4.8, autonomous) Co-authored-by: Copilot App <[email protected]> --------- Co-authored-by: Copilot App <[email protected]> * fix: require explicit refresh for bundle manifest changes (#4477) * fix: reject bundle version changes during install * fix: support explicit local bundle refresh Assisted-by: OpenAI Codex (autonomous) * fix: clarify catalog requirements for local bundle refresh Exercise local manifest, directory, and ZIP refresh through the real extension installer with deterministic catalog artifacts. Preserve state on offline failure and verify the online retry refreshes the owned version. Assisted-by: OpenAI Codex (model: GPT-6 Astra, autonomous) * fix: require refresh for owned bundle component changes Compare recorded component metadata with the requested plan before primitive operations. Reject changed pins, sources, preset options, and removals even when the bundle version is unchanged. Preserve idempotent installs, reordering, and additions; exercise refresh through lifecycle and real-installer CLI regressions. Assisted-by: OpenAI Codex (autonomous) * feat: add `specify artifact` introspection (#4305) * Add deterministic contribution IDs and stack lookup IDs for resolved artifacts Every command, template, script, and hook contribution returned by preset and extension manifest surfaces now carries a computed opaque identifier of the form {layer}:{sourceId}:{kind}:{name}, and every resolved artifact-stack layer carries a matching lookupId derived from the same recipe. Assisted-by: GitHub Copilot (model: GPT-5.6 Sol, autonomous) Co-authored-by: Copilot App <[email protected]> Identifiers are computed at read time from author-declared manifest content only. No paths, timestamps, or file-content hashes contribute to derivation, so identifiers are stable across machines, reinstalls, and directory moves. Nothing is persisted to .specify/ or any cache. Hooks that collide within a source on (eventName, command) get a 12-hex SHA-256 discriminator computed from the canonical JSON of the entry's declared fields minus eventName/command. Two hook entries with byte-identical remaining fields are rejected at manifest load because there is no meaningful way to distinguish them. The change is purely additive: all existing name-based resolution behaviour is preserved, and no consumer keys off the new id or lookupId fields. Co-authored-by: Copilot App <[email protected]> Assisted-by: GitHub Copilot (model: claude-opus-4.7, autonomous) * feat: add `specify artifact` command exposing composition stacks as JSON Adds a new `specify artifact` command group with two subcommands: * `specify artifact list --json` — flat inventory of every command, template, and script SpecKit exposes for the current project. Each row carries a stable `id` (`{kind}:{name}`), an author-declared `name`, its `kind`, and a `description` string that is never omitted (empty string when the author declared none). * `specify artifact info <name> --json` — the same row plus its full ordered composition `stack`: highest-priority contributor first, with `active` marking the winner `PresetResolver.resolve_content` would return and `hidden` marking rows shadowed by a higher-priority `replace`. Each stack entry carries a portable POSIX `manifestPath` (or `null` for the core baseline) and a `lookupId` from the contribution-id grammar so the output round-trips against `specify preset info` and `specify extension info`. The two commands share one strict JSON error envelope on stderr (`{ "error": "..." }`) with exit code 1 for the three logical errors (unknown artifact, ambiguous artifact, not a Spec Kit project) and exit code 2 for the "`--json` is required" usage error. stdout is always empty on error, so the two streams stay independently parseable. Implementation lives in a new `src/specify_cli/artifacts/` subpackage that mirrors the existing `presets/` and `extensions/` layout — pure logic in `__init__.py` and thin Typer wiring in `_commands.py`. The subpackage reuses `PresetResolver.collect_all_layers` for the actual composition math and only reshapes each layer into a `StackLayer` JSON row, so `active` and `hidden` stay in lockstep with the resolver's winner-selection logic. Skills (`.github/skills/**/SKILL.md`) are intentionally excluded from the inventory — they are integration-specific installation output, not a shipped asset family. The command still surfaces the underlying command that a skill was generated from. Tests: * `tests/test_artifact_command.py` — 32 tests: contract shape, sort order, empty-inventory behavior, kind-hint parsing, ambiguous-name error, unknown-artifact error, not-a-project error, skills exclusion, CLI wiring end-to-end (`--json` required, JSON envelope shape, stderr-only errors, empty stdout on error, UTF-8 with no BOM), and preset-replace hiding the core layer. * `tests/test_artifact_command_parity.py` — 6 tests: `manifestPath` uses forward slashes on every OS and is never absolute, the `active` row corresponds to the resolver's actual winner, and the pretty-printed JSON has no trailing whitespace and ends in exactly one newline. All 38 new tests pass. Full presets + extensions regression suite is green modulo pre-existing Windows-symlink-privilege failures that predate this branch. Assisted-by: GitHub Copilot (model: claude-opus-4.7, autonomous) Co-authored-by: Copilot App <[email protected]> Copilot-Session: 4a40fb96-1bbe-4fb2-99d8-411170046cb0 * Potential fix for pull request finding 'Module is imported with 'import' and 'import from'' Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com> * Potential fix for pull request finding 'Module is imported with 'import' and 'import from'' Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com> * Potential fix for pull request finding 'Unused import' Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com> * Project preset artifacts by entry type Assisted-by: GitHub Copilot (model: GPT-5.6, autonomous) Co-authored-by: nicolehaugen <[email protected]> * Represent project override artifact layers Assisted-by: GitHub Copilot (model: GPT-5.6, autonomous) Co-authored-by: nicolehaugen <[email protected]> * Preserve artifact JSON init-dir errors Assisted-by: GitHub Copilot (model: GPT-5.6, autonomous) Co-authored-by: nicolehaugen <[email protected]> * Canonicalize core script artifacts Assisted-by: GitHub Copilot (model: GPT-5.6, autonomous) Co-authored-by: nicolehaugen <[email protected]> * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <[email protected]> * Fix artifact inventory resolver filtering Assisted-by: GitHub Copilot (model: GPT-5.6, autonomous) Co-authored-by: nicolehaugen <[email protected]> * Add resolver tests for single-runtime core scripts Assisted-by: GitHub Copilot (model: GPT-5.6, autonomous) Co-authored-by: nicolehaugen <[email protected]> * Cache artifact resolver lookups Assisted-by: GitHub Copilot (model: GPT-5.6, autonomous) Co-authored-by: nicolehaugen <[email protected]> * Handle artifact resolver failures Assisted-by: GitHub Copilot (model: GPT-5.6, autonomous) Co-authored-by: nicolehaugen <[email protected]> * Document artifact resolution error Assisted-by: GitHub Copilot (model: GPT-5.6, autonomous) Co-authored-by: nicolehaugen <[email protected]> * Include convention-based artifacts in inventory Assisted-by: GitHub Copilot (model: GPT-5.6, autonomous) Co-authored-by: nicolehaugen <[email protected]> * Restore legacy flat core script lookup Assisted-by: GitHub Copilot (model: GPT-5.6, autonomous) Co-authored-by: nicolehaugen <[email protected]> * Extend convention discovery to presets in artifact inventory Co-authored-by: nicolehaugen <[email protected]> * Fix manifest path portability and export ArtifactResolutionError Co-authored-by: nicolehaugen <[email protected]> * Bound artifact manifest search to project root Assisted-by: GitHub Copilot (model: GPT-5.6, autonomous) Co-authored-by: nicolehaugen <[email protected]> * Cover project-root artifact manifests Assisted-by: GitHub Copilot (model: GPT-5.6, autonomous) Co-authored-by: nicolehaugen <[email protected]> * Handle directory artifact manifest lookups Assisted-by: GitHub Copilot (model: GPT-5.6, autonomous) Co-authored-by: nicolehaugen <[email protected]> * Fall back to top-level preset name in artifact stacks Assisted-by: GitHub Copilot (model: GPT-5.6, autonomous) Co-authored-by: nicolehaugen <[email protected]> * Include project-local core artifacts in inventory Co-authored-by: nicolehaugen <[email protected]> * Address inline review feedback on artifact resolver helpers Co-authored-by: nicolehaugen <[email protected]> * Reuse manifest/registry APIs in artifact contribution enumeration Co-authored-by: nicolehaugen <[email protected]> * Pass layer explicitly to _iter_pack_contributions instead of inferring from parent dir Co-authored-by: nicolehaugen <[email protected]> * Fix core command namespacing and validate names for kind-scoped lookups Assisted-by: GitHub Copilot (model: GPT-5.6, autonomous) Co-authored-by: nicolehaugen <[email protected]> * Skip manifest contributions without a usable identifier Assisted-by: GitHub Copilot (model: GPT-5.6, autonomous) Co-authored-by: nicolehaugen <[email protected]> * Hoist test-local imports to module scope in artifact/assets tests Assisted-by: GitHub Copilot (model: Claude Sonnet 4.6, autonomous) Co-authored-by: nicolehaugen <[email protected]> * fix: resolve artifact inventory and validation review regressions Co-authored-by: nicolehaugen <[email protected]> * perf: avoid duplicate read in core command inventory Co-authored-by: nicolehaugen <[email protected]> * fix: classify dotted override-only artifacts as commands Co-authored-by: nicolehaugen <[email protected]> * fix: accept single-segment artifact commands Co-authored-by: nicolehaugen <[email protected]> * fix: fail closed on corrupt artifact registries Co-authored-by: nicolehaugen <[email protected]> * fix: trust inventory for artifact info lookups Assisted-by: GitHub Copilot (model: GPT-5.6, autonomous) Co-authored-by: nicolehaugen <[email protected]> * fix: validate registry before artifact info Assisted-by: GitHub Copilot (model: GPT-5.6, autonomous) Co-authored-by: nicolehaugen <[email protected]> * fix: resolve artifact description by layer precedence, not enumeration order Assisted-by: GitHub Copilot (model: claude-sonnet-4.6, autonomous) Co-authored-by: nicolehaugen <[email protected]> * fix: validate subdir before wheel bundle lookup in _locate_core_asset_dir Co-authored-by: nicolehaugen <[email protected]> * fix: detect duplicate hooks after command canonicalization Assisted-by: GitHub Copilot (model: GPT-5.6, autonomous) Co-authored-by: nicolehaugen <[email protected]> * fix: reuse normalized hook entries for duplicate detection Assisted-by: GitHub Copilot (model: GPT-5.6, autonomous) Co-authored-by: nicolehaugen <[email protected]> * fix: align core command candidate ordering Assisted-by: GitHub Copilot (model: GPT-5.6, autonomous) Co-authored-by: nicolehaugen <[email protected]> * test: cover manifest-backed artifact parity Assisted-by: GitHub Copilot (model: GPT-5.6, autonomous) Co-authored-by: nicolehaugen <[email protected]> * fix: align artifact IDs with resolver identity Assisted-by: GitHub Copilot (model: GPT-5.6, autonomous) Co-authored-by: nicolehaugen <[email protected]> * fix: skip invalid local artifact name components Co-authored-by: nicolehaugen <[email protected]> * fix: filter invalid local artifact IDs from inventory Co-authored-by: nicolehaugen <[email protected]> * fix: align artifact preset enumeration with resolver Assisted-by: GitHub Copilot (model: GPT-5.6, autonomous) Co-authored-by: nicolehaugen <[email protected]> * test: remove tautological artifact tests and strengthen id assertion Assisted-by: GitHub Copilot (model: GPT-5.6, autonomous) Co-authored-by: nicolehaugen <[email protected]> * fix: preserve documented hook duplicate semantics Co-authored-by: nicolehaugen <[email protected]> * fix: dedupe hook contributions last-wins Assisted-by: GitHub Copilot (model: GPT-5.6, autonomous) Co-authored-by: nicolehaugen <[email protected]> * docs: clarify hook identifier deduplication Assisted-by: GitHub Copilot (model: GPT-5.6, autonomous) Co-authored-by: nicolehaugen <[email protected]> * docs: remove hook discriminator references Assisted-by: GitHub Copilot (model: GPT-5.6, autonomous) Co-authored-by: nicolehaugen <[email protected]> * style: space identifier declarations Assisted-by: GitHub Copilot (model: GPT-5.6, autonomous) Co-authored-by: nicolehaugen <[email protected]> * fix: address unresolved review feedback on PR #4305 - Validate preset-registry corruption in artifact catalog (fail closed with ArtifactResolutionError, mirroring the extension-registry check) and add ``PresetRegistry.is_corrupt`` in the shape of ``ExtensionRegistry.is_corrupt``. - ``_locate_core_asset_dir`` now falls through to the source checkout when a wheel bundle is present but missing the requested family subdirectory, matching the "wheel, then source" fallback pattern used by the sibling bundled-extension/workflow/preset resolvers. - Enforce the identifier component contract at the shared derivation boundary: ``derive_named_id`` / ``derive_hook_id`` now revalidate every input via ``validate_component`` so raw filesystem-derived names cannot produce non-round-trippable lookup ids. - Overwrite ``eventName`` in hook contributions from the containing hook key instead of ``setdefault`` so author-supplied fields cannot contradict the derived ``name`` / ``id`` metadata. - Manifest-declared preset and extension resolver layers now use the manifest's validated ``id:`` for the ``lookupId`` ``sourceId`` component, so the join to ``iter_contributions()`` stays direct when the installed directory was renamed. Convention-only contributions still fall back to the directory / registry key; directory identity is retained on the layer via ``source`` / ``extension_id`` / ``extension_dir``. - Artifact catalog reuses the manifest's own contribution ``id`` verbatim when yielding declared contributions so it stays consistent with the resolver. - Docs: clarify in ``docs/reference/presets.md`` and ``extensions/EXTENSION-API-REFERENCE.md`` that manifest contribution ``id`` and resolver ``lookupId`` share the same grammar but only join directly when the installed directory matches the manifest-declared ``id:``. - Restore the ``## File System Layout`` heading before the ``.specify/`` tree in ``extensions/EXTENSION-API-REFERENCE.md`` and add it to the ToC. - Use one consistent import style for ``specify_cli._assets`` in ``tests/test_assets.py`` (module import only) and update the existing test-suite entries whose behavior was locked to the resolver's old directory-key ``lookupId``. Co-authored-by: nicolehaugen <[email protected]> * fix: keep on-disk preset/extension identity separate from lookupId The resolver now emits manifest.id in the ``lookupId``'s ``sourceId`` component for manifest-declared preset and extension layers, so code that had been extracting the on-disk directory name from ``lookupId`` (in ``_derive_manifest_path`` and ``_build_stack``) now points to the wrong path when the installed directory was renamed. Carry the directory identity as explicit ``preset_id`` / ``pack_dir`` keys on preset layer dicts (extension layers already carried ``extension_id`` / ``extension_dir``). Update ``_derive_manifest_path`` and ``_build_stack`` to prefer those explicit keys before falling back to ``lookupId`` parsing, so the display name and manifest path in the stack row keep tracking the actual on-disk directory. Extend the mismatch tests to lock down that ``presetId`` and ``manifestPath`` point to the renamed on-disk directory even when ``lookupId`` uses the manifest id. Assisted-by: GitHub Copilot (model: GPT-5.6, autonomous) Co-authored-by: nicolehaugen <[email protected]> * fix: remove stale lookupId parsing fallback and tighten malformed lookupId validation Co-authored-by: nicolehaugen <[email protected]> * fix: route resolver core fallback through shared asset resolver, describe project overrides, document specify artifact Assisted-by: GitHub Copilot (model: claude-opus-4.7, autonomous) Co-authored-by: nicolehaugen <[email protected]> * refactor: drop redundant derive_named_id import-visibility assignment Assisted-by: GitHub Copilot (model: claude-opus-4.7, autonomous) Co-authored-by: nicolehaugen <[email protected]> * fix: align artifact inventory and lookup ID validation Address the latest review feedback for root-level legacy templates and unsupported lookup ID kinds. Assisted-by: GitHub Copilot (model: GPT-5.6 Luna, autonomous) Co-authored-by: Copilot App <[email protected]> Copilot-Session: 33a4ecee-211d-4be0-9842-7a433c5ada8f * fix: fail closed on malformed artifact registries Treat missing registry collection keys as corruption and map filesystem read failures to the artifact JSON error envelope. Assisted-by: GitHub Copilot (model: GPT-5.6 Luna, autonomous) Co-authored-by: Copilot App <[email protected]> Copilot-Session: 33a4ecee-211d-4be0-9842-7a433c5ada8f * fix: preserve convention artifact descriptions Use the existing artifact description extractors for convention-based preset and extension files. Assisted-by: GitHub Copilot (model: GPT-5.6 Luna, autonomous) Co-authored-by: Copilot App <[email protected]> Copilot-Session: 33a4ecee-211d-4be0-9842-7a433c5ada8f * fix: align artifact override resolution Assisted-by: GitHub Copilot (model: GPT-5.4, autonomous) Co-authored-by: nicolehaugen <[email protected]> * Refactor artifact inventory candidates Co-authored-by: nicolehaugen <[email protected]> * Address artifact inventory review Co-authored-by: nicolehaugen <[email protected]> * Address artifact inventory review Assisted-by: GitHub Copilot (model: GPT-5.4, autonomous) Co-authored-by: nicolehaugen <[email protected]> * Address artifact inventory review Assisted-by: GitHub Copilot (model: GPT-5.4, autonomous) Co-authored-by: nicolehaugen <[email protected]> * source-agnostic artifact IDs; built-in tier recognized by exclusion, never by name. Co-authored-by: nicolehaugen <[email protected]> * fix: reject malformed artifact layer provenance Co-authored-by: nicolehaugen <[email protected]> * Tighten artifact provenance handling Assisted-by: GitHub Copilot (model: GPT-5.4, autonomous) Co-authored-by: nicolehaugen <[email protected]> * Refactor shared asset directory lookup Assisted-by: GitHub Copilot (model: GPT-5.4, autonomous) Co-authored-by: nicolehaugen <[email protected]> * Document shared asset families Assisted-by: GitHub Copilot (model: GPT-5.4, autonomous) Co-authored-by: nicolehaugen <[email protected]> * Avoid full artifact content scans Assisted-by: GitHub Copilot (model: GPT-5.4, autonomous) Co-authored-by: nicolehaugen <[email protected]> * Clarify artifact resolution guard Assisted-by: GitHub Copilot (model: GPT-5.4, autonomous) Co-authored-by: nicolehaugen <[email protected]> * Restore resolver core provenance Assisted-by: GitHub Copilot (model: GPT-5.4, autonomous) Co-authored-by: nicolehaugen <[email protected]> * Reuse artifact inventory layers Assisted-by: GitHub Copilot (model: GPT-5.4, autonomous) Co-authored-by: nicolehaugen <[email protected]> * Simplify preset resolve assertion Assisted-by: GitHub Copilot (model: GPT-5.4, autonomous) Co-authored-by: nicolehaugen <[email protected]> * Restore source-agnostic artifact provenance Assisted-by: GitHub Copilot (model: GPT-5.4, autonomous) Co-authored-by: nicolehaugen <[email protected]> * artifact catalog: `id` is the source-agnostic round-trip key; `info` accepts `id`; docs and issue #4212 updated. Co-authored-by: nicolehaugen <[email protected]> * fix: keep layer_kind_from_lookup_id and derive_hook_id in agreement on hook layers Co-authored-by: nicolehaugen <[email protected]> * artifact: reuse shared project resolver, rename handlers, dedupe validation Co-authored-by: nicolehaugen <[email protected]> * fix: align artifact info existence and resolver naming Co-authored-by: nicolehaugen <[email protected]> * artifact: reuse PresetResolver.templates_dir in _project_core_asset_root Assisted-by: GitHub Copilot (model: claude-opus-4.7, autonomous) Co-authored-by: nicolehaugen <[email protected]> * fix: guard stale registry entries in artifact convention discovery Assisted-by: GitHub Copilot (model: claude-opus-4.7, autonomous) Co-authored-by: nicolehaugen <[email protected]> * fix: include stack in artifact list json Assisted-by: GitHub Copilot (model: GPT-5 Codex, autonomous) Co-authored-by: nicolehaugen <[email protected]> * docs: document artifact list stack records Assisted-by: GitHub Copilot (model: GPT-5.6, autonomous) Co-authored-by: nicolehaugen <[email protected]> * feat: add artifact layer source paths Assisted-by: GitHub Copilot (model: GPT-5.6, autonomous) Co-authored-by: nicolehaugen <[email protected]> * docs: clarify artifact sourcePath provenance Assisted-by: GitHub Copilot (model: GPT-5.6, autonomous) Co-authored-by: nicolehaugen <[email protected]> * refactor: clarify sourcePath derivation flow Assisted-by: GitHub Copilot (model: GPT-5.6, autonomous) Co-authored-by: nicolehaugen <[email protected]> * refactor: document artifact source path fallback Assisted-by: GitHub Copilot (model: GPT-5.6, autonomous) Co-authored-by: nicolehaugen <[email protected]> * refactor: expose registrar output path helper Assisted-by: GitHub Copilot (model: GPT-5.6, autonomous) Co-authored-by: nicolehaugen <[email protected]> * refactor: centralize registrar skill output check Assisted-by: GitHub Copilot (model: GPT-5.6, autonomous) Co-authored-by: nicolehaugen <[email protected]> * fix: only use materialized command output for the active stack row Assisted-by: GitHub Copilot (model: claude-sonnet-4.6, autonomous) Co-authored-by: nicolehaugen <[email protected]> * docs(artifacts): cross-link contribution identifier grammar Add a direct link from docs/reference/artifacts.md to the contribution-identifiers section of the extension API reference next to the existing presets.md link, so readers of the artifact CLI reference can find the id/lookupId grammar without re-deriving it here. Assisted-by: GitHub Copilot (model: claude-opus-4.7, autonomous) Co-authored-by: Copilot App <[email protected]> * fix(artifacts): enforce identifier grammar and gate manifestPath on declared contributions Three related correctness fixes for the artifact-stack pipeline surfaced during PR #4305 review: 1. _identifier.py: add source_id_from_lookup_id() helper mirroring layer_kind_from_lookup_id, and enforce the project/'_' sentinel in derive_named_id (project layer requires source_id == '_'; preset/extension layers reject '_'). Swap the split(':', 2)[1] call in artifacts/__init__.py to use the new helper so consumers no longer parse identifier grammar directly. 2. presets/__init__.py: thread a manifest_declared flag through collect_all_layers so downstream consumers can distinguish manifest-declared contributions from convention-only fallbacks. 3. artifacts/__init__.py: _derive_manifest_path returns None when the layer is not manifest-declared, so a stack row for a convention-only contribution no longer falsely reports a manifestPath pointing at a manifest that does not declare it. Tests: compact param-based coverage for source_id_from_lookup_id and derive_named_id sentinel rules; one preset + one extension test proving lookupId uses the manifest's validated id when it differs from the on-disk directory name; one end-to-end extension test proving a convention-only contribution reports manifestPath: null. Existing TestManifestPathPortability fixtures updated to set manifest_declared: True. Assisted-by: GitHub Copilot (model: claude-opus-4.7, autonomous) Co-authored-by: Copilot App <[email protected]> * chore(tests): remove trailing blank lines Assisted-by: GitHub Copilot (autonomous) Co-authored-by: Copilot <[email protected]> Copilot-Session: 33a4ecee-211d-4be0-9842-7a433c5ada8f * fix(presets): reuse parsed extension manifest identity Carry the validated extension manifest ID out of the manifest-first resolution helper so collect_all_layers does not re-read the manifest and fall back to a directory-based lookupId after a transient second-read failure. Assisted-by: GitHub Copilot (model: GPT-5.6 Sol, autonomous) Co-authored-by: Copilot App <[email protected]> Copilot-Session: 33a4ecee-211d-4be0-9842-7a433c5ada8f * fix(identifiers): enforce layer source sentinel when parsing Share the project underscore sentinel rule across named and hook constructors and lookupId parsing so malformed project and provider provenance is rejected consistently. Assisted-by: GitHub Copilot (model: GPT-5.6 Sol, autonomous) Co-authored-by: Copilot App <[email protected]> Copilot-Session: 33a4ecee-211d-4be0-9842-7a433c5ada8f * docs(identifiers): clarify built-in provenance contract Document that built-in artifact layers omit lookupId and round-trip through their source-agnostic public kind:name ID, while project overrides retain a synthetic stack identity. Assisted-by: GitHub Copilot (model: GPT-5.6 Sol, autonomous) Co-authored-by: Copilot App <[email protected]> Copilot-Session: 33a4ecee-211d-4be0-9842-7a433c5ada8f * fix(artifacts): preserve preset registry fallback Remove the artifact-specific preset corruption guard and retain Spec Kit's existing behavior of treating malformed preset registry data as an empty registry. Keep extension registry validation unchanged. Assisted-by: GitHub Copilot (model: GPT-5.6 Sol, autonomous) Co-authored-by: Copilot App <[email protected]> Copilot-Session: 33a4ecee-211d-4be0-9842-7a433c5ada8f * test(artifacts): drop preset corruption fallback coverage Do not establish a new artifact-specific contract test for the preset registry's pre-existing malformed-data fallback. Assisted-by: GitHub Copilot (model: GPT-5.6 Sol, autonomous) Co-authored-by: Copilot App <[email protected]> Copilot-Session: 33a4ecee-211d-4be0-9842-7a433c5ada8f * chore(changelog): remove manual unreleased entry Leave release-note generation to the existing release workflow, which derives versioned changelog entries from commit subjects. Assisted-by: GitHub Copilot (model: GPT-5.6 Sol, autonomous) Co-authored-by: Copilot App <[email protected]> Copilot-Session: 33a4ecee-211d-4be0-9842-7a433c5ada8f * docs(artifacts): clarify layer resolution semantics Document that active reflects Spec Kit's existing layer precedence rather than successful content composition, and limit artifact resolution failures to errors encountered while collecting the stack. Assisted-by: GitHub Copilot (model: GPT-5.6 Sol, autonomous) Co-authored-by: Copilot App <[email protected]> Copilot-Session: 33a4ecee-211d-4be0-9842-7a433c5ada8f * docs(artifacts): explain resolver reuse Document why artifact inventory resolves each candidate through Spec Kit's existing single-artifact path and defers unmeasured shared caching. Assisted-by: GitHub Copilot (model: GPT-5.6 Sol, autonomous) Co-authored-by: Copilot App <[email protected]> Copilot-Session: 33a4ecee-211d-4be0-9842-7a433c5ada8f * fix(presets): preserve legacy layer resolution Keep filesystem-derived project, preset, and extension layers resolvable when legacy names cannot be represented by the contribution-ID grammar. Such layers omit lookupId while manifest-declared contributions remain strict. Assisted-by: GitHub Copilot (model: GPT-5.6 Sol, autonomous) Co-authored-by: Copilot App <[email protected]> Copilot-Session: 33a4ecee-211d-4be0-9842-7a433c5ada8f * fix: preserve README convention resolution Keep root-level README templates aligned with the existing resolver and artifact inventory instead of introducing a filename-specific exclusion. Assisted-by: GitHub Copilot (model: GPT-5.6 Sol, autonomous) Co-authored-by: Copilot App <[email protected]> Copilot-Session: 33a4ecee-211d-4be0-9842-7a433c5ada8f * fix: preserve existing script resolution Assisted-by: GitHub Copilot (model: GPT-5.6 Sol, autonomous) Co-authored-by: Copilot App <[email protected]> Copilot-Session: 33a4ecee-211d-4be0-9842-7a433c5ada8f * refactor: isolate artifact provenance Keep lookup identifiers and provenance projection inside the artifact catalog while restoring existing resolver, extension, hook, asset, registrar, and integration behavior. Assisted-by: GitHub Copilot (model: GPT-5.6 Sol, autonomous) Co-authored-by: Copilot App <[email protected]> Copilot-Session: 33a4ecee-211d-4be0-9842-7a433c5ada8f * refactor: split artifact catalog modules Separate artifact models, catalog inventory, and resolver stack projection while preserving the existing package API and command behavior. Assisted-by: GitHub Copilot (model: GPT-5.6 Sol, autonomous) Co-authored-by: Copilot App <[email protected]> Copilot-Session: 33a4ecee-211d-4be0-9842-7a433c5ada8f * style: normalize artifact resolution EOF Assisted-by: GitHub Copilot (model: GPT-5.6 Sol, autonomous) Co-authored-by: Copilot App <[email protected]> Copilot-Session: 33a4ecee-211d-4be0-9842-7a433c5ada8f * fix: expose both override artifact kinds Remove filename-based kind guessing for root project overrides and let the existing resolver validate both command and template candidates. Assisted-by: GitHub Copilot (model: GPT-5.6 Sol, autonomous) Co-authored-by: Copilot App <[email protected]> Copilot-Session: 33a4ecee-211d-4be0-9842-7a433c5ada8f * docs: use Spec Kit product spelling Assisted-by: GitHub Copilot (model: GPT-5.6 Sol, autonomous) Co-authored-by: Copilot App <[email protected]> Copilot-Session: 33a4ecee-211d-4be0-9842-7a433c5ada8f * fix: confine artifact script references Reject anchored, traversing, and symlink-escaping script references before artifact discovery reads files outside the selected script root. Assisted-by: GitHub Copilot (model: GPT-5.6 Sol, autonomous) Co-authored-by: Copilot App <[email protected]> Copilot-Session: 33a4ecee-211d-4be0-9842-7a433c5ada8f * test: cover composing stack visibility Assisted-by: GitHub Copilot (model: GPT-5.6 Sol, autonomous) Co-authored-by: Copilot App <[email protected]> Copilot-Session: 33a4ecee-211d-4be0-9842-7a433c5ada8f --------- Co-authored-by: Copilot App <[email protected]> Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com> Co-authored-by: copilot-swe-agent[bot] <[email protected]> Co-authored-by: nicolehaugen <[email protected]> Co-authored-by: Copilot Autofix powered by AI <[email protected]> Co-authored-by: nicolehaugen <[email protected]> Copilot-Session: 4a40fb96-1bbe-4fb2-99d8-411170046cb0 Copilot-Session: 33a4ecee-211d-4be0-9842-7a433c5ada8f * [extension] Update MAQA — Multi-Agent & Quality Assurance extension to v0.3.1 (#4544) * Update MAQA extension to v0.3.1 Update maqa extension submitted by @GenieRobot: - extensions/catalog.community.json (version, download_url, metadata) - docs/community/extensions.md community extensions table Closes #4452 Assisted-by: GitHub Copilot (model: gpt-5.2-codex, autonomous) Co-authored-by: Copilot <[email protected]> * Decrease command count from 5 to 4 Co-authored-by: Copilot Autofix powered by AI <[email protected]> --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Copilot <[email protected]> Co-authored-by: Ken Schlobohm <[email protected]> Co-authored-by: Copilot Autofix powered by AI <[email protected]> * Update DocGuard — CDD Enforcement extension to v0.34.9 (#4545) Update docguard extension submitted by @raccioly: - extensions/catalog.community.json (version, download_url, updated_at) - docs/community/extensions.md (existing row already current) Closes #4537 Assisted-by: GitHub Copilot (model: gpt-5.2-codex, autonomous) Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Copilot <[email protected]> --------- Signed-off-by: shaurya2k06 <[email protected]> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Manfred Riem <[email protected]> Co-authored-by: Copilot App <[email protected]> Co-authored-by: Ali jawwad <[email protected]> Co-authored-by: Claude Opus 5 (1M context) <[email protected]> Co-authored-by: Noor ul ain <[email protected]> Co-authored-by: WOLIKIMCHENG <[email protected]> Co-authored-by: root <[email protected]> Co-authored-by: Copilot Autofix powered by AI <[email protected]> Co-authored-by: Nguyen Thanh Dat <[email protected]> Co-authored-by: Shaurya Srivastava <[email protected]> Co-authored-by: RKS <[email protected]> Co-authored-by: nicolehaugen <[email protected]> Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com> Co-authored-by: copilot-swe-agent[bot] <[email protected]> Co-authored-by: nicolehaugen <[email protected]> Co-authored-by: nicolehaugen <[email protected]> Copilot-Session: 4a40fb96-1bbe-4fb2-99d8-411170046cb0 Copilot-Session: 33a4ecee-211d-4be0-9842-7a433c5ada8f
Implements #4274, which was the write-up you asked for at the end of #4230.
The problem
_unresolvable_termanswers one question — does every operand in this condition resolve to something? — by walking the expression itself: filters, thenor/and/not, then comparisons, then list literals, down to the leaves.That walk is a second implementation of the parsing in
_evaluate_simple_expression, kept in step with it by hand. Two helpers exist purely to restate rules the evaluator already has, and both say so:_looks_numeric— "Mirror the evaluator's numeric literal test exactly." A barefloat()accepts1e3; the evaluator does not._is_literal— "Mirror the evaluator's literal tests exactly."startswith/endswithaccepts'a' 'b'; the evaluator does not.When the two drift, nothing breaks loudly. The gate keeps answering, just wrongly, and the wrong answer is a paste-ready correction that silently inverts a condition. Seven of the nine findings in #4230 were that same defect in different clothes — the gate disagreeing with the evaluator about where the operands are. Each round fixed one shape; nothing stopped a tenth.
The change
_evaluate_simple_expressionhas exactly one place where a substring stops being grammar and becomes a name to resolve — its final line,_resolve_dot_path. Literals return before it; operands, filter arguments and list elements all arrive there by construction. So let the evaluator report what it reaches:_collect_leavesruns the probe_evaluator_rejectsalready uses, with the sink armed._unresolvable_leafkeeps only the namespace rules — root membership, path-segment shape, and theitemindex narrowing from the last round of #4230. The gate now contains no grammar at all.A
ContextVarrather than a module global, so concurrent probes cannot append into each other's list; it isNoneoutside a probe, so a normal evaluation costs one.get().Two properties this rests on
Both are asserted rather than assumed, because if either changed the gate would go quietly blind rather than fail:
or/andare not short-circuited._evaluate_simple_expressionevaluates both sides and only then combines them, so a leaf is recorded whatever the other side is worth.test_both_sides_of_a_boolean_are_reportedpins it.A probe run can raise on its own placeholder values, which is what
_evaluator_rejectssorts out. The leaves seen before that point are real — the evaluator reached them — so they are kept rather than discarded. Discarding them would losebogusininputs.tags | join(bogus), which is the filter-argument case an earlier round of #4230 had to add by hand.test_leaves_seen_before_a_probe_error_are_keptpins it.Verification
expressions.py: 109 lines removed, 84 added.All 336 existing tests pass unchanged, including the 20 cases of
test_operands_must_be_literals_or_known_pathsthat took eight rounds to get right. That is the main evidence: the new gate agrees with the old one on every shape review found, without knowing about any of them.One test changed rather than passed:
test_literal_test_mirrors_the_evaluatortested the mirror, and the mirror is gone. It becomestest_literal_handling_comes_from_the_evaluatorand asserts the same knowledge —1e3is not a number to the evaluator,'a' 'b'is not one literal — through_unresolvable_term. That is the property that actually mattered; the old test could pass while the two had drifted.Five tests added for the mechanism itself. Checked by breaking it:
_unresolvable_termsuiteitem[0]narrowingtest_an_indexed_item_root_keeps_the_correctionuvx [email protected] check src tests— clean. Wider run (tests/unit,test_workflows.py,test_extensions.py): 1988 passed, and the set of failing test names is identical before and after — 24 symlink tests that cannot run unprivileged on Windows.Update: the segment grammar was still duplicated, and #4416 / #4417 (2026-09-09)
c6c3ef7finishes what the first commit started. The gate had stopped restating the operator grammar, but it still restated the shape of a path segment:_PATH_SEGMENTand an inlinere.fullmatchboth described the index form that_resolve_dot_pathmatches with its own regex — three copies of one rule, kept in step by hand. That form is now named once as_INDEXED_SEGMENTbeside_resolve_dot_path, and the gate asks it. The regex is copied verbatim, so behaviour is unchanged; what changes is that widening indexing now reaches the gate for free.That was not cosmetic. I measured it before writing it:
c6c3ef7task_list[-1].fileNone'task_list[-1]' is not a valid path segment(inputs.a or inputs.b) and inputs.cFalseAnswering your question directly. I took the evaluator-side hunks only of #4416 and #4417 — no edit to
_unresolvable_term,_PATH_SEGMENTor anything else in the gate — and applied them on top of this branch:task_list[-1].file'b.md'task_list[0].file'a.md'(inputs.a or inputs.b) and inputs.cTrue(inputs.n)5inputs.a or inputs.b and inputs.cTrueSo yes: once this lands, both of @NgoQuocViet2001's PRs reduce to their evaluator hunks. The gate-side changes they each carry — #4416's widened
_PATH_SEGMENTandindexed_root, #4417's "mirror the evaluator's group unwrapping" block in_unresolvable_term— become unnecessary, because the gate no longer has an opinion of its own to keep in sync. Their diffs shrink and stop touching the region this PR rewrites, which should make the rebase mechanical rather than a merge argument.One correction to my own framing, since it matters for sequencing: this refactor does not fix their bugs.
task_list[-1]still resolves toNoneon this branch and(a or b) and cstill reads false — those are evaluator defects and their PRs are what fix them. What this branch changes is that the gate now agrees with the evaluator instead of holding a second opinion, so their one-line evaluator fixes are sufficient on their own.dd4b723adds two regression tests for exactly that property, because it is easy to claim and easy to lose — a fresh copy of the grammar in the gate would keep every other test green. One patches_INDEXED_SEGMENTand asserts the gate follows; the other makes the evaluator stop treating a group as a leaf and asserts the gate stops checking it. I verified both by reintroducing the drift (giving the gate its own segment regex again), which fails the first with the real message rather than an import error. I deliberately did not mirror their test bodies: assertingtask_list[-1] == 'b.md'here would be a test for a fix this PR does not contain, and would pass or fail based on whether their PR is present.tests/unit/test_condition_expression_block.py: 343 passed.tests/test_workflows.py+ that file: the set of failing test names is byte-identical before and after these two commits — 20 symlink tests that cannot run unprivileged on Windows.AI disclosure
Per CONTRIBUTING: this pull request was developed with AI assistance — I used Claude Code as a coding agent for the code, the tests and the measurements above, reviewing and directing it throughout, and the experiment applying #4416/#4417's hunks was run and checked by me. This comment and the PR body were also written with that assistance. Apologies for the omission on the original submission; it was an oversight, not an attempt to hide it.