Skip to content

fix: windows-only e2e in the GH workflow - #5

Merged
tsaqib merged 1 commit into
mainfrom
win-fixes-2
May 26, 2026
Merged

tsaqib merged 1 commit into
mainfrom
win-fixes-2

Conversation

@tsaqib

@tsaqib tsaqib commented May 26, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Checklist

  • All commits are signed off (DCO) and cryptographically signed. If not, run git rebase --signoff --exec 'git commit --amend --no-edit -S' $(git merge-base HEAD origin/main) && git push --force-with-lease
  • Tests added or updated
  • Docs updated if behavior or interfaces change
  • I have read CONTRIBUTING.md

@tsaqib
tsaqib merged commit b258423 into main May 26, 2026
2 checks passed
@tsaqib
tsaqib deleted the win-fixes-2 branch May 26, 2026 01:30
tsaqib added a commit that referenced this pull request May 29, 2026
…ename copilot fixture (#47)

* fix(skills-evals): rename copilot_workspace fixture to workspace

The 3 copilot scenario tests (test_copilot_todo,
test_copilot_work_item_extends, test_copilot_reminders_supersedes_todo)
referenced a `copilot_workspace` pytest fixture that does not exist in
conftest.py. The conftest only provides `workspace`. Every recent
copilot eval errored at fixture-setup time with:

  fixture 'copilot_workspace' not found
  available fixtures: ..., workspace

The copilot smoke test passes because it builds its own workspace
directly without the fixture. The three scenario tests must use the
same conftest-provided `workspace` fixture every other backend's tests
use.

Verified locally with --collect-only against the 3 files: 3 tests
collected, no errors.

Pre-existing bug, surfaces only now because earlier failures
(STAX_AGENT_KEY missing in #44, syntax/import errors in #43) masked
the collection-time error.

Signed-off-by: Tanzim Saqib <[email protected]>

* ci(skills-eval): pin Claude Code to 2.1.150

2.1.156 (current @latest) emits a `system`-role message at
messages[1] of the wire payload. DeepSeek's Anthropic-compat
endpoint rejects it with `400 Failed to deserialize the JSON
body into the target type: messages[1].role: unknown variant
'system', expected 'user' or 'assistant'`.

Reproduces on both claude workflows against the post-#44 main:

  https://github.com/stackific/stax/actions/runs/26618513746
  https://github.com/stackific/stax/actions/runs/26618519248

2.1.150 is the most recent release I can verify doesn't carry
the regression (chosen by working back from 2.1.156; the gap
between 2.1.150 and 2.1.156 is six patch releases). Pinning
rather than @latest matches the policy the install-step comment
has always claimed.

Bump intentionally after verifying the smoke test still passes.

Signed-off-by: Tanzim Saqib <[email protected]>

* fix(copilot-driver): inline SKILL.md to actually invoke /scope and /ship

Copilot CLI's `-p <prompt>` does NOT resolve leading slash commands the
way Claude Code does. Running `copilot -p "/scope build a single HTML
todo list app"` passes the prompt verbatim and the model treats it as
a literal request to build the app — skipping the scope planning step,
never creating any work-item plan files. Direct evidence from run
26618829987 (copilot user-scope, fix/copilot-workspace-fixture):

  [driver] initial_prompt: /scope build a single HTML and localStorage-
    based todo list app
  [driver] spawn: copilot -p /scope build a single HTML ...
  [driver] stdout #5: Done. Created `todo.html` — a single-file ...

The fixture-rename commit (b254fa8) made the collection-time error go
away and exposed this deeper, pre-existing pathology that has kept all
three copilot scenario tests (todo / work_item_extends /
reminders_supersedes_todo) failing since they were introduced in #10.
None have ever passed in CI; the prior fixture bug masked it as a
collection error rather than a scenario error.

Fix mirrors the cline driver's `compose_skill_prompt` strategy (which
itself dates back to opencode's pre-`--command` era): detect a
`/scope` or `/ship` prefix in the prompt, read the corresponding
SKILL.md off disk, prepend its body, append the same CI directive
every other inlining driver uses, and hand the composed text to
`copilot -p`. The model then sees the actual skill instructions and
follows them instead of treating the slash as ornamental.

Discovery paths probed (project then user) mirror copilot's
documented skill lookup list (constants.go: `.agents/skills/`,
`.claude/skills/`, `.github/skills/` at project; `~/.copilot/skills/`,
`~/.agents/skills/` at user) plus `~/.claude/skills/` because copilot
tests install via `stax init --agents claude` (the transitional shape
in conftest.AGENT_INIT_VALUE_FOR_KEY) which lands skills under
`.claude/skills/`.

Non-slash prompts (the smoke test's `Respond with the single word:
ok`) pass through unchanged so the no-skill code path keeps working;
`run.initial_prompt` retains the original slash form so logs and
transcripts read the way the test author wrote them.

CI_DIRECTIVE is copied verbatim from cline_driver — kept verbatim, not
imported, because each driver's docstring asserts cross-driver parity
is by-value and a `from cline_driver import CI_DIRECTIVE` would couple
the modules in a way that hides accidental drift in code review.

Signed-off-by: Tanzim Saqib <[email protected]>

---------

Signed-off-by: Tanzim Saqib <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant