You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
fix(ci): flaky unit tests — prompt-during-run and hook timeout #121
Two unit tests fail intermittently in CI (GitHub Actions Linux runner), causing unit (linux) job to report failure. These are pre-existing flakes unrelated to any recent PR changes.
Failing Tests
1. prompt submitted during an active run is included in the next LLM input
Location: Likely in test/session/ or test/prompt/
Failure mode: Timing-dependent — test expects a prompt submitted during an active LLM run to be queued and included in the next input. Race condition between prompt submission and run completion.
Frequency: Fails consistently in CI, may pass locally due to faster I/O
Root cause hypothesis: Non-deterministic timing in async prompt submission pipeline. Test lacks explicit synchronization barrier.
Suggested fix: Add explicit waitFor/flush before assertion, or mock the timing to remove non-determinism.
2. hook.execute > runHook > timeout returns pass
Location: Likely in test/hook/
Failure mode: Takes 5000.98ms — just barely exceeds the 5000ms timeout threshold
Frequency: Fails in CI where runners are slower, passes locally
Root cause hypothesis: The timeout boundary is too tight for CI environments. The test expects the hook to resolve within exactly 5000ms but CI jitter pushes it over.
Suggested fix: Either increase the timeout threshold by a small margin (e.g., 5100ms) or use a more generous timeout in CI environments via process.env.CI detection.
Impact
These failures are non-blocking for PRs that don't touch session/hook code
unit results (linux) secondary job also reports failure as a consequence
Developers may waste time investigating false positives
Evidence
CI logs from PR #117-#120 (all show identical failures):
(fail) prompt submitted during an active run is included in the next LLM input [283.00ms]
(fail) hook.execute > runHook > timeout returns pass [5000.98ms]
Suggested Actions
Investigate and fix the timing issue in prompt-during-run test
Adjust hook timeout test to account for CI jitter
Consider adding @flaky tag or retry annotation as interim measure
Check if these are known upstream issues (anomalyco/opencode)
Prompt-during-run test — PRE-EXISTING RACE CONDITION
Root cause: NOT a timeout issue. Test fails at ~300-400ms (well within 10s limit)
The test expects prompt.prompt() to properly queue while a held LLM call is active
Effect.sleep(50) barriers and increased polling intervals did not resolve it
This is a genuine race condition in the upstream session prompt pipeline
The test's llm.wait(1) + immediate second prompt submission creates a window where the session state may not be consistent
CI shows fail_on_failure: false — this is informational, not a PR blocker
Recommendation
The hook timeout fix is complete and should be merged
The prompt-during-run flake needs a deeper fix in packages/opencode/src/session/prompt.ts (synchronization barrier between concurrent prompt submissions)
This is upstream test code — consider filing upstream or accepting as known flake
Root cause identified: NOT a flaky test. CI runner downgrade from blacksmith-4vcpu to ubuntu-latest (2 shared vCPU) causes timing-sensitive tests to fail. Superseded by #129 which tracks the proper fix.
Hook timeout fix included in PR #127 (5s → 15s). Prompt-during-run race needs runner restoration (Option A in #129) or deeper synchronization fix.
Activity
fix(ci): resolve flaky tests and check-duplicates crash (#121, #122)
fix(ci): increase hook timeout to 15s, widen prompt poll interval
fix(ci): add Effect.sleep barriers for prompt-during-run race condition
terisuke commentedon Apr 6, 2026
PR #127 investigation results
Hook timeout test — FIXED
sleep 10+ 200ms kill timeout overhead exceeded test harness limitPrompt-during-run test — PRE-EXISTING RACE CONDITION
prompt.prompt()to properly queue while a held LLM call is activeEffect.sleep(50)barriers and increased polling intervals did not resolve itllm.wait(1)+ immediate second prompt submission creates a window where the session state may not be consistentfail_on_failure: false— this is informational, not a PR blockerRecommendation
packages/opencode/src/session/prompt.ts(synchronization barrier between concurrent prompt submissions)terisuke commentedon Apr 6, 2026
Root cause identified: NOT a flaky test. CI runner downgrade from
blacksmith-4vcputoubuntu-latest(2 shared vCPU) causes timing-sensitive tests to fail. Superseded by #129 which tracks the proper fix.Hook timeout fix included in PR #127 (5s → 15s). Prompt-during-run race needs runner restoration (Option A in #129) or deeper synchronization fix.