Skip to content

Conversation

@sisyphus-dev-ai
Copy link
Collaborator

@sisyphus-dev-ai sisyphus-dev-ai commented Jan 5, 2026

Summary

Fixes CI test failure in src/cli/doctor/runner.test.ts where the duration measurement test was flaky.

Problem

The test expected >= 10ms duration after sleeping for 10ms, but occasionally received 9ms due to:

  • Timer precision limitations
  • Measurement overhead
  • Non-deterministic setTimeout behavior

Solution

Increased the sleep duration from 10ms to 50ms while keeping the >= 10ms assertion. This provides sufficient buffer to prevent flakiness while still validating that duration measurement works correctly.

Testing

  • ✅ Specific test now passes consistently: bun test src/cli/doctor/runner.test.ts
  • ✅ Full test suite passes: 607 pass, 0 fail across 53 files

Closes #507


Summary by cubic

Increased sleep in the runner duration test from 10ms to 50ms to eliminate CI flakiness, preventing occasional under-10ms readings from timer precision and overhead while still asserting >=10ms duration. Fixes #507.

Written for commit df9a927. Summary will update on new commits.

The test was flaky because it expected >= 10ms duration after sleeping
for 10ms, but due to timer precision and measurement overhead, it
sometimes received 9ms. Increasing the sleep to 50ms provides sufficient
buffer while still testing the duration measurement functionality.

Fixes #507
@sisyphus-dev-ai sisyphus-dev-ai mentioned this pull request Jan 5, 2026
@code-yeongyu code-yeongyu merged commit 0f890c1 into dev Jan 5, 2026
4 checks passed
@code-yeongyu code-yeongyu deleted the fix/flaky-timing-test branch January 5, 2026 11:20
@greptile-apps
Copy link

greptile-apps bot commented Jan 5, 2026

Greptile Summary

Fixed flaky test in src/cli/doctor/runner.test.ts by increasing sleep duration from 10ms to 50ms while keeping the assertion at >= 10ms.

Key Changes:

  • Changed setTimeout(r, 10) to setTimeout(r, 50) in the duration measurement test (line 33)
  • Provides 5x buffer to prevent timing-related test failures
  • Maintains the same validation logic for duration measurement

Technical Context:
The original 10ms sleep with a >= 10ms assertion was too tight due to timer precision limitations, measurement overhead, and non-deterministic setTimeout behavior. The fix follows TDD best practices from AGENTS.md by making a flaky test reliably pass without changing its purpose.

Test Results:

  • Specific test now passes consistently
  • Full test suite: 607 pass, 0 fail across 53 files

Confidence Score: 5/5

  • This PR is safe to merge with no risk
  • The fix correctly addresses a flaky test by increasing the sleep duration from 10ms to 50ms, providing a 5x buffer while maintaining the same >= 10ms assertion. This is a minimal, focused change that follows TDD best practices by making a failing test reliably pass without altering the test's purpose of validating duration measurement functionality
  • No files require special attention

Important Files Changed

Filename Overview
src/cli/doctor/runner.test.ts Increased sleep duration from 10ms to 50ms in duration measurement test to prevent timing-related flakiness while maintaining the same >= 10ms assertion

Sequence Diagram

sequenceDiagram
    participant Test as Test Suite
    participant runCheck as runCheck()
    participant perf as performance.now()
    participant check as check.check()
    participant timer as setTimeout

    Test->>runCheck: Execute test with CheckDefinition
    runCheck->>perf: start = performance.now()
    Note over perf: Record start time
    runCheck->>check: await check.check()
    check->>timer: setTimeout(resolve, 50ms)
    Note over timer: Sleep for 50ms<br/>(changed from 10ms)
    timer-->>check: resolve after 50ms
    check-->>runCheck: return { status: "pass", ... }
    runCheck->>perf: performance.now() - start
    Note over perf: Calculate elapsed time<br/>(~50ms measured)
    runCheck->>runCheck: Math.round(duration)
    Note over runCheck: Round to integer
    runCheck-->>Test: return result with duration
    Test->>Test: expect(duration).toBeGreaterThanOrEqual(10)
    Note over Test: Assert >= 10ms<br/>(passes with 50ms sleep)
Loading

Copy link

@cubic-dev-ai cubic-dev-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 1 file

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Fix CI

3 participants