Skip to content

test(eval): cover container interruption on cancellation#3649

Merged
Sayt-0 merged 1 commit into
mainfrom
test/3585-eval-cancellation-signal
Jul 15, 2026
Merged

test(eval): cover container interruption on cancellation#3649
Sayt-0 merged 1 commit into
mainfrom
test/3585-eval-cancellation-signal

Conversation

@Sayt-0

@Sayt-0 Sayt-0 commented Jul 15, 2026

Copy link
Copy Markdown
Member

Summary

Add daemon-free regression coverage for the eval container cancellation fix introduced in #3615.

The test replaces docker with a helper process, cancels an in-flight evaluation, and verifies that the Docker CLI receives SIGINT and exits promptly. This protects against reverting to exec.CommandContext's default SIGKILL behavior, which cannot be proxied to the container and can leave --rm eval containers running.

Issue expectations

Expectation Coverage
Cancelled evals signal the Docker CLI instead of killing it directly Verified through an interrupt marker written by the helper process
Cancellation returns promptly Verified before the 10-second WaitDelay backstop
No Docker daemon is required The test uses a temporary fake docker executable
The regression is detected Removing the custom cmd.Cancel causes the test to fail with signal: killed

Validation

  • task build
  • task test
  • task lint
  • go vet ./pkg/evaluation
  • focused regression test, 10 consecutive runs

The full test suite was run without the sandbox-injected HTTP proxy because that proxy intercepts private-address SSRF fixtures.

Closes #3585

@Sayt-0
Sayt-0 requested a review from a team as a code owner July 15, 2026 09:44
@Sayt-0
Sayt-0 enabled auto-merge July 15, 2026 09:50
@aheritier aheritier added area/testing Test infrastructure, CI/CD, test runners, evaluation kind/test Test-only changes labels Jul 15, 2026
@Sayt-0
Sayt-0 force-pushed the test/3585-eval-cancellation-signal branch from 3dd3212 to 52b7640 Compare July 15, 2026 11:29
@Sayt-0
Sayt-0 merged commit 5c71b64 into main Jul 15, 2026
4 checks passed
@Sayt-0
Sayt-0 deleted the test/3585-eval-cancellation-signal branch July 15, 2026 11:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/testing Test infrastructure, CI/CD, test runners, evaluation kind/test Test-only changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

eval: cancelled/interrupted runs orphan --rm eval containers (never reaped)

3 participants