Skip to content

fix(testsuite): honor WaitForCancellation when canceling activities - #1545

Open
Thanatos9404 wants to merge 2 commits into
cadence-workflow:masterfrom
Thanatos9404:fix/testsuite-activity-wait-for-cancellation
Open

Thanatos9404 wants to merge 2 commits into
cadence-workflow:masterfrom
Thanatos9404:fix/testsuite-activity-wait-for-cancellation

Conversation

@Thanatos9404

Copy link
Copy Markdown

What changed?
With WaitForCancellation set, the test environment now waits for a canceled activity to complete, fail or accept the cancellation, and its heartbeats report the cancellation. As on the server, an activity scheduled in the same decision task or waiting for a retry is still canceled at once, and it is not retried once cancellation is requested. Each workflow now keeps its own mocked service, so this also works in child workflows.

Why?
Fixes #1368. The test environment ignored the option and canceled every activity immediately, unlike a real worker.

How did you test it?
Added unit tests next to Test_ChildWorkflowCancel; the cases that wait for the activity fail without this change. Ran the internal tests, gofmt, go vet and revive.

Potential risks
Tests that set WaitForCancellation and relied on the immediate cancel now wait for the activity, as with a real worker. Heartbeats are throttled to 80% of HeartbeatTimeout, so such tests may need a shorter HeartbeatTimeout.

The workflow test environment resolved every activity cancellation at
once, so the WaitForCancellation activity option had no effect and a
running activity never saw that it was canceled.

When the option is set, the test environment now keeps waiting for the
activity, reports the cancellation on its heartbeats and delivers
whatever the activity returns, like a real worker. As on the server, an
activity scheduled in the same decision task or waiting for a retry is
still canceled at once, and an activity is not retried after its
cancellation was requested. Each workflow now keeps its own mocked
service, so heartbeats from activities in child workflows are matched
too.

Fixes cadence-workflow#1368

Signed-off-by: Yashvardhan Thanvi <[email protected]>
Comment thread internal/internal_workflow_testsuite.go Resolved
…t activities

Tests that mock an activity which blocks without heartbeating are now
waited for after a cancel, so say how such a mock can return once it is
canceled, and that heartbeats are throttled to 80% of HeartbeatTimeout.

Signed-off-by: Yashvardhan Thanvi <[email protected]>
@gitar-bot

gitar-bot Bot commented Oct 20, 2026

Copy link
Copy Markdown
Code Review ✅ Approved 1 closed / 1 findings

🟡 Medium risk · Changing testsuite cancellation handling to wait for activity completion may cause existing mocks without cancellation handling to hang until test timeout.

Honors WaitForCancellation in the test environment so canceled activities wait for completion, failure, or cancellation acknowledgment, and their heartbeats report the cancellation. Addresses the hang risk for the cadence server's BatchWorkflowV2 tune-signal test on SDK bump. No open issues found.

✅ 1 closed
✅ Cross-Repo: cadence server's BatchWorkflowV2 tune-signal test will hang on SDK bump

📄 internal/internal_workflow_testsuite.go:811-819 📄 CHANGELOG.md:15
Fix here and in cadence-workflow/cadence

Breaks in: cadence-workflow/cadence - service/worker/batcher/workflow_v2_test.go - TestBatchWorkflowV2_TuneSignal

In cadence-workflow/cadence, BatchWorkflowV2 sets WaitForCancellation: true. When it gets a tune signal it calls cancel() and then future.Get(). TestBatchWorkflowV2_TuneSignal in service/worker/batcher/workflow_v2_test.go mocks the first activity so it blocks on firstActivityDone. That channel is closed only in t.Cleanup, and the mock never heartbeats or returns.

The signal arrives in a later decision task than the one that scheduled the activity, and the activity is not waiting for a retry. So the new branch at internal_workflow_testsuite.go:811 sets cancelRequested and returns without resolving the future. Before this PR the future resolved at once with a CanceledError. Now the activity has to see the cancellation on a heartbeat or return on its own, and this mock does neither. Once the server bumps the SDK, the test hangs until the testsuite timeout panic fires.

The PR means to make this change (it matches a real worker, and "Potential risks" mentions it), so the consumer test is what has to change. Here, the CHANGELOG should flag it as a behavior change for tests, so whoever bumps the SDK knows to look at blocking mocked activities.

Here: Keep the new behavior. Expand the CHANGELOG entry to warn that tests using WaitForCancellation with mocked activities that block and never heartbeat will now wait for those activities, and can hang until they return.

There: In cadence-workflow/cadence service/worker/batcher/workflow_v2_test.go TestBatchWorkflowV2_TuneSignal, make the first mocked activity return once it is canceled: have it heartbeat and select on ctx.Done(), or unblock it from env.SetOnActivityCanceledListener. Either way the future resolves after cancel().

In cadence-workflow/cadence - service/worker/batcher/workflow_v2_test.go:

env.SetOnActivityCanceledListener(func(*activity.Info) {
	once.Do(func() { close(firstActivityDone) })
})
// and in the first mocked activity, return a canceled error after unblocking:
// <-firstActivityDone
// return HeartBeatDetails{}, context.Canceled
Verification Objectives 3/3 Repos 2 inspected · PRs 1 inspected
Functional validation 3 covered / 3 objectives

#1368 3 covered / 3 objectives

✅ WorkflowTestSuite honors the WaitForCancellation activity option when the workflow is cancelled
✅ On workflow cancellation, the workflow waits for an in-progress activity with WaitForCancellation set to finish before completing
✅ Unit test where a mocked activity with WaitForCancellation is cancelled via env.CancelWorkflow() succeeds with the activity result and the activity completing

Rules No rules evaluated

Broader impact Repos 2 inspected PRs 1 inspected
Cross-repo 2 repositories inspected 0 open impacts

cadence-workflow/cadence Inspected
cadence-workflow/cadence-samples Inspected

Cross-PR 1 PR inspected 0 open impacts

PR #1536: feat: Support JitterCoefficient in RetryPolicy Inspected

For more verification details, comment gitar display:verbose.

Options

Auto-apply is off → Gitar will not commit updates to this branch.
Display: compact → Showing compact verification results.

Comment with these commands to change the behavior for this request:

Auto-apply Compact
gitar auto-apply:on         
gitar display:verbose         

Was this helpful? React with 👍 / 👎 | Gitar

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

WaitForCancellation activityOption does not work in WorkflowTestSuite

1 participant