Skip to content

test: cover chained invoke replay and child resume - #1068

Merged
zhongkechen merged 17 commits into
mainfrom
codex/otel-case11-replay
Oct 28, 2026
Merged

zhongkechen merged 17 commits into
mainfrom
codex/otel-case11-replay

Conversation

@zhongkechen

@zhongkechen zhongkechen commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Issue Link, if available

Companion requirements PR aws/aws-durable-execution-conformance-tests#131 is merged. Java counterpart: aws/aws-durable-execution-sdk-java#824. Python counterpart: aws/aws-durable-execution-sdk-python#796. The SDK PRs can merge independently.

Description

Case 11 returned immediately after its child invoke, so the shared requirement could not observe that completed invoke on a later caller replay. The suspended-child scenario previously belonged to the disabled long-running workflow, leaving the ordinary invocation and execution suites without that coverage.

Case 11 now waits at otel-invoke-replay-barrier after the child returns, then returns the saved result after the callback resumes it. Its caller follows PENDING → PENDING → SUCCEEDED, with one child execution and one terminal invoke export. Case 27 adds a child that performs a one-second durable wait and returns its original payload after resuming. Both views check two child invocations, stable wait identity, continuation links, and the full six-event caller history required by case 27.

Both the shared orchestrator and requirement catalog default to the conformance repository's main branch, which includes the matching 27-case catalog and omits the disabled long-running jobs. Pull requests, pushes, and manual runs use main; manual runs can still override conformance_test_ref with a commit SHA or branch. The package documentation and workflow wiring guard reflect these defaults.

Design and simpler alternatives

  • Keeping case 11 unchanged cannot exercise its completed invoke again. The callback barrier adds the required replay boundary using the existing shared driver; its submitter is a no-op because the driver completes the callback after the invocation ends.
  • Reusing otel_23_long_chained_invoke.ts unchanged cannot satisfy case 27. That handler requires long-chained-invoke and delay_seconds, and emits otel-long-invoke / otel-long-invoke-target-wait. Case 27 supplies chained-invoke-suspend without a delay input and requires otel-suspended-invoke / otel-suspended-target-wait. A shared factory would need scenario, operation-name, and delay parameters to share a small straight-line workflow. Keeping the explicit handlers makes the required durable identities visible without adding configuration or branches.
  • Combining the case-11 and case-27 integration helpers would need separate control paths for a callback barrier with three caller invocations versus child-wait continuation with two caller invocations. Each existing helper already serves both tracing views. Keep those distinct workflows and put missing-configuration coverage in the existing case-27 helper rather than adding another harness.
  • Keeping only caller-history event counts cannot detect an ordering regression. Replace the two filtered count assertions with the exact case-27 EventType sequence; it covers the same counts and enforces the required order without another helper or control path.
  • Removing the target guard would lose the explicit missing-configuration diagnostic. Keep it so an absent or empty OTEL_INVOKE_TARGET_FUNCTION_NAME fails the execution before any child invoke starts. Tests run the actual handler for both invalid values in both views and check the error and absence of ChainedInvokeStarted events.

The selected design retains the scenario handlers, the explicit callback boundary, and the child-completion gate. The child is released only after the caller's first pauseExecution() completes, establishing InvocationCompleted before ChainedInvokeSucceeded. Case 27 asserts that order directly. Its full ordered history assertion replaces the two filtered history-count assertions and preserves their guarantees. SDK behavior and durable operation identities are unchanged.

Existing test expectations changed

The expectation changes are in packages/aws-durable-execution-sdk-js-conformance-tests-otel/__tests__/templates.test.ts:

  • template handler references > finds a reference in every template function: 61 → 65 because case 27 adds a caller and target in each of two tracing views, four functions total. The new total is 60 ordinary-suite functions plus five long-running template functions.
  • requirement coverage > covers the invocation and execution suites: 26 → 27 in each range because both catalogs now include case 27. These assertions continue to enforce exact wiring and catalog coverage; they do not relax an SDK contract.
  • otel-conformance-tests workflow > pins the shared orchestrator to a full commit SHA becomes defaults the shared orchestrator and requirements to main: it now checks the orchestrator reference, manual input default, and automatic-run fallback, including preservation of the manual override. This implements the requested default change without changing SDK behavior.

In packages/aws-durable-execution-sdk-js-otel/src/__tests__/helpers/chained-invoke-suspension.ts, keeps child wait continuation telemetry through caller completion replaces separate counts of one ChainedInvokeStarted and one ChainedInvokeSucceeded with the exact sequence ExecutionStarted, ChainedInvokeStarted, InvocationCompleted, ChainedInvokeSucceeded, InvocationCompleted, ExecutionSucceeded. This strengthens both tracing views to enforce the caller history already required by otel-execution-27.yaml and otel-invocation-27.yaml; it preserves the existing count guarantees.

Demo/Screenshots

No UI change. The local tests exercise the deployed handler sources through LocalDurableTestRunner and inspect replay phases, returned payloads, operation identity, terminal exports, and configuration failures.

Checklist

  • I have filled out every section of the PR template
  • I have tested the change locally; cloud conformance runs separately in CI

Testing

Unit Tests

All 43 conformance wiring checks pass. TypeScript checks pass for the conformance handlers and OTel sources, including the test helper; Biome passes for the changed TypeScript file. Core, testing, OTel, and deployable conformance-handler builds pass.

For the workflow-default change, the updated guard first failed against the existing pin (Expected: "main", Received: "6120aed9e3f8770a946e6e8bb3260a22f6f246f9"), with the other 42 checks passing. After updating all three references, all 43 checks and the handler typecheck pass. Core and OTel builds, Biome on the changed test, and git diff --check pass. Both workflow YAML files parse, and every supplied input and secret matches the shared orchestrator currently on main. Cloud conformance for this update remains unverified until CI completes.

Integration Tests

The two case-11 tests check one target invocation, the full 13-event history, and no second terminal invoke export after replay. The target gate explicitly orders the first InvocationCompleted before ChainedInvokeSucceeded, independently of child speed and telemetry flush timing.

The case-27 helper has three cases per view: successful suspension/resume, absent target configuration, and empty target configuration. The success case checks the full six-event caller history. Environment replacement is restored after each negative case so the successful workflow retains its configured target.

History-order regression validation used a temporary scheduling probe: remove the target gate and delay the caller's first PENDING return until the child invoke completes. Both success tests passed their original assertions despite ChainedInvokeSucceeded preceding the first InvocationCompleted. Adding the ordered assertion made both success tests fail, with the four configuration tests still passing. Restoring the original gate made all six case-27 tests pass. The probe is not included in the commit; it establishes the local assertion gap, not service timing.

All eight focused integration cases pass (two case-11 cases and six case-27 cases). Before this follow-up, removing the target guard while preserving TypeScript validity still passed both case-27 tests and all 43 wiring checks. With the new coverage, that same deletion makes all four negative cases fail. Restoring the guard returns all eight integration cases to green. The mutation is not included in the commit.

The existing cloud run at 98216cdb, https://github.com/aws/aws-durable-execution-sdk-js/actions/runs/40377005155, passed all 108 checks across invocation/execution views on ADOT/X-Ray and Community/S3, including cases 11 and 27 in every combination. This is evidence for the service trace carrier and ancestry assertions; local tests alone do not establish them. Cloud CI will rerun for the follow-up commit.

Examples

Updated case 11 and added case 27 handlers, SAM mappings, and documentation. Configuration failures and caller-history ordering are covered through the existing local integration helpers.

@github-actions

This comment has been minimized.

@zhongkechen
zhongkechen temporarily deployed to ai-pr-review-runtime October 23, 2026 19:39 — with GitHub Actions Inactive
@github-actions

This comment has been minimized.

@zhongkechen
zhongkechen temporarily deployed to ai-pr-review-runtime October 23, 2026 22:53 — with GitHub Actions Inactive
@github-actions

This comment has been minimized.

@zhongkechen zhongkechen changed the title test: cover case 11 completed invoke replay test: cover chained invoke replay and child resume Oct 23, 2026
@zhongkechen
zhongkechen temporarily deployed to ai-pr-review-runtime October 23, 2026 23:06 — with GitHub Actions Inactive
@github-actions

This comment has been minimized.

@zhongkechen
zhongkechen temporarily deployed to ai-pr-review-runtime October 23, 2026 23:59 — with GitHub Actions Inactive
@github-actions

This comment has been minimized.

@zhongkechen
zhongkechen temporarily deployed to ai-pr-review-runtime October 24, 2026 00:17 — with GitHub Actions Inactive
@github-actions

This comment has been minimized.

@zhongkechen
zhongkechen temporarily deployed to ai-pr-review-runtime October 24, 2026 20:27 — with GitHub Actions Inactive
@github-actions

This comment has been minimized.

@zhongkechen
zhongkechen temporarily deployed to ai-pr-review-runtime October 24, 2026 20:41 — with GitHub Actions Inactive
@github-actions

This comment has been minimized.

Keep main's shared OTel orchestrator revision and align both requirement catalog defaults with it. The newer revision contains the replay and child-resume requirements plus subsequent assertion fixes.
@zhongkechen
zhongkechen temporarily deployed to ai-pr-review-runtime October 26, 2026 18:22 — with GitHub Actions Inactive
@github-actions

This comment has been minimized.

@zhongkechen
zhongkechen temporarily deployed to ai-pr-review-runtime October 27, 2026 19:24 — with GitHub Actions Inactive
@github-actions

This comment has been minimized.

@zhongkechen
zhongkechen temporarily deployed to ai-pr-review-runtime October 27, 2026 21:14 — with GitHub Actions Inactive
@github-actions

This comment has been minimized.

Replace filtered chained-invoke event counts with the complete required history sequence in both tracing views. The stronger assertion detects a child completion recorded before the caller invocation completes.
@zhongkechen
zhongkechen temporarily deployed to ai-pr-review-runtime October 27, 2026 23:56 — with GitHub Actions Inactive
@github-actions

Copy link
Copy Markdown

Codex AI review

No actionable findings; the prior findings are addressed. Review was limited to read-only inspection: tests were not executed, and the external conformance workflow’s current main revision was not independently verified.

Reviewed commit 0c2202717097681a8b1e08ace7393c468fac363d. Workflow run

@zhongkechen
zhongkechen merged commit b92df51 into main Oct 28, 2026
55 checks passed
@zhongkechen
zhongkechen deleted the codex/otel-case11-replay branch October 28, 2026 18:00

This branch was previously deployed

1 inactive deployment
ai-pr-review-runtime — 0c220271 Deployed Oct 27, 2026 by zhongkechen via ai-pr-review / Codex review / Generate Codex review #2283
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants