Skip to content

feat(sdk): add subType to step and createCallback - #939

Merged
ParidelPooya merged 1 commit into
mainfrom
feat/sdk-step-callback-subtype
Oct 1, 2026
Merged

ParidelPooya merged 1 commit into
mainfrom
feat/sdk-step-callback-subtype

Conversation

@ParidelPooya

@ParidelPooya ParidelPooya commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

SDK: subType on step and createCallback

  1. A function can compose several durable operations into one larger operation.
  2. ChildConfig.subType already lets such a function label its child context.
  3. The steps and callbacks inside that context had no such option. They always carried the subtype Step or Callback.
  4. So execution history and plugins could tell those parts apart only by parsing their names.

StepConfig.subType and CreateCallbackConfig.subType now set the subtype of a step or a callback. They have the same shape as ChildConfig.subType. The SDK records the subtype on the checkpoint and passes it to plugin events. Replay compares it with the checkpoint. So a replayed operation whose subtype changed terminates as non-deterministic. An unset or empty subtype selects the default, Step or Callback.

The core README gains an "Operation Subtypes" section with an example.

Why the retry check changes too

  1. When a step waits for a retry, the invocation suspends with RETRY_SCHEDULED.
  2. The check compared the operation's subtype with Step.
  3. A step with a custom subtype does not match.
  4. So such a step would have suspended with WAIT_SCHEDULED or CALLBACK_PENDING.

So the check now compares the operation type with STEP. Only steps reach the retry-waiting state. A waitForCondition is a step with the subtype WaitForCondition. So it now also suspends with RETRY_SCHEDULED while it waits for its next check.

Testing SDK: the local checkpoint server validates SubType like the service

  1. The service accepts a subtype of 1 to 32 characters from [a-zA-Z0-9-_].
  2. It rejects any other subtype with HTTP 400 ValidationException, at START and at SUCCEED, for STEP and CONTEXT updates.
  3. It rejects the whole request before it applies any update.
  4. The message lists each failed constraint and names the update by its 1-based position. For example: 1 validation error detected: Value 'bad subtype!' at 'updates.1.member.subType' failed to satisfy constraint: Member must satisfy regular expression pattern: [a-zA-Z0-9-_]+.
  5. The SDK classifies the 4xx as CheckpointUnrecoverableExecutionError. So the execution fails.

The local checkpoint server accepted any subtype. So a test with an invalid subtype passed locally and failed in AWS. Now the local server runs the same check first, and throws the same ValidationException with the same message.

A second change was needed for the SDK to see that error as AWS sends it:

  1. The checkpoint server runs in a worker thread, and sends API errors to the main thread with postMessage.
  2. postMessage clones an Error with its message and stack only. It drops the name of a subclass, and properties such as $metadata.
  3. The SDK classifies a checkpoint failure by its name and its HTTP status in $metadata.
  4. So the worker now sends errors as plain objects, and the main thread rebuilds them with name, $fault, and $metadata.

Existing errors from the local server, such as InvalidParameterValueException for an invalid state transition, carry no HTTP status. So their classification does not change.

Follow-up PR:

The SDK does not check the subtype, so the service rejects an invalid one only after the step has already run. With the default retry semantics, a step with subtype Order Charge ran its code once and then the execution failed. A check in the SDK when step() or createCallback() is called would prevent that.

@ParidelPooya ParidelPooya changed the title feat(sdk): add subType to step and createCallback, and validate subtypes feat(sdk): add subType to step and createCallback Oct 1, 2026
StepConfig.subType and CreateCallbackConfig.subType set the subtype of a
step or a callback. They have the same shape as ChildConfig.subType. The
SDK records the subtype on the checkpoint and passes it to plugin events.
Replay compares it with the checkpoint. An unset or empty subtype selects
the default, Step or Callback.

The retry check compared the subtype with "Step". A step with a custom
subtype does not match. So the check now compares the operation type
with STEP, and such a step that waits for a retry still suspends with
RETRY_SCHEDULED.

The testing SDK's local checkpoint server now validates SubType as the
service does: 1 to 32 characters from [a-zA-Z0-9-_]. An invalid subtype
rejects the whole request with HTTP 400 ValidationException and the
service's message. The worker thread used to drop the name and $metadata
of its errors. It now sends them as plain objects, so the SDK classifies
local checkpoint errors as it classifies the service's errors.
@ParidelPooya
ParidelPooya force-pushed the feat/sdk-step-callback-subtype branch from 9847fa3 to fb42320 Compare October 1, 2026 04:10

@yaythomas yaythomas left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is strictly speaking "for every update" not just "for STEP and CONTEXT updates"

could you pls create tracking issues to implement in the sibling SDKs, referencing this one?

@ParidelPooya
ParidelPooya merged commit bfbb07f into main Oct 1, 2026
36 checks passed
@ParidelPooya
ParidelPooya deleted the feat/sdk-step-callback-subtype branch October 1, 2026 17:12
ParidelPooya added a commit that referenced this pull request Oct 2, 2026
#939 is merged, and both packages sit at versions that npm already
serves: the core SDK at 2.5.0 and the testing SDK at 1.2.0. Both are
also already tagged (`sdk-2.5.0/test-1.2.0`). So neither package can be
released until its version moves.

This bumps:

- `@aws/durable-execution-sdk-js` from 2.5.0 to 2.6.0.
- `@aws/durable-execution-sdk-js-testing` from 1.2.0 to 1.3.0.

Both bumps are minor.

1. The core SDK adds `StepConfig.subType` and
`CreateCallbackConfig.subType` (#939). Both are new optional public API.
No published API was removed or renamed. Without the option, the SDK
records the same subtypes as before.
2. The testing SDK's local checkpoint server now validates `SubType` as
the service does, and rejects an invalid one with HTTP 400
`ValidationException` (#939). A test that sent an invalid subtype passed
before and fails now, as the same handler fails in AWS. This is new
behavior, not a removed API, so it is a minor bump.

The testing SDK's peer range on the core moves from `>=1.0.1 <=2.5.0` to
`>=1.0.1 <=2.6.0`. That change is required:
`src/__tests__/peer-compatibility.test.ts` asserts that the core SDK's
current version satisfies the declared range. Every earlier core bump
raised this bound the same way.

No other package needs a change. The OTel plugin declares `>=2.4.0` and
the Insight plugin `>=2.1.0`, and 2.6.0 satisfies both.

The next PR adds `@aws/durable-execution-sdk-js-helpers`, which requires
core `>=2.6.0`. It cannot install against the 2.5.0 in `main`, so it
waits for this one.

### Testing

- `npm ci` succeeds with the updated lockfile. The lockfile changes only
the three version lines, as in #933.
- Core SDK: `npm run build` and `npm test` pass (1281 tests).
- Testing SDK: `npm run build` and `npm test` pass (1204 passed, 3
skipped), including `peer-compatibility.test.ts`.
ParidelPooya added a commit that referenced this pull request Oct 3, 2026
…package (#949)

This PR adds two packages. Both are at `0.1.0-alpha.0`, and every export
is marked `@experimental`.

#### `@aws/durable-execution-sdk-js-extras`

A durable function sometimes needs to run work that is too long or too
heavy for one Lambda invocation, such as a CI build. A [Lambda
MicroVM](https://docs.aws.amazon.com/lambda/latest/dg/lambda-microvms-guide.html)
can run that work. This package adds two operations that drive a MicroVM
from a durable function:

- `microvm(context, name, input, config)` launches a new MicroVM,
delivers one job, waits for the result, and terminates the MicroVM.
- `microvmSession(context, name, config, handler)` launches one MicroVM
and lets the handler send it several jobs with `vm.invoke(ctx, name,
input, options)`. The jobs share the MicroVM's files and processes. The
session terminates the MicroVM when the handler returns or throws.

The function invocation ends while the MicroVM works. So no Lambda
compute is billed during the wait.

1. Both operations, and `vm.invoke`, are plain functions that take the
context first. They are built only from `runInChildContext`, `step`, and
`createCallback`. So they replay deterministically in the root context,
in a child context, and in `map` and `parallel` items.
2. Each part records its own subtype, such as `MicrovmLaunch`,
`MicrovmRequest`, and `MicrovmCallback`. The package exports them as
`MicrovmOperationSubType`.
3. Those subtypes need the `subType` option of `step` and
`createCallback`, which #939 added in core SDK 2.6.0. An older SDK
ignores the option and records `Step` and `Callback`. A later upgrade
would then change the recorded subtypes, and replay would fail
executions in flight.
4. So the package's peer range on the core SDK is `>=2.6.0`.

Other behavior, documented in the package README:

- A job reaches the MicroVM through the `run` lifecycle hook payload
when it fits in 4,096 Unicode code points, and over HTTP to the MicroVM
endpoint otherwise. The launch step records the delivery, so every
replay follows it. A session always uses HTTP.
- Delivery retries in two tiers. A short retry window runs inside one
step attempt, and the step retry strategy runs after it. Every call in
the first tier ends 10 seconds before the invocation times out. In a
session, the wait for a resume counts toward that window.
- A session can opt in to let the worker suspend the MicroVM when it is
idle (`autoSuspendOnIdle: true`, `autoSuspendIdleTime`). The next
`vm.invoke` resumes it. It is off by default, so a default session needs
only the permissions of `microvm` with HTTP delivery.

#### `@aws/durable-execution-sdk-js-microvm-worker`

This package runs in the MicroVM image. It is the other side of the
contract that the extras package sends.

- It answers the `run`, `resume`, `suspend`, and `terminate` lifecycle
hooks. It acts only on the first valid `run` hook.
- It runs each job in the background. It ignores a second delivery of a
job while the job runs, and after it ended. It remembers the callback
IDs of the last 1,000 finished jobs.
- It sends `SendDurableExecutionCallbackHeartbeat` while a job runs, and
reports the outcome with `SendDurableExecutionCallbackSuccess` or
`SendDurableExecutionCallbackFailure`. Completion reporting retries up
to 5 times, and each attempt ends after 30 seconds.
- In a session that asks for it, it suspends its own MicroVM after the
idle time, and refuses jobs with 503 while it suspends.

The two packages do not depend on each other. One runs in the function,
and the other runs in the MicroVM. So the JSON contract is declared in
both. A contract test checks that the two declarations match, and a
composed test runs the request step against the real worker listener.

#### Repository changes

- The root `build` and `test` scripts include both packages.
- `lintcommit.js` accepts the `extras` and `microvm-worker` scopes.
- `iterate-publish-npm.sh` lists both packages. The `insight-mcp`
packaging test requires every non-private `@aws/` package to be in that
list. Their version `0.1.0-alpha.0` is a prerelease, so the next release
publishes them under the `beta` dist-tag, as it did for the Insight
packages.
- The CHANGELOG gains an "Added" entry for both packages.
- The core README links to the extras package from its "Operation
Subtypes" section.
- `package-lock.json` adds `@aws-sdk/client-lambda-microvms` 3.1144.0.
That client needs newer shared `@aws-sdk` and `@smithy` packages. So 20
of them move forward by a minor or patch version. No package is removed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants