fix(sdk): return FAILED when replay validation detects non-determinism - #867

…AT map items (#894)

`context.map` and `context.parallel` with `nesting: FLAT` rebuilt an
**empty** result on replay when the batch was large enough to be
summarized, even though every item had succeeded.

A batch whose aggregate result exceeds the checkpoint size limit is
stored as a summary with `ReplayChildren`, and each later resumption
reconstructs it from the per-item checkpoints in
`ConcurrencyController.replayItems`. That reconstruction decided whether
an item had finished by probing the item's context checkpoint — but with
`FLAT` nesting the per-item contexts are virtual and never checkpointed,
so every item read as unfinished and was skipped.

Not cosmetic: code deriving control flow from the batch result, such as
a fan-out sized from the results, then creates a different set of
operations on replay than it did live. The replay-consistency check
detects the divergence and raises `NonDeterministicExecutionError`,
which SDKs at or below 2.3.0 reported as a `PENDING` response rather
than a failure. While other work is still draining `PENDING` is a legal
answer, so the corruption is invisible; once nothing is left pending the
service rejects the response with "Cannot return PENDING status with no
pending operations" and the execution fails after deterministic retries.
Batches whose result fits in one checkpoint are replayed by
deserialization and never enter this path.

## Change

The batch payload now records which items reached a terminal state, one
character per item in index order. Replay reads that record instead of
inferring it. Only terminality is recorded — an item's
`SUCCEEDED`/`FAILED` outcome still comes from re-driving it.

Terminality resolves in order of authority:

1. the recorded statuses, with any context record the item has required
to agree
2. the item's own context checkpoint, for payloads with no recorded
statuses
3. a probe of the item's operations, for payloads predating the field

The agreement requirement in (1) is one-sided on purpose. A marker
saying unfinished overrides a terminal context record, because the
statuses are what the live `BatchResult` contained. The reverse is not
honoured: re-driving an item whose context checkpoint is non-terminal
hits `checkForNonResolvingPromise` and the reconstruction never settles
(#751).

The probe in (3) walks every recorded operation and requires all of them
to be terminal. Reading only the first is unsound — a mid-flight item
can have a `SUCCEEDED` first operation, and re-driving it would execute
its unfinished operations for real.

Two payload shapes are recognised, since `ReplayChildren` is reached two
ways: the summary envelope carrying `itemStatuses`, and the full
serialized result written when `childOperationsDepth` opts a context
into child preservation, which has no summary envelope but does carry
`all: [{ index, status }]`. Parsing is validated as strictly as
`parseRecordedCompletionReason`.

There is deliberately no size handling on the record. Every batch item
runs its body inside a context, and a context is meant to wrap at least
one durable operation, so each item costs at least one. The service caps
an execution at 30,000 operations, which bounds any batch at roughly
that many items — about 30 KB of markers at one byte each, an order of
magnitude inside the 256 KB checkpoint limit. Capping the marker count
instead would be actively harmful: dropping the field sends replay back
to probing, and probing cannot recover an item whose body created no
durable operation, which is the corruption this record exists to
prevent. A test pins the bound at the largest batch the service allows.

## Behaviour change

A virtual item with no durable operation of its own is now re-driven on
replay, so any part of its mapper body not wrapped in a durable
operation runs again. This is the documented contract for virtual
contexts and what `NESTED` already does, but it is new on this path.

That shape is against best practice regardless — a context is a
container for durable operations, and one with nothing durable inside it
is not checkpointed, so its work re-runs on every replay anyway. The
TSDoc for `DurableContext.runInChildContext` and `MapFunc` now says so,
and #896 tracks an eslint rule for it.

## Testing

New `examples/map/flat-summarized-replay` covering the rebuild across a
suspension for `FLAT` and `NESTED`, an item that checkpoints no durable
operation, and a small-result control. It is `localOnly`: the suspension
depends on the local runner's virtual clock, so deploying it would only
produce a skipped cloud test. Every case asserts at least two
invocations, so none can pass without actually suspending.

Against unpatched code both `FLAT` cases fail (`replayedResultCount` 0
against a live count of 8) while the controls pass. 14 new unit cases
cover both directions of a statuses/record disagreement, the
`childOperationsDepth` shape, a crafted `itemStatuses` value, the probe
rejecting a mid-flight item, and the size bound at the service's
operation limit.

The `PENDING` misreport that hid this was fixed separately on mainline
by #867; this addresses the underlying corruption.

Bumps @aws/durable-execution-sdk-js from 2.3.0 to 2.3.1. This PR exists to
cut the release tag for what has already landed on main; the only changes
are the version field, the testing package's peer-dependency upper bound,
and the lockfile entries for both.

Patch rather than minor: all three SDK changes since sdk-2.3.0 are fixes,
and none adds public API. The checkpoint payload gains one field (#894),
which is compatible in both directions — an older SDK ignores it, and a
newer SDK falls back to probing on payloads written without it.

**Fixed**

- Summarized replay of a FLAT-nested map or parallel rebuilt an EMPTY result
even though every item had succeeded (#894). Such a batch is checkpointed as
a summary and reconstructed from its per-item checkpoints on each later
resumption, but the reconstruction decided whether an item had finished by
probing the item's context checkpoint — and with FLAT nesting those per-item
contexts are virtual and never checkpointed, so every item read as unfinished
and was skipped. Code deriving control flow from the batch result then created
a different set of operations on replay than it had live, which replay
validation reported as a non-determinism failure. The batch payload now
records which items reached a terminal state and replay reads that record.
One behaviour change on this path: a virtual item that performs no durable
operation of its own is now re-driven, so any part of its mapper body outside
a durable operation runs again — the same as NESTED already does, and a shape
the TSDoc now discourages.

- Replay validation now answers FAILED instead of PENDING (#867). A
NonDeterministicExecutionError is the only production error carrying
TerminationReason.CUSTOM, and runHandler had no branch for it, so it fell
through to the catch-all and answered {Status: "PENDING"} with no error and
no log line. PENDING asks the service to keep the execution alive and invoke
again, but the fault is deterministic: every replay reproduced it, the
diagnostic the SDK had already built never reached the customer, and once
nothing was actually pending the service rejected the response with "Cannot
return PENDING status with no pending operations". Suspend reasons are now an
allowlist and every other reason answers FAILED carrying the error, so a
reason added later without a branch fails closed instead of silently claiming
progress.

- wait-for-condition surfaces serdes failures instead of silently restarting
(#754). The handler used a bare catch around a native serdes.deserialize, so
a checkpointed polling state that could not be deserialized was swallowed and
the condition loop restarted from config.initialState. It now uses
safeDeserialize, which logs, rethrows SerDesException and terminates through
the termination manager, matching the Java and Python SDKs.

Bumps @aws/durable-execution-sdk-js from 2.3.0 to 2.3.1. This PR exists to
cut the release tag for what has already landed on main; the only changes
are the version field, the testing package's peer-dependency upper bound,
and the lockfile entries for both.

Patch rather than minor: all three SDK changes since sdk-2.3.0 are fixes,
and none adds public API. The checkpoint payload gains one field (#894),
which is compatible in both directions — an older SDK ignores it, and a
newer SDK falls back to probing on payloads written without it.

**Fixed**

- Summarized replay of a FLAT-nested map or parallel rebuilt an EMPTY result
even though every item had succeeded (#894). Such a batch is checkpointed as
a summary and reconstructed from its per-item checkpoints on each later
resumption, but the reconstruction decided whether an item had finished by
probing the item's context checkpoint — and with FLAT nesting those per-item
contexts are virtual and never checkpointed, so every item read as unfinished
and was skipped. Code deriving control flow from the batch result then created
a different set of operations on replay than it had live, which replay
validation reported as a non-determinism failure. The batch payload now
records which items reached a terminal state and replay reads that record.
One behaviour change on this path: a virtual item that performs no durable
operation of its own is now re-driven, so any part of its mapper body outside
a durable operation runs again — the same as NESTED already does, and a shape
the TSDoc now discourages.

- Replay validation now answers FAILED instead of PENDING (#867). A
NonDeterministicExecutionError is the only production error carrying
TerminationReason.CUSTOM, and runHandler had no branch for it, so it fell
through to the catch-all and answered {Status: "PENDING"} with no error and
no log line. PENDING asks the service to keep the execution alive and invoke
again, but the fault is deterministic: every replay reproduced it, the
diagnostic the SDK had already built never reached the customer, and once
nothing was actually pending the service rejected the response with "Cannot
return PENDING status with no pending operations". Suspend reasons are now an
allowlist and every other reason answers FAILED carrying the error, so a
reason added later without a branch fails closed instead of silently claiming
progress.

- wait-for-condition surfaces serdes failures instead of silently restarting
(#754). The handler used a bare catch around a native serdes.deserialize, so
a checkpointed polling state that could not be deserialized was swallowed and
the condition loop restarted from config.initialState. It now uses
safeDeserialize, which logs, rethrows SerDesException and terminates through
the termination manager, matching the Java and Python SDKs.

Bumps @aws/durable-execution-sdk-js from 2.3.0 to 2.3.1. This PR exists
to
cut the release tag for what has already landed on main; the only
changes
are the version field, the testing package's peer-dependency upper
bound,
the lockfile entries for both, and a CHANGELOG roll.

The CHANGELOG had not been rolled since 2.0.0, so `[Unreleased]` was
carrying
three releases' worth of notes at once: the
runtimes-without-AsyncLocalStorage
entry that shipped in 2.3.0, the eslint-plugin 1.1.0 entries that
shipped
separately, and this release's fixes. Those are now under `[2.3.0]`,
`[eslint-plugin 1.1.0]` and `[2.3.1]`, with `[Unreleased]` left empty.
No
wording changed; the one addition is an entry for #867, which had none.

Note that 2.1.0 through 2.3.0 remain only partially documented here —
the full
notes for those releases live in their bump PR bodies (#822 and earlier)
and
were never copied into this file. Reconstructing them is left for a
follow-up.

Patch rather than minor: all three SDK changes since sdk-2.3.0 are
fixes,
and none adds public API. The checkpoint payload gains one field (#894),
which is compatible in both directions — an older SDK ignores it, and a
newer SDK falls back to probing on payloads written without it.

**Fixed**

- Summarized replay of a FLAT-nested map or parallel rebuilt an EMPTY
result
even though every item had succeeded (#894). Such a batch is
checkpointed as
a summary and reconstructed from its per-item checkpoints on each later
resumption, but the reconstruction decided whether an item had finished
by
probing the item's context checkpoint — and with FLAT nesting those
per-item
contexts are virtual and never checkpointed, so every item read as
unfinished
and was skipped. Code deriving control flow from the batch result then
created
a different set of operations on replay than it had live, which replay
validation reported as a non-determinism failure. The batch payload now
records which items reached a terminal state and replay reads that
record.
One behaviour change on this path: a virtual item that performs no
durable
operation of its own is now re-driven, so any part of its mapper body
outside
a durable operation runs again — the same as NESTED already does, and a
shape
the TSDoc now discourages.

- Replay validation now answers FAILED instead of PENDING (#867). A
NonDeterministicExecutionError is the only production error carrying
TerminationReason.CUSTOM, and runHandler had no branch for it, so it
fell
through to the catch-all and answered {Status: "PENDING"} with no error
and
no log line. PENDING asks the service to keep the execution alive and
invoke
again, but the fault is deterministic: every replay reproduced it, the
diagnostic the SDK had already built never reached the customer, and
once
nothing was actually pending the service rejected the response with
"Cannot
return PENDING status with no pending operations". Suspend reasons are
now an
allowlist and every other reason answers FAILED carrying the error, so a
reason added later without a branch fails closed instead of silently
claiming
progress.

- wait-for-condition surfaces serdes failures instead of silently
restarting
(#754). The handler used a bare catch around a native
serdes.deserialize, so
a checkpointed polling state that could not be deserialized was
swallowed and
the condition loop restarted from config.initialState. It now uses
safeDeserialize, which logs, rethrows SerDesException and terminates
through
the termination manager, matching the Java and Python SDKs.