fix(idempotency): clear the in progress record when the response can't be serialized - #8740

Issue number:

closes #8721

Summary

Changes

When the decorated function returns a value that the output serializer or json.dumps can't convert, delete the in progress record before raising IdempotencyPersistenceLayerError, the same way the record is deleted when the function itself raises. Cleanup runs at both conversion points: the output serializer's to_dict in the handler, and json.dumps in BasePersistenceLayer.save_success. Both run before anything is written. The error type and message are unchanged.

A failure while writing the COMPLETED record still keeps the record, because the write may have landed and deleting it would let a retry run the function twice. New tests guard that boundary.

The "Handling exceptions" docs section and its example now describe the new failure mode. They say cleanup is attempted, retries wait for expiry if the deletion itself fails, and write failures keep the record.

User experience

Before: returning e.g. a datetime, set or bytes failed the invocation, but left the record INPROGRESS. Every retry with the same payload raised IdempotencyAlreadyInProgressError until the in progress expiry, or for expires_after_seconds (one hour by default) without register_lambda_context.

After: the invocation still fails with the same error, but the next call with the same payload runs the function again.

Known limitation, not introduced here: deletion is by key, without an ownership check. If a function outlives expires_after_seconds and another invocation reacquires the key, a cleanup can remove the other invocation's record. Upstream's existing cleanup for function exceptions already has this behaviour. I reproduced it on unchanged develop with a scripted replacement record (fake Redis client, public decorator API). This PR adds a new trigger to that existing cleanup, but no new mechanism. An ownership-aware delete (DynamoDB conditional delete, Redis compare-and-delete) seems worth a separate issue. I'm happy to open one if you agree.

Verification

AI assistants (Claude and Codex) helped implement and review this change. I ran the commands below and read the output.

• New tests against the unchanged upstream implementation: the 6 serialization-cleanup cases fail. The 4 write-failure retention cases pass on both, since they are guards. With the fix, all 10 pass.
• Counterfactual: widening cleanup to also cover a failed persistence write makes all 4 retention guards fail.
• Idempotency unit and functional tests: 212 passed on Python 3.12. The functional idempotency suite also passed on Python 3.10, 3.11, 3.13 and 3.14 (run before the final, behaviour-preserving helper refactor; 3.12 was rerun after it).
• Full gate on Python 3.12, make -k pr ty exited 0: 4012 passed, 73 skipped, 10 deselected, plus all 10 performance tests. Ruff, mypy (949 source files), ty, docs lint, security and complexity checks passed. An earlier gate flagged the handler module at complexity rank B; the serializer cleanup moved into a small helper and the gate was rerun clean.
• Tests cover DynamoDB through botocore Stubber and Redis through the existing fake client, using only the public decorator API. The issue reproduces locally without AWS resources, so no live service was called.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Disclaimer: We value your time and bandwidth. As such, any pull requests created on non-triaged issues might not be successful.