Repository navigation
Bug: CachePersistenceLayer raises IdempotencyValidationError on retries with the same payload when payload_validation_jmespath is set #8655
Description
Expected Behaviour
With CachePersistenceLayer (Redis/Valkey) and payload_validation_jmespath set, a retry with the same payload should return the stored response, the same way it does with DynamoDBPersistenceLayer.
Current Behaviour
The second call with the exact same payload raises IdempotencyValidationError: Payload does not match stored record for this event key.
_put_in_progress_record stores the payload hash under validation_key_attr, but _update_record overwrites the whole key with a new SET that only contains data, status and expiration. The hash is lost when the record moves to COMPLETED, so on the next call payload_hash is read back as "" and never matches.
DynamoDBPersistenceLayer._update_record writes the validation key when payload_validation_enabled is true, so only the Redis layer is affected.
Code snippet
from aws_lambda_powertools.utilities.idempotency import IdempotencyConfig, idempotent_function
from aws_lambda_powertools.utilities.idempotency.persistence.redis import CachePersistenceLayer
class InMemoryClient:
"""Minimal client with the get/set/delete calls the layer uses."""
def __init__(self):
self.store = {}
def get(self, name):
return self.store.get(name)
def set(self, name, value, ex=0, nx=False):
if nx and name in self.store:
return None
self.store[name] = value
return True
def delete(self, name):
self.store.pop(name, None)
client = InMemoryClient()
persistence_layer = CachePersistenceLayer(client=client)
config = IdempotencyConfig(event_key_jmespath="order_id", payload_validation_jmespath="amount")
@idempotent_function(data_keyword_argument="order", persistence_store=persistence_layer, config=config)
def process_order(order):
return {"order_id": order["order_id"]}
order = {"order_id": "o-1", "amount": 10}
print(process_order(order=order)) # {'order_id': 'o-1'}
print(list(client.store.values())[0]) # no "validation" field in the stored record
print(process_order(order=order)) # IdempotencyValidationErrorPossible Solution
Add the validation key to the mapping in CachePersistenceLayer._update_record, like _put_in_progress_record already does:
if self.payload_validation_enabled:
item["mapping"][self.validation_key_attr] = data_record.payload_hashThe existing test test_idempotent_function_and_lambda_handler_redis_validation only covers a different payload, so a test for a same-payload retry would catch this. I'm happy to send a PR.
Steps to Reproduce
Run the snippet above. No Redis server is needed, the in-memory client follows the same get/set/delete calls. The same happens with a real Redis or Valkey client.
Powertools for AWS Lambda (Python) version
latest (also on develop)
AWS Lambda function runtime
3.13
Packaging format used
PyPi
Debugging logs
aws_lambda_powertools.utilities.idempotency.exceptions.IdempotencyValidationError: Payload does not match stored record for this event key
Activity
boring-cyborg commentedon Oct 22, 2026
Thanks for opening your first issue here! We'll come back to you as soon as we can.
In the meantime, check out the #python channel on our Powertools for AWS Lambda Discord: Invite link
powertools-for-aws-oss-automation commentedon Oct 22, 2026
Thanks for opening your first issue here! We'll come back to you as soon as we can.
In the meantime, check out our GitHub Discussions: Join the conversation
thebadalov commentedon Oct 22, 2026
I'd like to work on this one. I already have the fix and a regression test ready (same-payload retry with
payload_validation_jmespathon the Redis layer), and I checked it against a real Redis server as well. Could you assign it to me? I'll open the PR once it's triaged.leandrodamascena commentedon Oct 24, 2026
Thanks @thebadalov, confirmed. I reproduced it on
developwith your snippet and with a real Valkey 8, and the two-line fix in_update_recordmakes the same-payload retry return the stored response while a different payload still raises. This was also the first item on my Idempotency list in #8603, so good catch.It's yours. Please send the PR with a test for the same-payload retry on the Redis layer. This will go into 3.40.0.
thebadalov commentedon Oct 26, 2026
Thanks @leandrodamascena for confirming it and for checking it on Valkey too! I've opened #8656 with the fix and a test for the same-payload retry on the Redis layer.
powertools-for-aws-oss-automation commentedon Nov 4, 2026
Warning
This issue is now closed. Please be mindful that future comments are hard for our team to see.
If you need more assistance, please either reopen the issue, or open a new issue referencing this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.
github-actions commentedon Nov 10, 2026
This is now released under 3.40.0 version!