Use DynamoDB TTL to auto-expire old task results#5275
Conversation
auvipy
left a comment
auvipy
left a comment
There was a problem hiding this comment.
you need to add tests for the proposed change.
Implement a TTL attribute on the DynamoDB result backend and configure the DB to auto expire documents using the TTL.
73fd8a3 to
3b55881
Compare
|
check the flake8 error |
georgepsarakis
left a comment
georgepsarakis
left a comment
There was a problem hiding this comment.
Nice work so far! I have noted some considerations, let me know if you need any help or clarifications.
|
|
||
| def test_update_table_ttl_change_pending(self): | ||
| self.backend._client = MagicMock() | ||
| mock_describe_ttl = self.backend._client.describe_time_to_live = MagicMock() |
There was a problem hiding this comment.
moto supports table TTL, so you could use it here instead of creating Mock objects, as a means of functional testing, especially since the local DynamoDB does not support TTL yet.
| achieved_state = current_status == expected | ||
| sleep(1) | ||
|
|
||
| def _update_table_ttl(self): |
There was a problem hiding this comment.
I think this operation should be optional, since some users may want to retain results.
| self._timestamp_field.data_type: str(_now) | ||
| }, | ||
| self._ttl_field.name: { | ||
| self._ttl_field.data_type: str(int(_now) + int(self.expires)) |
There was a problem hiding this comment.
Perhaps we should handle cases where expires is None/0 which disables expiration, as seen in the documentation?
What happens if you set NULL as the TTL field value, if this is possible? Would that perhaps disable expiration for this document?
Additionally, DynamoDB should be added in the documentation list of result backends supporting expiration.
|
@crgwbr Any chance you'll get to finish this before 4.3 GA? |
|
@crgwbr would you like to resume work here perhaps? |
4 participants
Description
Implement a TTL attribute on the DynamoDB result backend and configure the DB to
auto expire documents using the TTL.