Skip to content

Events Retry Logic with limited attempts#132

Merged
wpscholar merged 21 commits intomainfrom
FIX/PRESS7-375-fix-retry-logic
May 8, 2025
Merged

Events Retry Logic with limited attempts#132
wpscholar merged 21 commits intomainfrom
FIX/PRESS7-375-fix-retry-logic

Conversation

@geckod22
Copy link
Copy Markdown
Contributor

@geckod22 geckod22 commented Apr 7, 2025

Proposed changes

This is an attempt to manage the retry logic with retry limit attempts in order to help the migration module to be able to send the migration complete event also if in the first attempt something gone wrong.

This refers to https://jira.newfold.com/browse/PRESS7-375

Type of Change

Production

  • Bugfix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Dependency update
  • Refactoring / housekeeping (changes to files not directly related to functionality)

Development

  • Tests
  • Dependency update
  • Environment update / refactoring
  • Documentation Update

Video

Checklist

  • I have read the CONTRIBUTING doc
  • I have viewed my change in a web-browser
  • Linting and tests pass locally with my changes
  • I have added tests that prove my fix is effective or that my feature works
  • I have added necessary documentation (if appropriate)

Further comments

@wpscholar
Copy link
Copy Markdown
Member

Looking good so far @geckod22
Let's add a PHPUnit test to validate that the event is removed when the attempt limit is hit. We should also have a unit test to validate that the timestamp sent to Hiive matches the one in the database.

@geckod22
Copy link
Copy Markdown
Contributor Author

geckod22 commented Apr 8, 2025

Looking good so far @geckod22 Let's add a PHPUnit test to validate that the event is removed when the attempt limit is hit. We should also have a unit test to validate that the timestamp sent to Hiive matches the one in the database.

Hi @wpscholar about the timestamp validation, it seems the events are taken from the database where the created_at is not changed (we only increment the attempts column), so on retry they still have the same timestamp; or do you mean a different thing?

@wpscholar
Copy link
Copy Markdown
Member

Looking good so far @geckod22 Let's add a PHPUnit test to validate that the event is removed when the attempt limit is hit. We should also have a unit test to validate that the timestamp sent to Hiive matches the one in the database.

Hi @wpscholar about the timestamp validation, it seems the events are taken from the database where the created_at is not changed (we only increment the attempts column), so on retry they still have the same timestamp; or do you mean a different thing?

@geckod22 Yes, we want to always pass the created_at value from the database. This makes it easy to de-dupe in the event we successfully send more than one event.

@AleTorrisi
Copy link
Copy Markdown
Contributor

Hi @wpscholar there's a point here which is not pretty clear, so I wonder what I don't understand.
We store in database all events whose status is "failed".
Any time we create a queue of events, we retrieve existing failed events from the database and for each of them we attempt again. In case the new attempt produces a success, we forward the event to Hiive.
There's no action in the middle of this process, so the timestamp we sent to Hiive is the one we retrieved from the database, there's no possibility is different, so my concern is what we should test here with a PHP unit?
Thanks for your time :)

@wpscholar
Copy link
Copy Markdown
Member

Hi @wpscholar there's a point here which is not pretty clear, so I wonder what I don't understand. We store in database all events whose status is "failed". Any time we create a queue of events, we retrieve existing failed events from the database and for each of them we attempt again. In case the new attempt produces a success, we forward the event to Hiive. There's no action in the middle of this process, so the timestamp we sent to Hiive is the one we retrieved from the database, there's no possibility is different, so my concern is what we should test here with a PHP unit? Thanks for your time :)

@AleTorrisi Previously, we didn't send the timestamp from the database. This has since been corrected, but I mentioned that we should have a test for it, as I don't think we ever created a test for this. I want to make sure we never have a regression. Last time we had attempted retries, having a unique timestamp every time we sent an event presented significant issues with de-duping and getting correct data. Before we try this again, I want to ensure we have a test that validates the timestamp isn't changing with each retry.

@AleTorrisi
Copy link
Copy Markdown
Contributor

AleTorrisi commented Apr 10, 2025

@wpscholar That’s ok, we didn’t know that behavior was different before and that created_at was not set on event.
That said, me and Armando see reasonable apply a change in the module and use the event creation date as value to store in the database.
At the moment, when an event is pushed in the database, we populate the created_at column with the current timestamp

$time = current_time( 'mysql' );
Path: wp-module-data/includes/EventQueue/Queues/BatchQueue.php on line 38

This way will not be possible have a discrepancy between the event and database data.

Do you agree with this solution?

@AleTorrisi
Copy link
Copy Markdown
Contributor

The issue with creation date has been fixed in this
#137

@geckod22 geckod22 marked this pull request as ready for review April 22, 2025 08:12
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 22, 2025

@geckod22 geckod22 changed the title first commit Event Retry Logic Apr 22, 2025
@geckod22 geckod22 changed the title Event Retry Logic Events Retry Logic Apr 22, 2025
@geckod22 geckod22 changed the title Events Retry Logic Events Retry Logic with limited attempts Apr 22, 2025
@AleTorrisi
Copy link
Copy Markdown
Contributor

@wpscholar last commit should suits well your request

arunshenoy99
arunshenoy99 previously approved these changes May 6, 2025
Copy link
Copy Markdown
Member

@arunshenoy99 arunshenoy99 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. Once this is merged, maybe we can keep a branch open without these changes in case we need to revert again.

@AleTorrisi
Copy link
Copy Markdown
Contributor

@wpscholar default value set at 1

arunshenoy99
arunshenoy99 previously approved these changes May 7, 2025
Copy link
Copy Markdown
Member

@arunshenoy99 arunshenoy99 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@wpscholar @circlecube If this looks good to you, can we tag a release and include it in the May 14th plugin release?

@circlecube
Copy link
Copy Markdown
Member

@arunshenoy99, @BrianHenryIE wanted to add some unit tests here before merging.

Aiming at getting them added today and we can still merge and tag a module release.

@wpscholar wpscholar merged commit 2000f83 into main May 8, 2025
13 checks passed
@wpscholar wpscholar deleted the FIX/PRESS7-375-fix-retry-logic branch May 8, 2025 12:10
BrianHenryIE added a commit that referenced this pull request May 19, 2025
…retry-logic"

This reverts commit 2000f83, reversing
changes made to eec827e.
@BrianHenryIE BrianHenryIE mentioned this pull request May 19, 2025
14 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants