Skip to content

[Fix #5436] Store extending result in all backends#5661

Merged
auvipy merged 8 commits into
celery:masterfrom
safwanrahman:backend
Nov 29, 2019
Merged

[Fix #5436] Store extending result in all backends#5661
auvipy merged 8 commits into
celery:masterfrom
safwanrahman:backend

Conversation

@safwanrahman

Copy link
Copy Markdown
Contributor

This fixes #5436 except cassendra. I do not have any experience with cassendra, so dont know how to fix that.

@thedrow

thedrow commented Aug 4, 2019

Copy link
Copy Markdown
Contributor

What about backends such as MongoDB which override _store_result?

@safwanrahman

Copy link
Copy Markdown
Contributor Author

What about backends such as MongoDB which override _store_result?

I have changed the mongodb one also!

@thedrow

thedrow commented Aug 4, 2019

Copy link
Copy Markdown
Contributor

It shouldn't be hard to add the same capability to the Cassandra backend.
You'll need to edit these queries.
Add a condition here, here and here to use the correct table creation, insert statement and read statement according to the configuration value.
CQL is very much like SQL in this regard.

@georgepsarakis georgepsarakis left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work, seems like a good improvement! There are several failures in unit tests that seem directly related to the changes, let me know if you need help resolving those.

@safwanrahman

Copy link
Copy Markdown
Contributor Author

Thanks @geoffreybauduin. I am trying to run the test locally to fix the tests actually!

@ghost

ghost commented Oct 17, 2019

Copy link
Copy Markdown

hi guys,
do celery Result Stores support MongoDB. Thanks you vvery much
image

@georgepsarakis

georgepsarakis commented Oct 17, 2019

Copy link
Copy Markdown
Contributor

@sckhg1367 MongoDB is supported, probably the documentation needs to be updated:

https://github.com/celery/celery/blob/master/celery/backends/mongodb.py

Would you like to open a PR perhaps to include all available result stores?

@ghost

ghost commented Oct 17, 2019

Copy link
Copy Markdown

Thanks @geoffreybauduin.

@akshjain83

Copy link
Copy Markdown

Any updates on this? This is a crucial fix and looks like we are pretty close here. Can someone please prioritize this?

@thedrow thedrow added this to the 4.4.0 milestone Nov 14, 2019
@thedrow

thedrow commented Nov 14, 2019

Copy link
Copy Markdown
Contributor

I'll work on it.

@safwanrahman

Copy link
Copy Markdown
Contributor Author

Sorry I could not get time to work on this! @thedrow Can you let me know what do you think about failing tests?

@thedrow

thedrow commented Nov 14, 2019

Copy link
Copy Markdown
Contributor

I'll rebase first and then let you know.

@akshjain83

Copy link
Copy Markdown

Great thanks guys for taking a look into this.

@thedrow

thedrow commented Nov 18, 2019

Copy link
Copy Markdown
Contributor

I'm not sure I resolved the conflict correctly which is why I merged.
Can anyone take a look?

@auvipy

auvipy commented Nov 18, 2019

Copy link
Copy Markdown
Member

seems only 1 failure

@safwanrahman

Copy link
Copy Markdown
Contributor Author

@thedrow I think rebase is appropiate here. I will rebase it upon master and will take a look at the test failure.

@akshjain83

Copy link
Copy Markdown

Any updates here? Can someone please share the estimated timeline for releasing Milestone 4.4.0 ? Please let me know if I can help in some way, thanks.

@auvipy

auvipy commented Nov 19, 2019

Copy link
Copy Markdown
Member

30th Nov

@safwanrahman

Copy link
Copy Markdown
Contributor Author

@auvipy Which tests you are referring?

@auvipy

auvipy commented Nov 24, 2019

Copy link
Copy Markdown
Member

Comment thread celery/backends/base.py
def is_cached(self, task_id):
return task_id in self._cache

def _get_result_meta(self, result,

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

particularly this block and related changes tests

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@auvipy Do you mean writing new tests?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes some more test

@akshjain83

Copy link
Copy Markdown

@safwanrahman Are you working on the tests? I haven't contributed to this project before but I could try to help if you need any. Please let me know if we can still make it to the Milestone 4.4.0 release.

@auvipy

auvipy commented Nov 27, 2019

Copy link
Copy Markdown
Member

@safwanrahman Are you working on the tests? I haven't contributed to this project before but I could try to help if you need any. Please let me know if we can still make it to the Milestone 4.4.0 release.

you can push some tests on his forks branch :)

@akshjain83

akshjain83 commented Nov 27, 2019

Copy link
Copy Markdown

@auvipy @safwanrahman While I am touching this branch, just thought of checking with you guys if it would make sense to create a __init__ method/constructor in this class as we have changes in this PR here and we can leverage this parent init via super here.

@akshjain83

akshjain83 commented Nov 28, 2019

Copy link
Copy Markdown

@safwanrahman Are you working on the tests? I haven't contributed to this project before but I could try to help if you need any. Please let me know if we can still make it to the Milestone 4.4.0 release.

you can push some tests on his forks branch :)

@safwanrahman @auvipy I few tests that I would like to push but I don't have permissions to push to your fork. Please let me know how you would like me this few changes.

I will be out for the week so just created the PR to your branch here, please feel free to update if you would like to proceed meanwhile. Thanks.

@safwanrahman

Copy link
Copy Markdown
Contributor Author

@akshjain83 Thanks for the commit, sorry I did not get time to write the tests myself. I have cherry-picked the commit and pushed to the branch.
@auvipy possible to merge?

@auvipy
auvipy merged commit 984fb5f into celery:master Nov 29, 2019
@akshjain83

Copy link
Copy Markdown

No worries @safwanrahman Happy to contribute. @auvipy How does the timeline looks like for milestone 4.4.0 ? Any updated estimates?

@thedrow

thedrow commented Dec 4, 2019

Copy link
Copy Markdown
Contributor

I'm hoping to make a release by the weekend.

@akshjain83

Copy link
Copy Markdown

Thanks @thedrow