Hey thanks for working on this. I also worked on it in parallel as I didn't know you had started.

One thing maybe you can merge from my branch is the new env var config key: REDASH_WORKER_MAX_MEMORY_PER_CHILD
This is updated in two files:
https://github.com/getredash/redash/pull/2535/files#diff-324d42e08af87005833f42ea274a9abeR37
https://github.com/getredash/redash/pull/2535/files#diff-ad93995bb355251c02f87fc02db69351R53

my PR: #2535

I couldn't find your fork to create a PR from mine

@atharvai I have created a PR from your celery_upgrade branch to my celery4 branch.
Would it be possible for you to fix the conflicts?

@arikfr Review possible?

@mashrikt We haven't forgotten. We'll review this as soon as we have some time. :)

@RichardLitt Good to hear from you again!
Hope you're well.

Please note there is a new issue with celery 4.1.0 because of an unpinned dependency in celery. see mozilla#405

Is there any more progress on this? I imagine we'll want to upgrade to celery==4.1.1 too because of issue Mozilla folks noticed.

This was fixed in #2773. Thanks @mashrikt for your help!