Skip to content

Clean TraceBack to reduce memory leaks for exception task#6024

Merged
auvipy merged 11 commits into
celery:masterfrom
heedong-jung:master
Apr 7, 2020
Merged

Clean TraceBack to reduce memory leaks for exception task#6024
auvipy merged 11 commits into
celery:masterfrom
heedong-jung:master

Conversation

@heedong-jung

@heedong-jung heedong-jung commented Apr 6, 2020

Copy link
Copy Markdown
Contributor

Note: Before submitting this pull request, please review our contributing
guidelines
.

Description

Fixes: #6023

It seems that a frame leak is occurring in the use of the e value returned by 'Execption as e'. Del tb is not enough. So we clean up the traceback in the last part that can be explicit.

@auvipy auvipy left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you also add proper unit test?

@auvipy auvipy added this to the 4.4.x milestone Apr 6, 2020
@codecov

codecov Bot commented Apr 6, 2020

Copy link
Copy Markdown

Codecov Report

Merging #6024 into master will increase coverage by <.01%.
The diff coverage is 93.54%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #6024      +/-   ##
==========================================
+ Coverage   83.42%   83.43%   +<.01%     
==========================================
  Files         145      145              
  Lines       17186    17217      +31     
  Branches     2118     2126       +8     
==========================================
+ Hits        14338    14365      +27     
- Misses       2628     2630       +2     
- Partials      220      222       +2
Impacted Files Coverage Δ
celery/backends/base.py 95.77% <90%> (-0.12%) ⬇️
celery/app/trace.py 99.41% <95.23%> (-0.28%) ⬇️
celery/backends/asynchronous.py 59.72% <0%> (-0.93%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4695f8f...dfedf3d. Read the comment docs.

@auvipy auvipy left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please check the python 2.7 failure and increase code coverage.

Comment thread celery/backends/base.py Outdated
Comment thread t/unit/tasks/test_trace.py Outdated
Comment thread t/unit/tasks/test_trace.py Outdated
@auvipy
auvipy merged commit c4dd6e8 into celery:master Apr 7, 2020
Comment thread celery/backends/base.py
@heedong-jung heedong-jung mentioned this pull request Jun 25, 2020
18 tasks
thebarbershop pushed a commit to sendbird-graveyard/celery that referenced this pull request Jul 1, 2020
* Clean TraceBack to reduce memory leaks

* add unit test

* add unit test

* reject unittest

* Patch For Python 2.7 compatibility

* update unittest

* Register to the garbage collector by explicitly referring to f_locals.

* need more check

* update code coverage

* update Missing unit test

* 3.4 -> 3.5

Co-authored-by: heedong.jung <[email protected]>
thebarbershop pushed a commit to sendbird-graveyard/celery that referenced this pull request Jul 1, 2020
* Clean TraceBack to reduce memory leaks

* add unit test

* add unit test

* reject unittest

* Patch For Python 2.7 compatibility

* update unittest

* Register to the garbage collector by explicitly referring to f_locals.

* need more check

* update code coverage

* update Missing unit test

* 3.4 -> 3.5

Co-authored-by: heedong.jung <[email protected]>