Skip to content

core: Do not close event loop too early#4819

Merged
rultor merged 2 commits intomasterfrom
Makman2/NEXTGEN/early-event-loop-close
Nov 6, 2017
Merged

core: Do not close event loop too early#4819
rultor merged 2 commits intomasterfrom
Makman2/NEXTGEN/early-event-loop-close

Conversation

@Makman2
Copy link
Copy Markdown
Member

@Makman2 Makman2 commented Oct 31, 2017

... when scheduling all dependency bears. Bears that offload no tasks
may cause the event-loop to close too early while other bears had no
chance to submit their tasks.

Fixes #4505

Comment thread coalib/core/Core.py Outdated
# process, as when no tasks were offloaded the event-loop
# could hang up otherwise.
self._cleanup_bear(bear)
bears_without_tasks_to_cleanup.append(bear)
Copy link
Copy Markdown
Contributor

@shreyans800755 shreyans800755 Nov 1, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we keep this condition before looping over the tasks ?

Something like:

if not tasks:
    bears_without_tasks_to_cleanup.append(bear)
    continue

for task in tasks
...

Might not save too many clock cycles, but it will make code cleaner.

Copy link
Copy Markdown
Member Author

@Makman2 Makman2 Nov 1, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah why not, if you like I would also add an additional debug-message BearXYZ scheduled no tasks., so we still keep the information that a bear has been processed 👍

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm just thinking if this should be done in a separate commit... This commit diff is smaller when not shifting the logic away^^

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO this can be done in the same commit as we are adding this condition in this commit. It doesn't make sense to add something in first commit and change the same in subsequent one especially in same PR.

Copy link
Copy Markdown
Contributor

@shreyans800755 shreyans800755 Nov 2, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And there won't be a case where we only want to rollback one of the commits and not the other one. So, its better to keep single commit.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as we are adding this condition in this commit

which condition? The if is already there (if not tasks)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh my bad. Can be done in different commit.

Comment thread coalib/core/Core.py Outdated
# process, as when no tasks were offloaded the event-loop
# could hang up otherwise.
self._cleanup_bear(bear)
bears_without_tasks_to_cleanup.append(bear)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh my bad. Can be done in different commit.

Comment thread coalib/core/Core.py Outdated
:param bears:
A list of bear instances to be scheduled onto the process pool.
"""
bears_without_tasks_to_cleanup = []
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bears_without_tasks_to_cleanup -> bears_without_tasks

Comment thread coalib/core/Core.py Outdated
bears_without_tasks_to_cleanup.append(bear)

for bear in bears_without_tasks_to_cleanup:
# We need to recheck our runtime if something is left to process, as
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we still need this comment ?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hm yeah... I think this is more or less self explaining 👍

Comment thread tests/core/CoreTest.py Outdated
self.assertEqual(
len(bear.dependency_results[BearA]), 1)

def test_run_many_dependency_bears_with_zero_tasks(self):
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

test_run_many_dependency_bears_with_zero_tasks -> test_run_multiple_dependency_bears_with_zero_tasks

Comment thread tests/core/CoreTest.py Outdated
return (((i,), {}) for i in range(tasks_count))


class DependentOnManyZeroTaskBearsTestBear(TestBearBase):
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

DependentOnManyZeroTaskBearsTestBear -> DependentOnMultipleZeroTaskBearsTestBear

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it matter? :D

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just trying to be more accurate with words. 😃

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Kk ;)

@Makman2 Makman2 force-pushed the Makman2/NEXTGEN/early-event-loop-close branch from 231b747 to 76f25b0 Compare November 5, 2017 15:55
@Makman2 Makman2 force-pushed the Makman2/NEXTGEN/early-event-loop-close branch from 76f25b0 to aa720e2 Compare November 5, 2017 15:56
@shreyans800755
Copy link
Copy Markdown
Contributor

ack aa720e2

@shreyans800755
Copy link
Copy Markdown
Contributor

ack a7b8e0b

@gitmate-bot gitmate-bot added process/approved The PR is approved and will be merged soon and removed process/pending review labels Nov 5, 2017
@Makman2
Copy link
Copy Markdown
Member Author

Makman2 commented Nov 5, 2017

@gitmate-bot rebase

... when scheduling all dependency bears. Bears that offload no tasks
may cause the event-loop to close too early while other bears had no
chance to submit their tasks.

Fixes #4505
Moving the condition upwards makes code more understandable, as
further scheduling code is not called at all because using now
a `continue` statement. This change effectively doesn't change any
behaviour, but makes control flow cleaner.
@gitmate-bot
Copy link
Copy Markdown
Collaborator

Hey! This pull request is being rebased automatically. Please DO NOT push while rebase is in progress or your changes would be lost!

@gitmate-bot
Copy link
Copy Markdown
Collaborator

Automated rebase was successful!

@gitmate-bot gitmate-bot force-pushed the Makman2/NEXTGEN/early-event-loop-close branch from aa720e2 to 5cbbe19 Compare November 5, 2017 21:07
@gitmate-bot gitmate-bot added process/pending review process/approved The PR is approved and will be merged soon and removed process/approved The PR is approved and will be merged soon process/pending review labels Nov 5, 2017
@Makman2
Copy link
Copy Markdown
Member Author

Makman2 commented Nov 6, 2017

@rultor merge

@rultor
Copy link
Copy Markdown
Contributor

rultor commented Nov 6, 2017

@rultor merge

@Makman2 OK, I'll try to merge now. You can check the progress of the merge here

@gitmate-bot gitmate-bot removed the process/approved The PR is approved and will be merged soon label Nov 6, 2017
@rultor rultor merged commit 5cbbe19 into master Nov 6, 2017
@rultor
Copy link
Copy Markdown
Contributor

rultor commented Nov 6, 2017

@rultor merge

@Makman2 Done! FYI, the full log is here (took me 2min)

@Makman2 Makman2 deleted the Makman2/NEXTGEN/early-event-loop-close branch November 6, 2017 13:46
@gitmate-bot gitmate-bot added the process/approved The PR is approved and will be merged soon label Nov 6, 2017
@gitmate-bot gitmate-bot removed the process/approved The PR is approved and will be merged soon label Jun 2, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Development

Successfully merging this pull request may close these issues.

NextGen-Core: Do not close event loop if not all bears were scheduled

5 participants