Backport #3197 to 1.x-stable: Shut down profiler if any components failed#3216
Merged
ivoanjo merged 5 commits intoOct 27, 2023
Merged
Conversation
**What does this PR do?** This PR changes the profiler `Collectors::CpuAndWallTimeWorker` and `Scheduler` components to signal the top `Profiler` instance when they failed, so the `Profiler` can stop the other component as well. **Motivation:** While never reported by a customer, it ocurred to me that right now if one of the component fails, the other will keep working in a weird state. Of particular concern, if the scheduler failed, the `StackRecorder` would trigger a memory leak: it would start accumulating samples and never flush them. This PR changes the profiler so that if one component stops with a failure, the other will do so as well. **Additional Notes:** I'm not especially happy with how this change turned out. 1. The error handling in the scheduler somewhat duplicates similar logic in the `Core::Workers::Async` helper, which it uses. I wanted to avoid changing the shared helper just for this use-case, which I managed, but at the cost of a bit of duplication (including a duplicated log message, sigh). 2. I don't like the `on_failure_proc: nil`, but adding the argument would mean changing A LOT of tests, which is doable but has its own ugliness as well. I've considered passing the `on_failure_proc` via the constructor, but it's not entirely straightforward: the `Profiler` instance is created after the two others, so we'd need a way to break the loop. In the end, I decided to move forward with something, as I think it's important to plug this gap. **How to test the change?** This change includes test coverage.
I haven't yet gotten used to updating these as part of my usual workflow :sweatsmile:.
As pointed out during PR review by @AlexJF.
AlexJF
approved these changes
Oct 27, 2023
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is a backport of #3197 to the 1.x-stable branch.
I merely rebased the commits on top of the other branch, with no other changes.