Display running pipelines that did not pass through the build queue#502
Merged
nikita-tkachenko-datadog merged 1 commit intoFeb 6, 2025
Conversation
nikita-tkachenko-datadog
marked this pull request as ready for review
February 6, 2025 16:35
nikita-tkachenko-datadog
deleted the
nikita-tkachenko/display-running-pipelines-that-werent-queued
branch
February 6, 2025 16:42
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Requirements for Contributing to this repository
What does this PR do?
Ensures that (arguably) malformed pipelines that seem to be neither scripted nor declarative are displayed in Datadog - not only once finished, but also while running.
The way running pipelines work is that the plugin submits a pipeline to the backend once it has started, and later submits the same pipeline once it has finished - the backend updates pipeline record and the pipeline status changes in the UI.
The caveat with the running pipelines is that Datadog backend requires that a pipeline start time never changes: there are some technical limitations related to how pipeline data is stored, due to which the start time needs to be constant.
The problem is that the true start time of a pipeline is only known while the pipeline has left the build queue (while the pipeline is in the queue it has not started running yet), which for some pipelines happens after pipeline listeners have been invoked (this concerns the build listener, the graph listener, the step listener, the SCM checkout listener - all of these listeners may fire before pipeline leaves the build queue).
To ensure that the start time does not change the plugin does not submit pipeline data to the backend if the timestamp of that pipeline leaving the build queue is not known yet.
The problem is that some pipelines are executed without being added to the build queue.
Here is an example of such pipeline (missing the
pipeline {}and thenode {}top-level blocks):The workaround introduced in this PR is: when the
DatadogStepListenerdetects that a pipeline step has started executing, it checks the queue time data - if the data was not filled and the pipeline is executing already, it means that the pipeline missed the build queue. For such pipelines the queue time is set to 0.Description of the Change
Alternate Designs
Possible Drawbacks
Verification Process
Additional Notes
Release Notes
Review checklist (to be filled by reviewers)
changelog/label attached. If applicable it should have thebackward-incompatiblelabel attached.do-not-merge/label attached.kind/andseverity/labels attached at least.