Skip to content

APM Traces for Jenkins Build/Pipelines#96

Merged
drodriguezhdez merged 27 commits into
masterfrom
drodriguezhdez/apm_traces
Aug 20, 2020
Merged

APM Traces for Jenkins Build/Pipelines#96
drodriguezhdez merged 27 commits into
masterfrom
drodriguezhdez/apm_traces

Conversation

@drodriguezhdez

@drodriguezhdez drodriguezhdez commented Jul 22, 2020

Copy link
Copy Markdown
Collaborator

DISCLAIMER

This PR adds an experimental feature in alpha state.
Internal use only.

Requirements for Contributing to this repository

  • Fill out the template below. Any pull request that does not include enough information to be reviewed in a timely manner may be closed at the maintainers' discretion.
  • The pull request must only fix one issue at the time.
  • The pull request must update the test suite to demonstrate the changed functionality.
  • After you create the pull request, all status checks must be pass before a maintainer reviews your contribution. For more details, please see CONTRIBUTING.

What does this PR do?

  • Added the logic to send APM traces of Jenkins Builds/Pipelines using Datadog Agent.

Description of the Change

The DatadogTraceBuildLogic class implements the logic of sending the build traces. During the execution of the onStarted method, a new span that represents the Jenkins Build is created. This span is finished in the onComplete method. The BuildSpanManager class is used to propagate the same span between A method and the B method. This is required because it's not possible to store the complete active Span in a Jenkins Action. Additionally, the SpanContext of the Jenkins Build span is injected in the BuildSpanAction. This logic is invoked from the DatadogBuildListener class.

The DatadogTracePipelineLogic class implements the logic of sending the pipeline traces. This logic is only executed in the las FlowNode of the pipeline. The idea is, based on that last node, to recreate the full pipeline tree with the complete information of each node, and then, to send the spans which represent that tree. The BuildPipeline and BuildPipelineNode classes have been implemented to build the Jenkins Pipeline nodes tree. To correlate the Jenkins Build with the pipeline, the SpanContext is extracted from the BuildSpanAction. This logic is invoked from the DatadogGraphListener class.

The DatadogStepListener class implements the logic of extract the important information of each StepContext, and store them in the StepDataManager using the StepDescriptor as key. This information will be available during the construction of the node spans in the DatadogTracePipelineLogic execution.

To create and send the traces is used the APM Java Tracer API.
This logic is only available for the Datadog Agent mode in the Jenkins Plugin.

Finally, the DatadogGlobalConfiguration class and the UI template has been modified to support the configuration of the Trace Collection Port input and Enable Trace Collection flag.

Alternate Designs

Possible Drawbacks

Verification Process

Apart from the addition of the unit and integration tests to this PR, the Jenkins plugin with this feature is available in the Jenkins instance of the CI-App team.

Check the APM Traces view in Datadog corresponding to this Jenkins pipeline execution.

Additional Notes

You can find further information of this feature in the CI/CD observability technical design document.

Finally, do not hesitate to join to #ci-app Slack channel if you have more questions.

Release Notes

Review checklist (to be filled by reviewers)

  • Feature or bug fix MUST have appropriate tests (unit, integration, etc...)
  • PR title must be written as a CHANGELOG entry (see why)
  • Files changes must correspond to the primary purpose of the PR as described in the title (small unrelated changes should have their own PR)
  • PR must have one changelog/ label attached. If applicable it should have the backward-incompatible label attached.
  • PR should not have do-not-merge/ label attached.
  • If Applicable, issue must have kind/ and severity/ labels attached at least.

@drodriguezhdez drodriguezhdez self-assigned this Jul 22, 2020
@drodriguezhdez
drodriguezhdez force-pushed the drodriguezhdez/apm_traces branch from e0a53df to f1f5b4c Compare July 22, 2020 12:43
@drodriguezhdez
drodriguezhdez force-pushed the drodriguezhdez/apm_traces branch from f1f5b4c to 2628364 Compare July 22, 2020 13:13
@drodriguezhdez
drodriguezhdez force-pushed the drodriguezhdez/apm_traces branch from 513ccd1 to 74e1130 Compare July 29, 2020 15:00
@drodriguezhdez drodriguezhdez changed the title APM traces POC APM trace Jul 30, 2020
@drodriguezhdez
drodriguezhdez force-pushed the drodriguezhdez/apm_traces branch 5 times, most recently from b42049f to 8288376 Compare July 31, 2020 10:18
@drodriguezhdez drodriguezhdez changed the title APM trace Ci-App: APM Traces of Jenkins Build/Pipelines Jul 31, 2020
@drodriguezhdez
drodriguezhdez force-pushed the drodriguezhdez/apm_traces branch from 8288376 to 93789fd Compare July 31, 2020 11:05
@drodriguezhdez drodriguezhdez changed the title Ci-App: APM Traces of Jenkins Build/Pipelines APM Traces for Jenkins Build/Pipelines Jul 31, 2020
@drodriguezhdez drodriguezhdez added the changelog/Added Added features results into a minor version bump label Jul 31, 2020
@drodriguezhdez
drodriguezhdez marked this pull request as ready for review July 31, 2020 13:58

@sarah-witt sarah-witt left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall this looks great from the jenkins side! We'll need to update the development docs and regular docs too.

Comment thread src/test/java/org/datadog/jenkins/plugins/datadog/model/BuildPipelineTest.java Outdated
Comment thread src/main/java/org/datadog/jenkins/plugins/datadog/clients/DogStatsDClient.java Outdated
Comment thread src/main/java/org/datadog/jenkins/plugins/datadog/model/BuildPipeline.java Outdated
Comment thread src/main/java/org/datadog/jenkins/plugins/datadog/DatadogGlobalConfiguration.java Outdated
Comment thread src/main/java/org/datadog/jenkins/plugins/datadog/DatadogUtilities.java Outdated
Comment thread src/main/java/org/datadog/jenkins/plugins/datadog/clients/DatadogHttpClient.java Outdated
Comment thread src/main/java/org/datadog/jenkins/plugins/datadog/clients/DogStatsDClient.java Outdated
Comment thread src/main/java/org/datadog/jenkins/plugins/datadog/traces/BuildSpanAction.java Outdated
@github-actions github-actions Bot added the documentation Documentation related changes label Aug 17, 2020
@drodriguezhdez

Copy link
Copy Markdown
Collaborator Author

Updated DEVELOPMENT and README documentation.

FlorianVeaux
FlorianVeaux previously approved these changes Aug 19, 2020

@FlorianVeaux FlorianVeaux left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great 💯

@drodriguezhdez

Copy link
Copy Markdown
Collaborator Author

@sarah-witt did you have the opportunity of taking a look at my comments?

@drodriguezhdez
drodriguezhdez force-pushed the drodriguezhdez/apm_traces branch from 5c82b01 to ef98fe6 Compare August 20, 2020 13:20
@drodriguezhdez
drodriguezhdez merged commit 7f81d15 into master Aug 20, 2020
@drodriguezhdez
drodriguezhdez deleted the drodriguezhdez/apm_traces branch August 20, 2020 13:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/Added Added features results into a minor version bump documentation Documentation related changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants