Skip to content

CI Visibility support in Agentless mode#309

Merged
albertvaka merged 11 commits into
masterfrom
albertvaka/civis-agentless
Oct 27, 2022
Merged

CI Visibility support in Agentless mode#309
albertvaka merged 11 commits into
masterfrom
albertvaka/civis-agentless

Conversation

@albertvaka

@albertvaka albertvaka commented Sep 30, 2022

Copy link
Copy Markdown
Collaborator

What does this PR do?

CI Visibility was only available when running with an Agent, but not for Agentless. This allows CI Visibility to be enabled in Agentless mode.

Description of the Change

Implements the startBuildTrace, finishBuildTrace and sendPipelineTrace in DatadogHttpClient which previously were only implemented in the DatadogAgentClient.

For Agentless reporting we use CI Visibility Webhooks API, which unlike the Agent API doesn't receive traces. Because of this I had to add DatadogWebhookBuildLogic and DatadogWebhookPipelineLogic to generate payloads for the Webhooks API, equivalent to the existing DatadogBuildPipelineLogic and DatadogTracePipelineLogic that generate spans.

Known issues

These two "features" were working when using APM through the Agent and won't work in Agentless since the Webhooks API still doesn't support them:

  • Nested stages (stages that have another stage as their parent) (fixed)
  • Stage breakdown graph in Datadog

Verification Process

Run different kinds of jobs and pipelines with the two modes and compare the data on the Datadog app.

Review checklist (to be filled by reviewers)

  • Feature or bug fix MUST have appropriate tests (unit, integration, etc...)
  • PR title must be written as a CHANGELOG entry (see why)
  • Files changes must correspond to the primary purpose of the PR as described in the title (small unrelated changes should have their own PR)
  • PR must have one changelog/ label attached. If applicable it should have the backward-incompatible label attached.
  • PR should not have do-not-merge/ label attached.
  • If Applicable, issue must have kind/ and severity/ labels attached at least.

@albertvaka
albertvaka force-pushed the albertvaka/civis-agentless branch from 09e044d to 7b5c5b7 Compare September 30, 2022 13:49

@drodriguezhdez drodriguezhdez left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I dropped a couple of comments. Probably some refactor may be worth it. There are several classes that look pretty similar to each other.

Comment thread src/main/java/org/datadog/jenkins/plugins/datadog/DatadogGlobalConfiguration.java Outdated
* @return a boolean to signify the success or failure of the HTTP POST request.
*/
@SuppressFBWarnings("REC_CATCH_EXCEPTION")
public boolean postWebhook(String payload) {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This method looks pretty similar to postLogs and postApi. Maybe a refactor could be worth it?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried but there are subtle differences. We might be able to get rid of them, but I don't want to risk breaking something so I would do it on a separate PR.

@albertvaka
albertvaka marked this pull request as ready for review October 7, 2022 10:38

@sarah-witt sarah-witt left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great 🎉 Left a few nits! Can you also update the README.md describing the new environment variables/settings, as well as make a note that CI visibility is available for both methods?

Comment thread src/main/java/org/datadog/jenkins/plugins/datadog/DatadogUtilities.java Outdated
Comment thread src/main/java/org/datadog/jenkins/plugins/datadog/clients/DatadogHttpClient.java Outdated
Comment thread src/main/java/org/datadog/jenkins/plugins/datadog/clients/DatadogHttpClient.java Outdated
Comment thread src/main/java/org/datadog/jenkins/plugins/datadog/clients/DatadogHttpClient.java Outdated
Comment thread src/main/java/org/datadog/jenkins/plugins/datadog/traces/BuildWebhookAction.java Outdated
}
}

// If there is no labels and the node name is master,

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Making a note that in the future we will want to change this to "built-in" since that is what jenkins refers to the main node now.

@albertvaka
albertvaka force-pushed the albertvaka/civis-agentless branch from a7201b0 to 1fa4d07 Compare October 14, 2022 13:09
@github-actions github-actions Bot added the documentation Documentation related changes label Oct 14, 2022
@albertvaka
albertvaka force-pushed the albertvaka/civis-agentless branch from f25b24f to 29fd379 Compare October 26, 2022 13:24

@sarah-witt sarah-witt left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a few more small comments, otherwise looks good!

HttpURLConnection conn = null;
try {
logger.fine("Setting up HttpURLConnection...");
String urlParameters = "?service=" + DatadogUtilities.getDatadogGlobalDescriptor().getCiInstanceName();

@sarah-witt sarah-witt Oct 26, 2022

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For testing purposes (and abstracting code) what about making this a function createRequest or something and verifying the url and headers are valid?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've refactored the duplicated code into a separate method in 54c3da0

Comment thread src/main/java/org/datadog/jenkins/plugins/datadog/clients/DatadogHttpClient.java Outdated
albertvaka and others added 2 commits October 27, 2022 13:34
Co-authored-by: Sarah Witt <[email protected]>
For postApi this means some changes:
- The API key is not passed as a header instead of a URL param
- We now set the User Agent.
@albertvaka
albertvaka force-pushed the albertvaka/civis-agentless branch from 3249ae1 to 54c3da0 Compare October 27, 2022 11:59
@albertvaka
albertvaka merged commit e7e39d2 into master Oct 27, 2022
@albertvaka
albertvaka deleted the albertvaka/civis-agentless branch October 27, 2022 13:26
@sarah-witt sarah-witt added the changelog/Added Added features results into a minor version bump label Oct 28, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/Added Added features results into a minor version bump documentation Documentation related changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants