Skip to content

Implement automatic CI Visibility configuration for Maven projects#5043

Merged
nikita-tkachenko-datadog merged 13 commits into
masterfrom
nikita-tkachenko/maven-auto-configuration
Apr 28, 2023
Merged

Implement automatic CI Visibility configuration for Maven projects#5043
nikita-tkachenko-datadog merged 13 commits into
masterfrom
nikita-tkachenko/maven-auto-configuration

Conversation

@nikita-tkachenko-datadog

@nikita-tkachenko-datadog nikita-tkachenko-datadog commented Apr 10, 2023

Copy link
Copy Markdown
Contributor

What Does This Do

Updates Maven instrumentation so that CI Visibility configuration is done automatically:

  • when parent build process is run with the tracer attached, forked Surefire/Failsafe processes are automatically configured to run with the same tracer
  • when parent build process is run with the tracer attached, forked Javac processes are automatically configured to run with the DD Javac Plugin

Motivation

Simplifying onboarding for CI Visibility customers

@nikita-tkachenko-datadog nikita-tkachenko-datadog changed the title Nikita tkachenko/maven auto configuration WIP: Implement automatic CI Visibility configuration for Maven projects Apr 10, 2023
@nikita-tkachenko-datadog nikita-tkachenko-datadog added the comp: ci visibility Continuous Integration Visibility label Apr 10, 2023
@nikita-tkachenko-datadog
nikita-tkachenko-datadog force-pushed the nikita-tkachenko/maven-auto-configuration branch from 15d4369 to a38f28c Compare April 27, 2023 09:45
@pr-commenter

pr-commenter Bot commented Apr 27, 2023

Copy link
Copy Markdown

Benchmarks

Parameters

Baseline Candidate
commit 1.13.0-SNAPSHOT~bc3587bdbc 1.13.0-SNAPSHOT~972beee8a5
config baseline candidate
See matching parameters
Baseline Candidate
module Agent Agent
parent None None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 22 cases.

See unchanged results
scenario Δ mean execution_time
scenario:Startup-base-Agent unsure
[+0.023s; +0.118s] or [+0.250%; +1.286%]
scenario:Startup-base-Agent.start same
scenario:Startup-base-BytebuddyAgent same
scenario:Startup-base-GlobalTracer same
scenario:Startup-base-AppSec same
scenario:Startup-base-Remote Config same
scenario:Startup-base-Telemetry same
scenario:Startup-iast-Agent same
scenario:Startup-iast-Agent.start same
scenario:Startup-iast-BytebuddyAgent same
scenario:Startup-iast-GlobalTracer same
scenario:Startup-iast-AppSec same
scenario:Startup-iast-IAST same
scenario:Startup-iast-Remote Config same
scenario:Startup-iast-Telemetry same
scenario:Startup-waf-Agent same
scenario:Startup-waf-Agent.start same
scenario:Startup-waf-BytebuddyAgent same
scenario:Startup-waf-GlobalTracer same
scenario:Startup-waf-AppSec same
scenario:Startup-waf-Remote Config same
scenario:Startup-waf-Telemetry same

@nikita-tkachenko-datadog
nikita-tkachenko-datadog marked this pull request as ready for review April 27, 2023 13:08
@nikita-tkachenko-datadog nikita-tkachenko-datadog changed the title WIP: Implement automatic CI Visibility configuration for Maven projects Implement automatic CI Visibility configuration for Maven projects Apr 27, 2023

@albertvaka albertvaka left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there some automated testing we can do here?

Comment thread internal-api/src/main/java/datadog/trace/util/AgentUtils.java Outdated
: MavenPluginVersion.UNKNOWN;

if (mavenPluginVersion.isLaterThanOrEqualTo(ANNOTATION_PROCESSOR_PATHS_SUPPORTED_VERSION)) {
String lombokVersion = getLombokVersion(projectDependencies);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the need for lombok on older Maven versions documented anywhere?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not like we're adding Lombok here, this is about ensuring Lombok does not break if the project is already using it.

In newer Maven compiler plugin versions - the ones that support annotation processor paths - whenever we're adding DD Javac Plugin as annotation processor, Lombok (if it happens to be used in the project) stops working, unless we add it as an annotation processor as well.

This is an edge case that is documented in DD Javac Plugin documentation

.append(propertyName)
.append('=')
.append(e.getValue())
.append(System.lineSeparator());

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we split the arguments by newlines? Is that supported in all OSes? Shouldn't they be just split by spaces?

@nikita-tkachenko-datadog nikita-tkachenko-datadog Apr 27, 2023

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now I have doubts :)

I always used to use newlines when configuring Maven manually, and the surefire plugin code contains logic that replaces all the whitespace character with the literal space.

But now I'm not sure about the older plugin versions, and the documentation says nothing about it.

Replaced it with space to be on the safe side.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are a few more places where System.lineSeparator() is used. If it works we can leave it, but let's be consistent and use the same thing everywhere.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, I missed those. Changed everything to using spaces now

}

File agentJar = AgentUtils.getAgentJar();
modifiedArgLine.append("-javaagent:").append(agentJar.toPath());

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happens if you have already configured this manually & we try to auto-configure it on top of that? You might end with two -javaagent arguments. Did you test that scenario? Same for the compiler plugin.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I have verified it, since I assumed there can be customers who might just update to the new tracer versions without reading the docs.

The tracer contains a safeguard that protects it from initialising twice, and the compiler plugin does not break as well.

@nikita-tkachenko-datadog

Copy link
Copy Markdown
Contributor Author

Is there some automated testing we can do here?

Unit testing seems kind of pointless to me, as verifying that a number of methods was executed on a Maven domain object does not give much confidence in the end result.
Integration tests we cannot use, as autoconfig requires an actual agent distribution JAR to be present on the classpath.
It has to be either tracer smoke tests or our test environment.
I chose the latter since it was easier to implement.

@albertvaka albertvaka left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left one last comment about System.lineSeparator(), other than that LGTM 🚀

@nikita-tkachenko-datadog
nikita-tkachenko-datadog merged commit fe6c747 into master Apr 28, 2023
@nikita-tkachenko-datadog
nikita-tkachenko-datadog deleted the nikita-tkachenko/maven-auto-configuration branch April 28, 2023 08:49
@github-actions github-actions Bot added this to the 1.13.0 milestone Apr 28, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp: ci visibility Continuous Integration Visibility

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants