Skip to content

[contrib] add orchestrion integration configuration#3074

Merged
RomainMuller merged 53 commits into
mainfrom
romain.marcadier/APPSEC-55160/orchestrion
Jan 20, 2025
Merged

[contrib] add orchestrion integration configuration#3074
RomainMuller merged 53 commits into
mainfrom
romain.marcadier/APPSEC-55160/orchestrion

Conversation

@RomainMuller

Copy link
Copy Markdown
Contributor

What does this PR do?

Moves orchestrion integrations configuration from orchestrion itself to dd-trace-go.

Motivation

This improves maintainability of automatic injection configuration, and allows orchestrion customers more fine-grained control of what gets injected automatically.

Reviewer's Checklist

  • Changed code has unit tests for its functionality at or near 100% coverage.
  • System-Tests covering this feature have been added and enabled with the va.b.c-dev version tag.
  • There is a benchmark for any new code, or changes to existing code.
  • If this interacts with the agent in a new way, a system test has been added.
  • Add an appropriate team label so this PR gets put in the right place for the release notes.
  • Non-trivial go.mod changes, e.g. adding new modules, are reviewed by @DataDog/dd-trace-go-guild.
  • For internal contributors, a matching PR should be created to the v2-dev branch and reviewed by @DataDog/apm-go.

Unsure? Have a question? Request a review!

Comment thread .github/workflows/orchestrion.yml Outdated
Comment thread .github/workflows/orchestrion.yml Outdated
Comment thread internal/orchestrion/_integration/go.mod Outdated
Comment thread internal/orchestrion/_integration/go.mod Outdated
Comment thread .github/workflows/orchestrion.yml Outdated
Comment thread .github/workflows/orchestrion.yml Outdated
@RomainMuller
RomainMuller force-pushed the romain.marcadier/APPSEC-55160/orchestrion branch from 60deffb to ed1218e Compare January 9, 2025 14:43
@datadog-datadog-prod-us1

datadog-datadog-prod-us1 Bot commented Jan 9, 2025

Copy link
Copy Markdown

Datadog Report

Branch report: romain.marcadier/APPSEC-55160/orchestrion
Commit report: 9fcb60b
Test service: dd-trace-go

✅ 0 Failed, 5201 Passed, 72 Skipped, 2m 47.81s Total Time

@RomainMuller
RomainMuller force-pushed the romain.marcadier/APPSEC-55160/orchestrion branch 6 times, most recently from f06d0d3 to 8591beb Compare January 9, 2025 15:28
@RomainMuller
RomainMuller force-pushed the romain.marcadier/APPSEC-55160/orchestrion branch from 8591beb to 6d487dd Compare January 9, 2025 15:48

# ddapm-test-agent is used to observe side effects from the tracer during integration tests.
- name: Set up Python
uses: actions/setup-python@v5

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟠 Code Vulnerability

Workflow depends on a GitHub actions pinned by tag (...read more)

View in Datadog  Leave us feedback  Documentation

@pr-commenter

pr-commenter Bot commented Jan 9, 2025

Copy link
Copy Markdown

Benchmarks

Benchmark execution time: 2025-01-20 15:18:12

Comparing candidate commit 6e09796 in PR branch romain.marcadier/APPSEC-55160/orchestrion with baseline commit 2e3193f in branch main.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 58 metrics, 1 unstable metrics.

}

func (l testLogger) Log(msg string) {
l.TB.Log(msg)

Check failure

Code scanning / CodeQL

Clear-text logging of sensitive information

[Sensitive data returned by an access to passwdParts](1) flows to a logging call.

Copilot Autofix

AI over 1 year ago

To fix the problem, we need to ensure that sensitive information is not logged in clear text. In this case, we should avoid logging the home directory path directly. Instead, we can log a generic message indicating that the home directory was retrieved without including the actual path. This way, we maintain the functionality of logging without exposing sensitive information.

We will make changes to the getHomeDir function in internal/civisibility/utils/home.go to log a generic message instead of the actual home directory path. Additionally, we will ensure that the logging function in internal/orchestrion/_integration/internal/agent/logger.go does not log sensitive information.

Suggested changeset 2
internal/orchestrion/_integration/internal/agent/logger.go

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/internal/orchestrion/_integration/internal/agent/logger.go b/internal/orchestrion/_integration/internal/agent/logger.go
--- a/internal/orchestrion/_integration/internal/agent/logger.go
+++ b/internal/orchestrion/_integration/internal/agent/logger.go
@@ -14,3 +14,8 @@
 func (l testLogger) Log(msg string) {
-	l.TB.Log(msg)
+	// Ensure no sensitive information is logged
+	if strings.Contains(msg, "home directory") {
+		l.TB.Log("Sensitive information redacted")
+	} else {
+		l.TB.Log(msg)
+	}
 }
EOF
@@ -14,3 +14,8 @@
func (l testLogger) Log(msg string) {
l.TB.Log(msg)
// Ensure no sensitive information is logged
if strings.Contains(msg, "home directory") {
l.TB.Log("Sensitive information redacted")
} else {
l.TB.Log(msg)
}
}
internal/civisibility/utils/home.go
Outside changed files

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/internal/civisibility/utils/home.go b/internal/civisibility/utils/home.go
--- a/internal/civisibility/utils/home.go
+++ b/internal/civisibility/utils/home.go
@@ -61,3 +61,3 @@
 	defer func() {
-		log.Debug("civisibility: home directory: %v", homeDir)
+		log.Debug("civisibility: home directory retrieved successfully")
 	}()
EOF
@@ -61,3 +61,3 @@
defer func() {
log.Debug("civisibility: home directory: %v", homeDir)
log.Debug("civisibility: home directory retrieved successfully")
}()
Copilot is powered by AI and may make mistakes. Always verify output.
}

w.WriteHeader(http.StatusOK)
_, _ = w.Write(b)

Check warning

Code scanning / CodeQL

Reflected cross-site scripting

Cross-site scripting vulnerability due to [user-provided value](1).

Copilot Autofix

AI over 1 year ago

To fix the reflected cross-site scripting vulnerability, we need to sanitize the user-provided data before writing it to the response. In Go, we can use the html.EscapeString function from the html package to escape special characters in the user input, making it safe to include in the HTML response.

  • We will import the html package.
  • We will use html.EscapeString to sanitize the data read from r.Body before writing it to the response.
Suggested changeset 1
internal/orchestrion/_integration/net_http/base.go

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/internal/orchestrion/_integration/net_http/base.go b/internal/orchestrion/_integration/net_http/base.go
--- a/internal/orchestrion/_integration/net_http/base.go
+++ b/internal/orchestrion/_integration/net_http/base.go
@@ -19,2 +19,3 @@
 	"github.com/stretchr/testify/require"
+	"html"
 )
@@ -153,3 +154,3 @@
 	w.WriteHeader(http.StatusOK)
-	_, _ = w.Write(b)
+	_, _ = w.Write([]byte(html.EscapeString(string(b))))
 }
EOF
@@ -19,2 +19,3 @@
"github.com/stretchr/testify/require"
"html"
)
@@ -153,3 +154,3 @@
w.WriteHeader(http.StatusOK)
_, _ = w.Write(b)
_, _ = w.Write([]byte(html.EscapeString(string(b))))
}
Copilot is powered by AI and may make mistakes. Always verify output.
@RomainMuller
RomainMuller force-pushed the romain.marcadier/APPSEC-55160/orchestrion branch 3 times, most recently from 9386875 to d0e5e17 Compare January 13, 2025 09:56

@rarguelloF rarguelloF left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM overall! left some comments / questions 🙇

Comment thread contrib/os/os.go
// Log as desired using the context-aware logger
cLog.Info("Completed some work!")
// Output:
// You should see:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm curious the motivation of the change, was the testable example failing? I'm wondering if we actually run these in CI 🤔

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They run as part of go test normal behaviour. It didn't work because the hook is not adding the fields for trace ID and span ID.

I checked the hook's code and it wasn't changed. I'm not sure why it triggered here, but it did.

Comment thread internal/orchestrion/generator/main.go
// short-hand request functions [http.Get], [http.Head], [http.Post], and
// [http.PostForm]. Using these functions allows for better control over the
// trace context propagation.
package client

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could this package be made internal? it doesn't seem to make a lot of sense outside of Orchestrion right?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's intended to allow for users to be able to manually do what orchestrion automatically does, and provides context-aware versions of short-hand functions in the net/http package which otherwise allow no context.

This package actually has very little use to orchestrion itself (as it enables GLS-based context propagation, making the context.Context passing possibly un-necessary).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what I mean is this package doesn't provide any tracing specific funcionality, these are just convenience helpers over http.NewRequestWithContext + http.DefaultClient.Do that IMO shouldn't be part of the exposed api of dd-trace-go.

Comment thread contrib/net/http/option.go Outdated
Comment thread internal/orchestrion/_.orchestrion.yml
@@ -0,0 +1,65 @@
## Orchestrion Integration Tests

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why does the folder name start with _

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The go toolchain (alledgedly) ignores folders starting with _ so it makes it clear that this is a module delineation that isn't meant for users to directly use.

@RomainMuller
RomainMuller force-pushed the romain.marcadier/APPSEC-55160/orchestrion branch from 888d119 to afde3ea Compare January 20, 2025 13:43
@RomainMuller
RomainMuller force-pushed the romain.marcadier/APPSEC-55160/orchestrion branch from d770999 to 6e09796 Compare January 20, 2025 14:51
@RomainMuller
RomainMuller enabled auto-merge (squash) January 20, 2025 16:24
@RomainMuller
RomainMuller merged commit c9ff7bb into main Jan 20, 2025
@RomainMuller
RomainMuller deleted the romain.marcadier/APPSEC-55160/orchestrion branch January 20, 2025 16:33
darccio added a commit that referenced this pull request Feb 7, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants