[LANGPLAT-437] Improve CI and dogfooding practices for `dd-trace-go` by kakkoyun · Pull Request #3727 · DataDog/dd-trace-go

kakkoyun · 2025-07-04T14:42:01Z

What does this PR do?

This PR introduces comprehensive development tooling and CI improvements to enhance code quality, consistency, and developer experience across the dd-trace-go repository.

Key changes include:

New CI Workflows: Added static-checks and generate workflows for comprehensive code validation
Development Scripts: Added lint.sh, format.sh, checklocks.sh, and enhanced existing scripts
Tooling Updates: Added golangci-lint v2.2.1 and shfmt to project dependencies
Documentation: Comprehensive updates to CONTRIBUTING.md with detailed development guides
CI Streamlining: Removed duplicate jobs and improved workflow efficiency

Motivation

https://datadoghq.atlassian.net/browse/LANGPLAT-437

The current development workflow lacks consistent tooling and comprehensive static analysis, leading to:

Inconsistent code formatting across the codebase
Manual linting processes that are error-prone
Lack of lock analysis for potential deadlocks
Fragmented CI workflows with duplicate jobs
Insufficient documentation for contributors

This PR addresses these issues by providing:

Unified development scripts for consistent local and CI environments
Comprehensive static analysis including deadlock detection
Improved CI efficiency with dedicated workflows
Better documentation to help contributors understand the development process

Reviewer's Checklist

Changed code has unit tests for its functionality at or near 100% coverage. (N/A - tooling changes)
System-Tests covering this feature have been added and enabled with the va.b.c-dev version tag. (N/A - tooling changes)
There is a benchmark for any new code, or changes to existing code. (N/A - tooling changes)
If this interacts with the agent in a new way, a system test has been added. (N/A - no agent interaction)
New code is free of linting errors. You can check this by running `golangci-lint run` locally. (All changes pass linting)
Add an appropriate team label so this PR gets put in the right place for the release notes. (Will add `build` label)
Non-trivial go.mod changes, e.g. adding new modules, are reviewed by @DataDog/dd-trace-go-guild. (Added golangci-lint and shfmt to _tools/go.mod)

Add comprehensive development scripts: - lint.sh: Run all linters (goimports, golangci-lint, checklocks) - format.sh: Format Go and shell files - checklocks.sh: Lock analysis for deadlock detection - fix-modules.sh: Module consistency management (replaces fix_modules.sh) Enhance existing scripts: - test.sh: Improved output formatting and error handling - ci-test-contrib.sh: Better CI integration These scripts provide consistent tooling for local development and CI environments. Signed-off-by: Kemal Akkoyun <[email protected]>

Add comprehensive static-checks workflow including: - Copyright header verification - Code generation validation - Module consistency checks - Comprehensive linting with golangci-lint - Lock analysis with checklocks Add dedicated generate workflow for code generation tasks. Streamline unit-integration-tests workflow by removing duplicate jobs that are now handled by static-checks. Fix script path references in smoke-tests workflow (fix_modules.sh -> fix-modules.sh). Signed-off-by: Kemal Akkoyun <[email protected]>

Add golangci-lint/v2 and shfmt to project tools for: - Comprehensive Go linting across the codebase - Shell script formatting and validation Update go.mod and go.sum with new dependencies. Update go.work.sum with workspace changes. Signed-off-by: Kemal Akkoyun <[email protected]>

CONTRIBUTING.md: - Add comprehensive CI workflow documentation - Document new development scripts (lint.sh, format.sh, etc.) - Add troubleshooting guides for CI failures - Improve formatting and structure - Update linting instructions with new tools README.md: - Fix formatting inconsistencies - Improve bullet point alignment - Clean up whitespace and formatting Signed-off-by: Kemal Akkoyun <[email protected]>

Update golangci-lint version in pre-commit config to v2.2.1 and add sync reminder comment. Add tmp directory to .gitignore for temporary development files. Signed-off-by: Kemal Akkoyun <[email protected]>

Fix line continuation formatting in dsn.go and remove unnecessary blank identifier in fixmodules/main.go. These are automated formatting changes from gofmt/goimports. Signed-off-by: Kemal Akkoyun <[email protected]>

Signed-off-by: Kemal Akkoyun <[email protected]>

…dor) Signed-off-by: Kemal Akkoyun <[email protected]>

darccio

LGTM

kakkoyun · 2025-07-17T10:29:14Z

lint and copyright required tests are moved thus they are not passing right now. I will change the required steps after merge.

kakkoyun · 2025-07-17T10:30:07Z

/merge --cancel

dd-devflow-routing-codex · 2025-07-17T10:30:11Z

View all feedbacks in Devflow UI.

2025-07-17 10:30:11 UTC ℹ️ Start processing command /merge --cancel

rarguelloF

Nice! 🚀

kakkoyun · 2025-07-17T10:38:49Z

/merge

dd-devflow-routing-codex · 2025-07-17T10:38:53Z

View all feedbacks in Devflow UI.

2025-07-17 10:38:52 UTC ℹ️ Start processing command /merge

2025-07-17 10:39:01 UTC ℹ️ MergeQueue: waiting for PR to be ready

This merge request is not mergeable yet, because of pending checks/missing approvals. It will be added to the queue as soon as checks pass and/or get approvals.
Note: if you pushed new commits since the last approval, you may need additional approval.
You can remove it from the waiting list with /remove command.

2025-07-17 10:50:40 UTC ⚠️ MergeQueue: This merge request was unqueued

[email protected] unqueued this merge request

kakkoyun · 2025-07-17T10:50:31Z

/remove

dd-devflow-routing-codex · 2025-07-17T10:50:34Z

View all feedbacks in Devflow UI.

2025-07-17 10:50:33 UTC ℹ️ Start processing command /remove

2025-07-17 10:50:37 UTC ℹ️ Devflow: /remove

Signed-off-by: Eliott Bouhana <[email protected]>

…ace-go` (#3727)" This reverts commit ac97b54.

Signed-off-by: Eliott Bouhana <[email protected]>

Add the reverted chages back from "Improve CI and dogfooding practices for dd-trace-go #3727" And also; - Add embedmd to development tools for automatic documentation embedding - Add 'make docs' command to update README files with current help outputs - Add CI check to ensure documentation stays up-to-date - Generate clean, idempotent help outputs for consistent documentation Signed-off-by: Kemal Akkoyun <[email protected]>

kakkoyun requested review from a team as code owners July 4, 2025 14:42

kakkoyun changed the title ~~build: add comprehensive development tooling and CI improvements~~ build/ci: Add comprehensive development tooling and CI improvements Jul 4, 2025

kakkoyun force-pushed the kakkoyun/add-checklocks branch 2 times, most recently from 7ec201a to 27fb69a Compare July 4, 2025 14:57

kakkoyun changed the title ~~build/ci: Add comprehensive development tooling and CI improvements~~ [LANGPLAT-437] Improve CI and dogfooding practices for dd-trace-go Jul 4, 2025

kakkoyun force-pushed the kakkoyun/add-checklocks branch from d99138f to fc54d77 Compare July 4, 2025 15:20

kakkoyun marked this pull request as draft July 4, 2025 15:22

kakkoyun force-pushed the kakkoyun/add-checklocks branch from fc54d77 to ba2e5eb Compare July 4, 2025 15:28

kakkoyun force-pushed the kakkoyun/add-checklocks branch 3 times, most recently from c8f8d7b to bb4860f Compare July 16, 2025 09:27

kakkoyun marked this pull request as ready for review July 16, 2025 09:29

kakkoyun force-pushed the kakkoyun/add-checklocks branch from bb4860f to f9b32bb Compare July 16, 2025 09:38

kakkoyun added 8 commits July 16, 2025 11:39

chore: update development configuration

98c1997

Update golangci-lint version in pre-commit config to v2.2.1 and add sync reminder comment. Add tmp directory to .gitignore for temporary development files. Signed-off-by: Kemal Akkoyun <[email protected]>

style: format Go code with gofmt

b67b92c

Fix line continuation formatting in dsn.go and remove unnecessary blank identifier in fixmodules/main.go. These are automated formatting changes from gofmt/goimports. Signed-off-by: Kemal Akkoyun <[email protected]>

Fix several issues

0e8518a

Signed-off-by: Kemal Akkoyun <[email protected]>

Update to 1.23.1

1508d80

Signed-off-by: Kemal Akkoyun <[email protected]>

kakkoyun force-pushed the kakkoyun/add-checklocks branch from f9b32bb to 1508d80 Compare July 16, 2025 09:40

Fix modules

b86eb16

Signed-off-by: Kemal Akkoyun <[email protected]>

kakkoyun force-pushed the kakkoyun/add-checklocks branch from ccf7bac to b86eb16 Compare July 16, 2025 09:50

Use forked dependency directly instead of replace (breaks go work ven…

b3602a8

…dor) Signed-off-by: Kemal Akkoyun <[email protected]>

kakkoyun force-pushed the kakkoyun/add-checklocks branch from feced67 to b3602a8 Compare July 16, 2025 09:59

darccio approved these changes Jul 16, 2025

View reviewed changes

kakkoyun force-pushed the kakkoyun/add-checklocks branch from deecbaa to aa5854a Compare July 16, 2025 12:12

dd-devflow Bot added the mergequeue-status: waiting label Jul 17, 2025

kakkoyun enabled auto-merge (squash) July 17, 2025 10:29

kakkoyun disabled auto-merge July 17, 2025 10:30

dd-devflow Bot added mergequeue-status: removed and removed mergequeue-status: waiting labels Jul 17, 2025

rarguelloF approved these changes Jul 17, 2025

View reviewed changes

dd-devflow Bot added mergequeue-status: waiting and removed mergequeue-status: removed labels Jul 17, 2025

kakkoyun merged commit ac97b54 into DataDog:main Jul 17, 2025
131 of 132 checks passed

dd-devflow Bot added mergequeue-status: removed and removed mergequeue-status: waiting labels Jul 17, 2025

kakkoyun deleted the kakkoyun/add-checklocks branch July 17, 2025 10:50

kakkoyun mentioned this pull request Jul 17, 2025

chore(ci): Update STS policy for reviewdog #3777

Merged

7 tasks

KrisKennawayDD mentioned this pull request Jul 17, 2025

[VULN-11260-11263] Fix Go Chi v5 host header injection vulnerability #3744

Closed

eliottness added a commit that referenced this pull request Jul 23, 2025

revert #3727

6730ad2

Signed-off-by: Eliott Bouhana <[email protected]>

eliottness mentioned this pull request Jul 24, 2025

revert #3727 #3807

Merged

7 tasks

eliottness added a commit that referenced this pull request Jul 24, 2025

Revert "[LANGPLAT-437] Improve CI and dogfooding practices for `dd-tr…

9f50384

…ace-go` (#3727)" This reverts commit ac97b54.

eliottness added a commit that referenced this pull request Jul 24, 2025

Revert "[LANGPLAT-437] Improve CI and dogfooding practices for `dd-tr…

31002d7

…ace-go` (#3727)" This reverts commit ac97b54.

eliottness added a commit that referenced this pull request Jul 24, 2025

revert #3727

1b3155f

Signed-off-by: Eliott Bouhana <[email protected]>

eliottness added a commit that referenced this pull request Jul 25, 2025

revert #3727 (#3807)

cc103fb

kakkoyun mentioned this pull request Jul 31, 2025

chore(tools): Improve develoment tool management #3819

Merged

7 tasks

kakkoyun mentioned this pull request Aug 1, 2025

chore(ci): Improve CI #3833

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[LANGPLAT-437] Improve CI and dogfooding practices for `dd-trace-go`#3727

[LANGPLAT-437] Improve CI and dogfooding practices for `dd-trace-go`#3727
kakkoyun merged 11 commits into
DataDog:mainfrom
kakkoyun:kakkoyun/add-checklocks

kakkoyun commented Jul 4, 2025 •

edited

Loading

Uh oh!

darccio left a comment

Uh oh!

kakkoyun commented Jul 17, 2025

Uh oh!

kakkoyun commented Jul 17, 2025

Uh oh!

dd-devflow-routing-codex Bot commented Jul 17, 2025 •

edited

Loading

Uh oh!

rarguelloF left a comment

Uh oh!

kakkoyun commented Jul 17, 2025

Uh oh!

dd-devflow-routing-codex Bot commented Jul 17, 2025 •

edited

Loading

Uh oh!

kakkoyun commented Jul 17, 2025

Uh oh!

dd-devflow-routing-codex Bot commented Jul 17, 2025 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

kakkoyun commented Jul 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Motivation

Reviewer's Checklist

Uh oh!

darccio left a comment

Choose a reason for hiding this comment

Uh oh!

kakkoyun commented Jul 17, 2025

Uh oh!

kakkoyun commented Jul 17, 2025

Uh oh!

dd-devflow-routing-codex Bot commented Jul 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rarguelloF left a comment

Choose a reason for hiding this comment

Uh oh!

kakkoyun commented Jul 17, 2025

Uh oh!

dd-devflow-routing-codex Bot commented Jul 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kakkoyun commented Jul 17, 2025

Uh oh!

dd-devflow-routing-codex Bot commented Jul 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

kakkoyun commented Jul 4, 2025 •

edited

Loading

dd-devflow-routing-codex Bot commented Jul 17, 2025 •

edited

Loading

dd-devflow-routing-codex Bot commented Jul 17, 2025 •

edited

Loading

dd-devflow-routing-codex Bot commented Jul 17, 2025 •

edited

Loading