Skip to content

contrib/database/sql: Close DB Stats goroutine on db.Close()#3025

Merged
mtoffl01 merged 23 commits into
mainfrom
mtoff/contrib-routines
Feb 3, 2025
Merged

contrib/database/sql: Close DB Stats goroutine on db.Close()#3025
mtoffl01 merged 23 commits into
mainfrom
mtoff/contrib-routines

Conversation

@mtoffl01

@mtoffl01 mtoffl01 commented Dec 10, 2024

Copy link
Copy Markdown
Contributor

What does this PR do?

Introduces a stop condition for the pollDBStats goroutine when db.Close() has been invoked by the user.

Motivation

Protecting against runaway goroutines.

Details

Background
To register a sql driver for instrumentation, a user must call sqltrace.Open, which passes our tracedConnector into the OpenDB function. This tracedConnector is used as the c driver.Connector argument to database/sql's OpenDB function, which is set on the DB.connector field.
When db.Close() is called by the user, the database/sql code checks whether the DB.connector satisfies the io.Closer interface and, if so, calls connector.Close().
Therefore, to identify when a user has called db.Close(), we can implement (and control) a tracedConnector.Close() method.

Implementation
A new tracedConnector.dbClose channel is closed when tracedConnector.Close() is invoked. The pollDBStats goroutine returns (stops) when there is activity on the dbClose channel.
In this way, we've tied the pollDBStats goroutine to the lifecycle of the db.

Next Steps

Once this approach is approved, we will do the same thing for the contrib/jackc/pgx pollPoolStats goroutine.

Reviewer's Checklist

  • Changed code has unit tests for its functionality at or near 100% coverage.
  • System-Tests covering this feature have been added and enabled with the va.b.c-dev version tag.
  • There is a benchmark for any new code, or changes to existing code.
  • If this interacts with the agent in a new way, a system test has been added.
  • Add an appropriate team label so this PR gets put in the right place for the release notes.
  • Non-trivial go.mod changes, e.g. adding new modules, are reviewed by @DataDog/dd-trace-go-guild.
  • For internal contributors, a matching PR should be created to the v2-dev branch and reviewed by @DataDog/apm-go.

Unsure? Have a question? Request a review!

@github-actions github-actions Bot added the apm:ecosystem contrib/* related feature requests or bugs label Dec 10, 2024
@datadog-datadog-prod-us1

datadog-datadog-prod-us1 Bot commented Dec 10, 2024

Copy link
Copy Markdown

Datadog Report

Branch report: mtoff/contrib-routines
Commit report: 29d925e
Test service: dd-trace-go

❌ 1 Failed (0 Known Flaky), 5014 Passed, 71 Skipped, 2m 44.91s Total Time

❌ Failed Tests (1)

  • TestMain - gopkg.in/DataDog/dd-trace-go.v1/contrib/database/sql - Details

    Expand for error
     Failed
     
     FAIL	gopkg.in/DataDog/dd-trace-go.v1/contrib/database/sql [build failed]
    

@pr-commenter

pr-commenter Bot commented Dec 10, 2024

Copy link
Copy Markdown

Benchmarks

Benchmark execution time: 2025-02-03 15:54:20

Comparing candidate commit 823375a in PR branch mtoff/contrib-routines with baseline commit 0a41ffd in branch main.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 58 metrics, 1 unstable metrics.

@mtoffl01
mtoffl01 force-pushed the mtoff/contrib-routines branch from 0692055 to 37b52f9 Compare December 17, 2024 01:12

@darccio darccio left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. I left some small nits.

})
}

func GetStopChan() chan struct{} {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
func GetStopChan() chan struct{} {
func StopChan() chan struct{} {

This would be a better name.

mu sync.Mutex
)

func InitStopChan() {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
func InitStopChan() {
func Init() {

Just in case this package evolves and needs to initialize more things beyond the stop chan.

@kakkoyun
kakkoyun self-requested a review January 17, 2025 16:18
@datadog-datadog-prod-us1

datadog-datadog-prod-us1 Bot commented Jan 23, 2025

Copy link
Copy Markdown

Datadog Report

Branch report: mtoff/contrib-routines
Commit report: 08ab2df
Test service: dd-trace-go

✅ 0 Failed, 5222 Passed, 72 Skipped, 1m 53.27s Total Time

@mtoffl01 mtoffl01 changed the title Allow goroutines not launched by the tracer to stop on tracer.Stop() contrib/database/sql: Close DB Stats goroutine on db.Close() Jan 24, 2025
@mtoffl01
mtoffl01 marked this pull request as ready for review January 24, 2025 16:50
@mtoffl01
mtoffl01 requested a review from a team as a code owner January 24, 2025 16:50
@mtoffl01
mtoffl01 requested a review from a team as a code owner January 24, 2025 16:50

@kakkoyun kakkoyun left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good 👍 I have a question though.

Comment thread contrib/database/sql/sql.go
@mtoffl01
mtoffl01 enabled auto-merge (squash) January 28, 2025 16:10
@mtoffl01
mtoffl01 merged commit dbfb8f2 into main Feb 3, 2025
@mtoffl01
mtoffl01 deleted the mtoff/contrib-routines branch February 3, 2025 16:09

// Close closes the dbClose channel
// This method will be invoked when DB.Close() is called, which we expect to occur only once: https://cs.opensource.google/go/go/+/refs/tags/go1.23.4:src/database/sql/sql.go;l=918-950
func (t *tracedConnector) Close() error {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This makes me realize we probably had a bug before, since this wraps the original driver.Connector, if the original one implemented Close() error, our traced version didn't (until this PR).

Maybe we should internally call the original connector Close method, in case it implements io.Closer?

(btw this is ok to do outside of this PR, since this issue has been there before).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

apm:ecosystem contrib/* related feature requests or bugs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants