Skip to content

Conversation

@birschick-bq
Copy link
Contributor

@birschick-bq birschick-bq commented May 22, 2025

This PR adds OpenTelemetry-compatible tracing support on the CSharp library.

Under the hood it uses Activity and ActivitySource from the net System.Diagnostics.DiagnosticSource package.

Inherit from TracingConnection

public class YourConnection: TracingConnection
{
    public YourConnection(IReadOnlyDictionary<string, string> properties)
        : base(properties)
    {
    }
}

By default, the activitySourceName will be created from the driver assembly name.

Then to instrument tracing, use one of the following overrides of

  • TraceActivity
  • TraceActivityAsync

Example:

 public void Connect(...)
 {
    TraceActivity((activity) =>
    {
        driver.OpenSession(...);
    });
 }

Each of these overrides starts a new Activity which will be non-null only if there is an active ActivityListener or OpenTelemetry exporter. The Activity is passed to the delegate Func/Action in case it need to add ActivityEvent, ActivityLink or Tags (KeyValuePair). When instrumenting tracing, you should always use the null-conditional operator (?. ) when accessing members on the passed delegate parameter, activity.

Example:

public IArrowArrayStream GetObjects(...)
{
    return TraceActivity((activity) =>
    {
        activity?.AddEvent("db.operation.name.start", [new("db.operation.name", nameof(Client.GetCatalogs))]);
        var result = driver.GetCatalogs(...);
        var eventActivity = activity?.AddEvent("db.operation.name.end",
            [
                new("db.operation.name", nameof(Client.GetCatalogs)),
                new("db.response.status_code", getCatalogsResp.Status.StatusCode.ToString())
            ]);
        return ...
    });
}

The default behavior is to invoke the delegate and if successful (i.e., no exception thrown), the Activity.SetStatus is set to Ok. If an exception is observed, then the Activity.SetStatus is set to Error and the exception is traced (Activity.AddException) as an event in the current Activity.

Callers can pass a traceparent string to any of the TraceActivity[Async] methods using the optional traceParent parameter. The parameter takes precedence over the property. The traceId from the traceParent parameter or TraceParent property will be adopted as the rootId for all trace Activity on that call or object. If TraceParent is null (initial or set later), then the Activity creates a new rootId for the beginning of the initial Activity in the stack.

Example:

public void Connect(..., string? traceParent = default)
{
    TraceActivity((activity) =>
    {
        activity?.AddEvent("opensession.start");
        var result = driver.OpenSession(...);
        activity?.AddEvent("opensession.end", ["status", result.Status]);
    }, traceParent: traceParent);
}

Identifiers used for events and tags should follow the OpenTelemetry semantic guidance ..
https://opentelemetry.io/docs/specs/semconv/database/database-spans/
https://opentelemetry.io/docs/specs/semconv/database/database-metrics/

@birschick-bq birschick-bq marked this pull request as ready for review May 26, 2025 23:39
@github-actions github-actions bot added this to the ADBC Libraries 19 milestone May 26, 2025
@CurtHagenlocher
Copy link
Contributor

Perhaps @alexguo-db or @eric-wang-1990 can look at the Databricks-specific changes also?

Copy link
Contributor

@CurtHagenlocher CurtHagenlocher left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! My main concern at this point is that the Activitys being created in ActivityTrace.cs don't seem to be getting Disposed. But I also have a collection of smaller nits and questions.

@birschick-bq birschick-bq marked this pull request as draft June 19, 2025 22:49
@birschick-bq birschick-bq marked this pull request as ready for review June 20, 2025 00:08
@birschick-bq
Copy link
Contributor Author

@alexguo-db / @eric-wang-1990
Sent in an invite to my repository, in case you want to contribute to this PR using this branch: birschick-bq:dev/birschick-bq/csharp-otel-tracing

@davidhcoe
Copy link
Contributor

@birschick-bq - will you get a chance to resolve the merge conflicts?

@birschick-bq
Copy link
Contributor Author

@birschick-bq - will you get a chance to resolve the merge conflicts?
@davidhcoe
Conflicts resolved

Copy link
Contributor

@CurtHagenlocher CurtHagenlocher left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, looks good to me. Just two small nits to point out.

@CurtHagenlocher CurtHagenlocher merged commit cde9e7b into apache:main Jun 24, 2025
7 checks passed
toddmeng-db pushed a commit to toddmeng-db/arrow-adbc that referenced this pull request Jul 10, 2025
…pache#2847)

This PR adds OpenTelemetry-compatible tracing support on the CSharp
library.

Under the hood it uses `Activity` and `ActivitySource` from the net
`System.Diagnostics.DiagnosticSource` package.

Inherit from `TracingConnection`

```csharp
public class YourConnection: TracingConnection
{
    public YourConnection(IReadOnlyDictionary<string, string> properties)
        : base(properties)
    {
    }
}
```

By default, the `activitySourceName` will be created from the driver
assembly name.

Then to instrument tracing, use one of the following overrides of
* `TraceActivity`
* `TraceActivityAsync`

Example:
```csharp
 public void Connect(...)
 {
    TraceActivity((activity) =>
    {
        driver.OpenSession(...);
    });
 }
```

Each of these overrides starts a new `Activity` which will be non-null
**only** if there is an active `ActivityListener` or `OpenTelemetry`
exporter. The `Activity` is passed to the delegate Func/Action in case
it need to add `ActivityEvent`, `ActivityLink` or `Tags`
(`KeyValuePair`). When instrumenting tracing, you should always use the
null-conditional operator (`?.` ) when accessing members on the passed
delegate parameter, `activity`.

Example:
```csharp
public IArrowArrayStream GetObjects(...)
{
    return TraceActivity((activity) =>
    {
        activity?.AddEvent("db.operation.name.start", [new("db.operation.name", nameof(Client.GetCatalogs))]);
        var result = driver.GetCatalogs(...);
        var eventActivity = activity?.AddEvent("db.operation.name.end",
            [
                new("db.operation.name", nameof(Client.GetCatalogs)),
                new("db.response.status_code", getCatalogsResp.Status.StatusCode.ToString())
            ]);
        return ...
    });
}
```

The default behavior is to invoke the delegate and if successful (i.e.,
no exception thrown), the `Activity.SetStatus` is set to `Ok`. If an
exception is observed, then the `Activity.SetStatus` is set to `Error`
and the exception is traced (`Activity.AddException`) as an event in the
current `Activity`.

Callers can pass a `traceparent` string to any of the
TraceActivity[Async] methods using the optional `traceParent` parameter.
The parameter takes precedence over the property. The `traceId` from the
`traceParent` parameter or `TraceParent` property will be adopted as the
`rootId` for all trace Activity on that call or object. If `TraceParent`
is null (initial or set later), then the `Activity` creates a new
`rootId` for the beginning of the initial `Activity` in the stack.

Example:
```csharp
public void Connect(..., string? traceParent = default)
{
    TraceActivity((activity) =>
    {
        activity?.AddEvent("opensession.start");
        var result = driver.OpenSession(...);
        activity?.AddEvent("opensession.end", ["status", result.Status]);
    }, traceParent: traceParent);
}
```

Identifiers used for events and tags should follow the OpenTelemetry
semantic guidance ..
https://opentelemetry.io/docs/specs/semconv/database/database-spans/
https://opentelemetry.io/docs/specs/semconv/database/database-metrics/
@birschick-bq birschick-bq deleted the dev/birschick-bq/csharp-otel-tracing branch August 13, 2025 20:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants