Skip to content
This repository was archived by the owner on Mar 4, 2026. It is now read-only.

OpenTelemetry spans being ended multiple times causing "Operation attempted on ended Span" errors #2320

@hsojod

Description

@hsojod

Please make sure you have searched for information in the following guides.

A screenshot that you have tested with "Try this API".

Image

Link to the code that reproduces this issue. A link to a public Github Repository or gist with a minimal reproduction.

https://github.com/hsojod/otel-spanner-sample

A step-by-step description of how to reproduce the issue, based on the linked reproduction.

Environment

  • @google-cloud/spanner version: 8.x.x (latest)
  • Node.js version: 20.x
  • express": "^5.1.0"
  • OpenTelemetry SDK version:
    • @opentelemetry/api: ^1.9.0
    • @opentelemetry/sdk-node: ^0.202.0
    • @opentelemetry/auto-instrumentations-node: ^0.60.0

A complete sample application demonstrating this bug is available at:
https://github.com/hsojod/otel-spanner-sample
To reproduce the issue:

  1. Clone and run the sample app

  2. Make a test request to trigger the bug:
    curl http://localhost:8080/

  3. Inspect the logs:
    The application outputs OpenTelemetry traces to the console. You will see the span lifecycle errors mixed in with the trace output, specifically the "Operation attempted on ended Span" errors described above.

The sample application is configured with the OpenTelemetry NodeSDK including auto-instrumentation and demonstrates the bug occurring during basic Spanner operations. Each request generates traces showing the problematic span management within the Spanner client library.

A clear and concise description of what the bug is, and what you expected to happen.

Bug Description

The Google Cloud Spanner Node.js client library has OpenTelemetry instrumentation that attempts to operate on spans that have already been ended, resulting in runtime errors. This appears to be a span lifecycle management issue within the library's tracing implementation.

Error Messages

Cannot execute the operation on ended Span {traceId: 052e2003b73b69c6aca3083d4be34ca0, spanId: 35d90b5b849811f5} Error: Operation attempted on ended Span {traceId: 052e2003b73b69c6aca3083d4be34ca0, spanId: 35d90b5b849811f5}
 at SpanImpl._isSpanEnded (node_modules/@opentelemetry/sdk-trace-base/build/src/Span.js:265:27)
 at SpanImpl.addEvent (node_modules/@opentelemetry/sdk-trace-base/build/src/Span.js:118:18)
 at Snapshot._update (node_modules/@google-cloud/spanner/build/src/transaction.js:986:14)
 at PartialResultStream.<anonymous> (node_modules/@google-cloud/spanner/build/src/transaction.js:822:26)

Cannot execute the operation on ended Span {traceId: 052e2003b73b69c6aca3083d4be34ca0, spanId: 4693552b6a7918ee} Error: Operation attempted on ended Span {traceId: 052e2003b73b69c6aca3083d4be34ca0, spanId: 4693552b6a7918ee}
 at SpanImpl._isSpanEnded (node_modules/@opentelemetry/sdk-trace-base/build/src/Span.js:265:27)
 at SpanImpl.end (node_modules/@opentelemetry/sdk-trace-base/build/src/Span.js:178:18)
 at Transform.<anonymous> (node_modules/@google-cloud/spanner/build/src/database.js:1204:22)

Cannot execute the operation on ended Span {traceId: 052e2003b73b69c6aca3083d4be34ca0, spanId: d3c5d5fe4b9e23e4} Error: Operation attempted on ended Span {traceId: 052e2003b73b69c6aca3083d4be34ca0, spanId: d3c5d5fe4b9e23e4}
 at SpanImpl._isSpanEnded (node_modules/@opentelemetry/sdk-trace-base/build/src/Span.js:265:27)
 at SpanImpl.end (node_modules/@opentelemetry/sdk-trace-base/build/src/Span.js:178:18)
 at PartialResultStream.<anonymous> (node_modules/@google-cloud/spanner/build/src/transaction.js:853:26)

Additional warnings:

CloudSpanner.Database.runStream 052e2003b73b69c6aca3083d4be34ca0-4693552b6a7918ee - You can only call end() on a span once.
CloudSpanner.Snapshot.runStream 052e2003b73b69c6aca3083d4be34ca0-d3c5d5fe4b9e23e4 - You can only call end() on a span once.

A clear and concise description WHY you expect this behavior, i.e., was it a recent change, there is documentation that points to this behavior, etc. **

Expected Behavior

OpenTelemetry spans should be properly managed with:

  • Spans only ended once
  • No operations attempted on ended spans
  • Proper span lifecycle management across async operations and streams
  • Provide configuration options: Add client configuration options to disable built-in tracing:

Metadata

Metadata

Assignees

Labels

api: spannerIssues related to the googleapis/nodejs-spanner API.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions