Skip to content

Define external references for generative AI inference content - #490

Merged
lmolkova merged 12 commits into
open-telemetry:mainfrom
dev404ai:content-storage-result
Oct 27, 2026
Merged

lmolkova merged 12 commits into
open-telemetry:mainfrom
dev404ai:content-storage-result

Conversation

@dev404ai

@dev404ai dev404ai commented Sep 2, 2026 •

Copy link
Copy Markdown
Contributor

Description

Fixes #45.

Adds opt-in _ref attributes that point to complete system instructions, input messages, output messages and tool definitions held in external storage. Each value is an absolute URI, and the convention does not prescribe its scheme, layout, serialization, retention or access mechanism.

Uploading content and emitting references should be enabled separately from the flags that record the inline content attributes. Following open-telemetry/semantic-conventions#4102, the four inline counterparts are annotated unbounded_size: true, and the policy pin moves so the _ref and unbounded policies run here.

Motivation

Complete prompts, messages and tool schemas can be too large or too sensitive to put on spans, yet incident triage and quality evaluation need them. A reference lets an operator go from an inference span to the stored content without making inline capture the only option.

The three original reference names and the one-attribute-per-value shape come from issue #45 and open-telemetry/semantic-conventions#2750. Tool definitions are added because they have the same storage need as system instructions and opentelemetry-util-genai already uploads them.

Prototype

The Anthropic scenario sends a Messages request to the mock server with system instructions, input messages and a tool definition, enables the upload hook through OTEL_INSTRUMENTATION_GENAI_COMPLETION_HOOK=upload, and records all four references on the inference span and the operation details event.

The references sit in the shared content group next to their inline counterparts, so the invoke agent spans inherit them as well. No agent scenario emits them yet.

Checklist

  • Motivation section filled in above
  • Reference scenarios updated for affected libraries
  • Towncrier fragment added under changelog.d/ for any change to the conventions that a consumer would care about

@dev404ai
dev404ai requested a review from a team as a code owner September 2, 2026 05:44
Copilot AI balanced review requested due to automatic review settings September 2, 2026 05:44
@linux-foundation-easycla

linux-foundation-easycla Bot commented Sep 2, 2026 •

Copy link
Copy Markdown

CLA Signed
The committers listed above are authorized under a signed CLA.

  • ✅ login: dev404ai / name: dev404ai (f5e81ce)

@dev404ai
dev404ai force-pushed the content-storage-result branch from f08e25a to f5e81ce Compare September 2, 2026 05:48

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The scenario does not cover the new span attributes, and the model omits the promised cardinality warnings.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Defines external GenAI content references and an event reporting asynchronous storage outcomes.

Changes:

  • Adds three reference attributes and storage statuses.
  • Adds a reference scenario covering stored, failed, and dropped uploads.
  • Regenerates documentation and compliance reports.
File summaries
File Description
model/gen-ai/registry.yaml Defines references and statuses.
model/gen-ai/spans.yaml Adds references to spans.
model/gen-ai/events.yaml Defines the result event.
reference/scenarios/opentelemetry-util-genai/scenario.py Implements upload scenarios.
reference/scenarios/opentelemetry-util-genai/README.md Documents scenario scope.
reference/scenarios/opentelemetry-util-genai/pyproject.toml Configures dependencies.
reference/scenarios/opentelemetry-util-genai/data.json Records conformance output.
reference/scenarios/opentelemetry-util-genai/conformance.yaml Configures scenario execution.
reference/src/semconv_genai/semconv_model.py Registers the event.
reference/src/semconv_genai/report.py Adds its documentation link.
reference/src/semconv_genai/data_files.py Orders the event in reports.
reference/reports/invoke-agent-internal-span.md Reports reference coverage.
reference/reports/invoke-agent-client-span.md Reports reference coverage.
reference/reports/inference-span.md Reports reference coverage.
reference/reports/gen-ai-content-storage-result-event.md Reports event coverage.
reference/reports/gen-ai-client-inference-operation-details-event.md Reports reference coverage.
reference/README.md Lists the new event scenario.
docs/registry/attributes/gen-ai.md Regenerates the attribute registry.
docs/gen-ai/openai.md Regenerates OpenAI conventions.
docs/gen-ai/gen-ai-spans.md Documents external storage references.
docs/gen-ai/gen-ai-events.md Documents the result event.
docs/gen-ai/gen-ai-agent-spans.md Regenerates agent span tables.
docs/gen-ai/azure-ai-inference.md Regenerates Azure conventions.
docs/gen-ai/aws-bedrock.md Regenerates Bedrock conventions.
docs/gen-ai/anthropic.md Regenerates Anthropic conventions.
changelog.d/+.enhancement.md Adds the release note.
Review details

Suppressed comments (2)

model/gen-ai/registry.yaml:951

  • The PR description says the attribute notes call out high cardinality, but this warning only mentions sensitive paths. Add the omitted cardinality warning so instrumentation authors understand the span cost risk described by the proposal.
      > This attribute may contain sensitive information such as storage paths.

model/gen-ai/registry.yaml:1001

  • The PR description says the attribute notes call out high cardinality, but this warning only mentions sensitive paths. Add the omitted cardinality warning so instrumentation authors understand the span cost risk described by the proposal.
      > This attribute may contain sensitive information such as storage paths.
  • Files reviewed: 26/27 changed files
  • Comments generated: 2
  • Review effort level: Balanced

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread model/gen-ai/registry.yaml Resolved

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The scenario misreports upload-hook calls as inference and fails for non-empty tool definitions.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Suppressed comments (1)

reference/scenarios/opentelemetry-util-genai/scenario.py:193

  • This second chat span also reports an inference operation even though the only library call is the upload completion hook. Per reference/CONTRIBUTING.md:101-113, a scenario may emit inference only for a library that owns the model-call boundary; move this reference-enrichment case to an actual provider or framework operation span instead.
        with _reference_tracer.start_as_current_span(
            "chat",
            kind=SpanKind.CLIENT,
            attributes={"gen_ai.operation.name": "chat"},
        ) as span:
  • Files reviewed: 26/27 changed files
  • Comments generated: 2
  • Review effort level: Balanced

@opentelemetry-pr-dashboard

opentelemetry-pr-dashboard Bot commented Sep 2, 2026 •

Copy link
Copy Markdown

Pull request dashboard status

Merged · refreshed 2026-10-27 17:02 UTC

Status above doesn't look right?
  • Anything look wrong? Report it with what you expected; it helps us improve the dashboard.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Several content-bearing span types lack matching reference attributes, and invoke-agent coverage remains unproven.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details
  • Files reviewed: 30/32 changed files
  • Comments generated: 2
  • Review effort level: Balanced

Comment thread model/gen-ai/spans.yaml Resolved
Comment thread model/gen-ai/spans.yaml Resolved

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The new public telemetry contract and asynchronous result semantics need final maintainer review.

Review details
  • Files reviewed: 27/29 changed files
  • Comments generated: 0 new
  • Review effort level: Balanced

Comment thread model/gen-ai/events.yaml Resolved
Comment thread model/gen-ai/registry.yaml Resolved
Comment thread model/gen-ai/registry.yaml Resolved
Comment thread model/gen-ai/registry.yaml Resolved
Comment thread model/gen-ai/registry.yaml Resolved
Comment thread model/gen-ai/registry.yaml Resolved
Comment thread model/gen-ai/registry.yaml Resolved
@aabmass aabmass self-assigned this Sep 3, 2026
@dev404ai dev404ai changed the title Define external content references and storage results Define external references for generative AI inference content Sep 3, 2026

@Zhuoxi2000 Zhuoxi2000 left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As a backend implementer, refs need a predictable stored format. Could the spec say writers SHOULD store the attribute's JSON? Otherwise readers cannot resolve refs across vendors.

@opentelemetry-pr-dashboard

This comment has been minimized.

@lmolkova lmolkova left a comment •

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Directionally, looks good!

Process wise:

# Conflicts:
#	docs/gen-ai/gen-ai-events.md
#	docs/gen-ai/gen-ai-spans.md
#	reference/scenarios/anthropic/scenario.py
@dev404ai

Copy link
Copy Markdown
Contributor Author

@lmolkova thanks, both items are in commit bce777b:

The branch is also merged with current main. The four references stay opt-in on gen_ai.client.inference and are now documented in client-inference.md.

Comment thread docs/gen-ai/gen-ai-spans.md Resolved
Comment thread docs/gen-ai/gen-ai-spans.md Resolved
Comment thread docs/gen-ai/gen-ai-spans.md Resolved
Comment thread docs/gen-ai/gen-ai-spans.md Resolved
Comment thread model/gen-ai/registry.yaml Resolved
Comment thread model/gen-ai/spans.yaml Resolved
Comment thread docs/gen-ai/gen-ai-spans.md Resolved
Comment thread docs/gen-ai/gen-ai-spans.md Resolved
Comment thread model/gen-ai/registry.yaml Resolved
@lmolkova
lmolkova self-requested a review October 13, 2026 18:09
@dev404ai

Copy link
Copy Markdown
Contributor Author

@lmolkova thanks, the comments from the 2026-10-13 review are addressed in 7c728cf, which also merges current main.

  • docs/gen-ai/gen-ai-spans.md: external handling is now SHOULD be enabled independently of the opt-in flags, and the duplicated sentences are removed. The hook receives the span and/or event instance, and tool definitions are covered in both paragraphs.
  • model/gen-ai/registry.yaml: the *_ref briefs say "An absolute URI", the "otherwise" wording is gone and the notes state that a reference may be recorded together with the attribute it points to. The notes now say what "opaque" means for consumers and mark credentials in the URI, such as signed URL parameters, as NOT RECOMMENDED.
  • model/gen-ai/spans.yaml: the four references moved into attributes.gen_ai.content next to their counterparts. That group is shared with invoke_agent, so the references now appear on those spans as well.
  • reference/scenarios/anthropic/scenario.py: the scenario enables the hook through OTEL_INSTRUMENTATION_GENAI_COMPLETION_HOOK=upload and load_completion_hook(). The wrapper it returns has no shutdown, so the scenario still flushes the wrapped upload hook before removing the temporary directory.

Annotating gen_ai.tool.call.arguments, gen_ai.tool.call.result, gen_ai.retrieval.documents and gen_ai.memory.records as unbounded is not part of this PR. I will open a separate pull request for it.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Agent reference coverage is missing, and the upload enablement language does not enforce the promised independent opt-in.

2 open issues

Review effort: Balanced

Comment thread docs/gen-ai/gen-ai-spans.md Resolved
Comment thread model/gen-ai/spans.yaml Resolved
Comment thread model/gen-ai/registry.yaml Outdated
note: |
This attribute identifies an external representation of the complete
value that would otherwise be recorded in `gen_ai.input.messages`.
Consumers MUST treat the URI as opaque.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Semantic conventions are not authoritative for consumers of telemetry, only for producers controlled by otel, plus it does not seem to be necessary to put any expectations on consumers.

An example of how it's used in the wild https://docs.cloud.google.com/stackdriver/docs/instrumentation/collect-view-multimodal-prompts-responses- the upload behavior and consumption are controlled by the same platform; credentials, destination, and consumption etc are all controlled by the same entity. It's easy to imagine that any other company can have a distro that uploads using their custom code to a storage of their preference with access model aligned with telemetry producer.

I think opaque can't come with MUST and I don't think it's helpful at all. Why do we care in semantic conventions about it?

Comment thread model/gen-ai/registry.yaml Resolved
Comment thread model/gen-ai/registry.yaml Resolved
Comment thread docs/gen-ai/gen-ai-spans.md Resolved
Comment thread docs/gen-ai/gen-ai-spans.md Resolved
Comment thread changelog.d/490.enhancement.md Resolved
…back

Keep the content upload configuration from main next to the external reference wording, update the `changelog.d` fragment to name the four reference attributes, and let the Anthropic scenario upload into a `mkdtemp()` directory instead of flushing the private wrapped hook.
@lmolkova
lmolkova added this pull request to the merge queue Oct 27, 2026
Merged via the queue into open-telemetry:main with commit 51a0f58 Oct 27, 2026
44 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

Specify attributes for gen_ai external storage references

6 participants