Skip to content

botocore: add basic tracing for bedrock ConverseStream#3204

Merged
xrmx merged 7 commits intoopen-telemetry:mainfrom
xrmx:bedrock-converse-stream
Jan 27, 2025
Merged

botocore: add basic tracing for bedrock ConverseStream#3204
xrmx merged 7 commits intoopen-telemetry:mainfrom
xrmx:bedrock-converse-stream

Conversation

@xrmx
Copy link
Copy Markdown
Contributor

@xrmx xrmx commented Jan 23, 2025 •
edited
Loading

Description

This introduces basic tracing for the ConverseStream service. I've chosen to delay the closing of the span after the response has been consumed instead of buffering all the response chunks since the buffer may become big.

Type of change

Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update

How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration.

  • tox -e py310-test-instrumentation-botocore

Does This PR Require a Core Repo Change?

  • Yes. - Link to PR:
  • No.

Checklist:

See contributing.md for styleguide, changelog guidelines, and more.

  • Followed the style guidelines of this project
  • Changelogs have been updated
  • Unit tests have been added
  • Documentation has been updated

@xrmx xrmx requested a review from a team as a code owner January 23, 2025 16:53
xrmx
xrmx commented Jan 23, 2025
View reviewed changes
Comment thread ...trumentation-botocore/src/opentelemetry/instrumentation/botocore/extensions/bedrock_utils.py Outdated
xrmx
xrmx commented Jan 23, 2025
View reviewed changes
Comment thread ...etry-instrumentation-botocore/src/opentelemetry/instrumentation/botocore/extensions/types.py Outdated
codefromthecrypt
codefromthecrypt approved these changes Jan 23, 2025
View reviewed changes
Copy link
Copy Markdown
Contributor

@codefromthecrypt codefromthecrypt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. Do you mind checking if this langchain code produces spans as intended?

from langchain_aws import ChatBedrock


def main():
    llm = ChatBedrock(
        beta_use_converse_api=True,
        model_id='amazon.titan-text-lite-v1',
        streaming=True,
    )
    response_stream = llm.stream("Write a short poem on OpenTelemetry.")
    for chunk in response_stream:
        for item in chunk.content:
            if 'text' in item:
                print(item['text'], end="")

if __name__ == "__main__":
    main()

@xrmx
Copy link
Copy Markdown
Contributor Author

xrmx commented Jan 24, 2025 •
edited
Loading 

from langchain_aws import ChatBedrock


def main():
    llm = ChatBedrock(
        beta_use_converse_api=True,
        model_id='amazon.titan-text-lite-v1',
        streaming=True,
    )
    response_stream = llm.stream("Write a short poem on OpenTelemetry.")
    for chunk in response_stream:
        for item in chunk.content:
            if 'text' in item:
                print(item['text'], end="")

if __name__ == "__main__":
    main()

It does:

{
    "name": "chat amazon.titan-text-lite-v1",
    "context": {
        "trace_id": "0xa584c79bfbd24ee4a69a470f62f5a785",
        "span_id": "0x330a7997ae2fd3c8",
        "trace_state": "[]"
    },
    "kind": "SpanKind.CLIENT",
    "parent_id": null,
    "start_time": "2025-01-24T08:43:40.741219Z",
    "end_time": "2025-01-24T08:43:44.649476Z",
    "status": {
        "status_code": "UNSET"
    },
    "attributes": {
        "rpc.system": "aws-api",
        "rpc.service": "Bedrock Runtime",
        "rpc.method": "ConverseStream",
        "aws.region": "us-east-1",
        "gen_ai.system": "aws.bedrock",
        "gen_ai.request.model": "amazon.titan-text-lite-v1",
        "gen_ai.operation.name": "chat",
        "aws.request_id": "c41388e1-9406-4376-8d43-8df022879225",
        "retry_attempts": 0,
        "http.status_code": 200,
        "gen_ai.usage.input_tokens": 12,
        "gen_ai.usage.output_tokens": 128,
        "gen_ai.response.finish_reasons": [
            "max_tokens"
        ]
    },
    "events": [],
    "links": [],
    "resource": {
        "attributes": {
            "telemetry.sdk.language": "python",
            "telemetry.sdk.name": "opentelemetry",
            "telemetry.sdk.version": "1.30.0.dev0",
            "telemetry.auto.version": "0.51b0.dev",
            "service.name": "unknown_service"
        },
        "schema_url": ""
    }
}

@xrmx xrmx force-pushed the bedrock-converse-stream branch from 5241f21 to 9af5b68 Compare January 24, 2025 08:52
@linux-foundation-easycla
Copy link
Copy Markdown

linux-foundation-easycla Bot commented Jan 24, 2025 •
edited
Loading

CLA Signed

The committers listed above are authorized under a signed CLA.

@xrmx xrmx force-pushed the bedrock-converse-stream branch from 9af5b68 to bd8a50f Compare January 24, 2025 08:53
@xrmx xrmx mentioned this pull request Jan 24, 2025
Merged
11 tasks
@xrmx xrmx mentioned this pull request Jan 25, 2025
Open
18 tasks
@xrmx xrmx merged commit 0bb1c42 into open-telemetry:main Jan 27, 2025
aryabharat pushed a commit to aryabharat/opentelemetry-python-contrib that referenced this pull request Feb 2, 2025
@xrmx @aryabharat
botocore: add basic tracing for bedrock ConverseStream (open-telemetr…
a812fc6 
…y#3204)

* Add tracing for ConverseStream

* Add converse stream example
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@lmolkova lmolkova lmolkova left review comments

+1 more reviewer

@codefromthecrypt codefromthecrypt codefromthecrypt approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

@codefromthecrypt codefromthecrypt

@lmolkova lmolkova

@nirga nirga

@gyliu513 gyliu513

@nemoshlag nemoshlag

@lzchen lzchen

@nikosokolik nikosokolik

@oxeye-nikolay oxeye-nikolay

@karthikscale3 karthikscale3

@alizenhom alizenhom

Labels

None yet

Projects

Python PR digest
Status: Done

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.