Initial APM side for aws bedrock#4937
Merged
Merged
Conversation
Contributor
Overall package sizeSelf size: 8.43 MB Dependency sizes| name | version | self size | total size | |------|---------|-----------|------------| | @datadog/libdatadog | 0.3.0 | 29.43 MB | 29.43 MB | | @datadog/native-appsec | 8.3.0 | 19.37 MB | 19.38 MB | | @datadog/native-iast-taint-tracking | 3.2.0 | 13.9 MB | 13.91 MB | | @datadog/pprof | 5.4.1 | 9.76 MB | 10.13 MB | | protobufjs | 7.2.5 | 2.77 MB | 5.16 MB | | @datadog/native-iast-rewriter | 2.6.1 | 2.59 MB | 2.73 MB | | @opentelemetry/core | 1.14.0 | 872.87 kB | 1.47 MB | | @datadog/native-metrics | 3.1.0 | 1.06 MB | 1.46 MB | | @opentelemetry/api | 1.8.0 | 1.21 MB | 1.21 MB | | import-in-the-middle | 1.11.2 | 112.74 kB | 826.22 kB | | source-map | 0.7.4 | 226 kB | 226 kB | | opentracing | 0.14.7 | 194.81 kB | 194.81 kB | | lru-cache | 7.18.3 | 133.92 kB | 133.92 kB | | pprof-format | 2.1.0 | 111.69 kB | 111.69 kB | | @datadog/sketches-js | 2.1.0 | 109.9 kB | 109.9 kB | | semver | 7.6.3 | 95.82 kB | 95.82 kB | | lodash.sortby | 4.7.0 | 75.76 kB | 75.76 kB | | ignore | 5.3.1 | 51.46 kB | 51.46 kB | | shell-quote | 1.8.1 | 44.96 kB | 44.96 kB | | istanbul-lib-coverage | 3.2.0 | 29.34 kB | 29.34 kB | | rfdc | 1.3.1 | 25.21 kB | 25.21 kB | | @isaacs/ttlcache | 1.4.1 | 25.2 kB | 25.2 kB | | tlhunter-sorted-set | 0.1.0 | 24.94 kB | 24.94 kB | | limiter | 1.1.5 | 23.17 kB | 23.17 kB | | dc-polyfill | 0.1.4 | 23.1 kB | 23.1 kB | | retry | 0.13.1 | 18.85 kB | 18.85 kB | | jest-docblock | 29.7.0 | 8.99 kB | 12.76 kB | | crypto-randomuuid | 1.0.0 | 11.18 kB | 11.18 kB | | path-to-regexp | 0.1.12 | 6.6 kB | 6.6 kB | | koalas | 1.0.2 | 6.47 kB | 6.47 kB | | module-details-from-path | 1.0.3 | 4.47 kB | 4.47 kB |🤖 This report was automatically generated by heaviest-objects-in-the-universe |
sabrenner
reviewed
Nov 26, 2024
sabrenner
reviewed
Nov 26, 2024
BenchmarksBenchmark execution time: 2024-12-24 17:13:49 Comparing candidate commit 6220304 in PR branch Found 0 performance improvements and 1 performance regressions! Performance is the same for 776 metrics, 21 unstable metrics. scenario:log-with-error-20
|
yahya-mouman
commented
Dec 2, 2024
sabrenner
reviewed
Dec 10, 2024
sabrenner
reviewed
Dec 10, 2024
sabrenner
reviewed
Dec 10, 2024
sabrenner
reviewed
Dec 10, 2024
sabrenner
reviewed
Dec 10, 2024
Co-authored-by: Sam Brenner <[email protected]>
…yahya/add-bedrock-support
sabrenner
reviewed
Dec 20, 2024
sabrenner
reviewed
Dec 20, 2024
sabrenner
left a comment
sabrenner
left a comment
Collaborator
There was a problem hiding this comment.
i think we're going to want to add DD_BEDROCK_SPAN_CHAR_LIMIT and DD_BEDROCK_SPAN_PROMPT_COMPLETION_SAMPLE_RATE to configuration as well, as I likely won't have a refactor for apm llm plugins in time to make this generic for you. we can go over how to do this offline!
tlhunter
reviewed
Dec 20, 2024
tlhunter
reviewed
Dec 20, 2024
tlhunter
reviewed
Dec 20, 2024
tlhunter
reviewed
Dec 20, 2024
tlhunter
reviewed
Dec 20, 2024
…yahya/add-bedrock-support
tlhunter
approved these changes
Dec 24, 2024
sabrenner
approved these changes
Jan 9, 2025
sabrenner
left a comment
sabrenner
left a comment
Collaborator
There was a problem hiding this comment.
This is really great stuff!! We can do the config changes (I/O character limit and sampling) in a follow-up PR next week once I merge in some shared utilities for the other LLM-based plugins. Left one small nit but can also be left for follow-up cleanup. Great work!
Comment on lines
+78
to
+80
| this.temperature = temperature !== undefined ? temperature : undefined | ||
| this.topP = topP !== undefined ? topP : undefined | ||
| this.maxTokens = maxTokens !== undefined ? maxTokens : undefined |
Collaborator
There was a problem hiding this comment.
nit: i think these checks (and the one on line 85) can just be this.temperature = temperature, or whatever the name is, as we default it do undefined anyways. let me know if i'm reading this wrong, though!
watson
pushed a commit
that referenced
this pull request
Jan 22, 2025
* Initial APM side for aws bedrock * add extract response tags * add extract response tags * remove hook * Update packages/datadog-plugin-aws-sdk/src/services/bedrockruntime.js * added example test for invoke amazon * added example test for invoke amazon * update test with todos * update test with todos * Drop underscore in name * Update packages/datadog-plugin-aws-sdk/test/bedrock.spec.js Co-authored-by: Sam Brenner <[email protected]> * Constants normalization * Add Mistral AI * Add aws bedrock rec * remove file * added tests with mocked responses * added jamba support to AI21 lab * update bedrock version * Update tests * remove only * Update response extractions to only pick up first completion/generation * Update packages/datadog-plugin-aws-sdk/src/services/bedrockruntime.js Co-authored-by: Sam Brenner <[email protected]> * Change from constants to a struct for model provider * format * switch case * Add classes for generations and requestParams * Make constructors name object based. and stringify prompt if it's not a string * stringify message if it's not a string * es lint * fix bad variable name * add extra tags * camelCase and lint * camelCase and lint --------- Co-authored-by: Sam Brenner <[email protected]>
Closed
watson
pushed a commit
that referenced
this pull request
Jan 22, 2025
* Initial APM side for aws bedrock * add extract response tags * add extract response tags * remove hook * Update packages/datadog-plugin-aws-sdk/src/services/bedrockruntime.js * added example test for invoke amazon * added example test for invoke amazon * update test with todos * update test with todos * Drop underscore in name * Update packages/datadog-plugin-aws-sdk/test/bedrock.spec.js Co-authored-by: Sam Brenner <[email protected]> * Constants normalization * Add Mistral AI * Add aws bedrock rec * remove file * added tests with mocked responses * added jamba support to AI21 lab * update bedrock version * Update tests * remove only * Update response extractions to only pick up first completion/generation * Update packages/datadog-plugin-aws-sdk/src/services/bedrockruntime.js Co-authored-by: Sam Brenner <[email protected]> * Change from constants to a struct for model provider * format * switch case * Add classes for generations and requestParams * Make constructors name object based. and stringify prompt if it's not a string * stringify message if it's not a string * es lint * fix bad variable name * add extra tags * camelCase and lint * camelCase and lint --------- Co-authored-by: Sam Brenner <[email protected]>
Merged
watson
pushed a commit
that referenced
this pull request
Jan 23, 2025
* Initial APM side for aws bedrock * add extract response tags * add extract response tags * remove hook * Update packages/datadog-plugin-aws-sdk/src/services/bedrockruntime.js * added example test for invoke amazon * added example test for invoke amazon * update test with todos * update test with todos * Drop underscore in name * Update packages/datadog-plugin-aws-sdk/test/bedrock.spec.js Co-authored-by: Sam Brenner <[email protected]> * Constants normalization * Add Mistral AI * Add aws bedrock rec * remove file * added tests with mocked responses * added jamba support to AI21 lab * update bedrock version * Update tests * remove only * Update response extractions to only pick up first completion/generation * Update packages/datadog-plugin-aws-sdk/src/services/bedrockruntime.js Co-authored-by: Sam Brenner <[email protected]> * Change from constants to a struct for model provider * format * switch case * Add classes for generations and requestParams * Make constructors name object based. and stringify prompt if it's not a string * stringify message if it's not a string * es lint * fix bad variable name * add extra tags * camelCase and lint * camelCase and lint --------- Co-authored-by: Sam Brenner <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
This PR adds support for AWS Bedrock through the InvokeModel function.
Adds initial instrumentation support for client-bedrock-runtime >=3.422.0, specifically for:
InvokeModelMotivation
Tracing the Bedrock model invocation calls would help our customers get insights with APM and the LLM Observability product on how they use this service.
Plugin Checklist
Additional Notes