Skip to content

Commit 8fd6da2

Browse files
authored
Python: Improve Bedrock service setup docs (#10772)
### Motivation and Context <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> Closing: #9910 ### Description <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> 1. Add detailed instructions on how to setup one's environment to use Bedrock services, including models and agents. 2. Re-enable bedrock integration tests. 3. Some other cleanups. ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [x] The code builds clean without any errors or warnings - [x] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [x] All unit tests pass, and I have added new tests where possible - [x] I didn't break anyone 😄
1 parent 75453a1 commit 8fd6da2

12 files changed

Lines changed: 122 additions & 33 deletions

File tree

‎python/.cspell.json‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -51,6 +51,7 @@
5151
"nopep",
5252
"NOSQL",
5353
"ollama",
54+
"Onnx",
5455
"onyourdatatest",
5556
"OPENAI",
5657
"opentelemetry",

‎python/samples/concepts/agents/bedrock_agent/README.md‎

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,37 @@
55
1. You need to have an AWS account and [access to the foundation models](https://docs.aws.amazon.com/bedrock/latest/userguide/model-access-permissions.html)
66
2. [AWS CLI installed](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html) and [configured](https://boto3.amazonaws.com/v1/documentation/api/latest/guide/quickstart.html#configuration)
77

8+
### Configuration
9+
10+
Follow this [guide](https://boto3.amazonaws.com/v1/documentation/api/latest/guide/quickstart.html#configuration) to configure your environment to use the Bedrock API.
11+
12+
Please configure the `aws_access_key_id`, `aws_secret_access_key`, and `region` otherwise you will need to create custom clients for the services. For example:
13+
14+
```python
15+
runtime_client=boto.client(
16+
"bedrock-runtime",
17+
aws_access_key_id="your_access_key",
18+
aws_secret_access_key="your_secret_key",
19+
region_name="your_region",
20+
[...other parameters you may need...]
21+
)
22+
client=boto.client(
23+
"bedrock",
24+
aws_access_key_id="your_access_key",
25+
aws_secret_access_key="your_secret_key",
26+
region_name="your_region",
27+
[...other parameters you may need...]
28+
)
29+
30+
bedrock_agent = BedrockAgent.create_and_prepare_agent(
31+
name="your_agent_name",
32+
instructions="your_instructions",
33+
runtime_client=runtime_client,
34+
client=client,
35+
[...other parameters you may need...]
36+
)
37+
```
38+
839
## Samples
940

1041
| Sample | Description |

‎python/samples/concepts/setup/ALL_SETTINGS.md‎

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,6 @@
1-
## AI Service Settings used across SK:
1+
# Semantic Kernel Settings
2+
3+
## AI Service Settings used across SK
24

35
| Provider | Service | Constructor Settings | Environment Variable | Required? | Settings Class |
46
| --- | --- | --- | --- | --- | --- |
@@ -36,7 +38,7 @@
3638
| Onnx | [OnnxGenAIChatCompletion](../../../semantic_kernel/connectors/ai/onnx/services/onnx_gen_ai_chat_completion.py) | template, <br> ai_model_path | N/A, <br> ONNX_GEN_AI_CHAT_MODEL_FOLDER | Yes, <br> Yes | [OnnxGenAISettings](../../../semantic_kernel/connectors/ai/onnx/onnx_gen_ai_settings.py) |
3739
| | [OnnxGenAITextCompletion](../../../semantic_kernel/connectors/ai/onnx/services/onnx_gen_ai_text_completion.py) | ai_model_path | ONNX_GEN_AI_TEXT_MODEL_FOLDER | Yes | |
3840

39-
## Memory Service Settings used across SK:
41+
## Memory Service Settings used across SK
4042

4143
| Provider | Service | Constructor Settings | Environment Variable | Required? | Settings Class |
4244
| --- | --- | --- | --- | --- | --- |
@@ -49,7 +51,7 @@
4951
| Redis | [RedisMemoryService](../../../semantic_kernel/connectors/memory/redis/redis_memory_store.py) | connection_string | REDIS_CONNECTION_STRING | Yes | [RedisSettings](../../../semantic_kernel/connectors/memory/redis/redis_settings.py) |
5052
| Weaviate | [WeaviateMemoryService](../../../semantic_kernel/connectors/memory/weaviate/weaviate_memory_store.py) | url, <br> api_key, <br> use_embed | WEAVIATE_URL, <br> WEAVIATE_API_KEY, <br> WEAVIATE_USE_EMBED | No, <br> No, <br> No | [WeaviateSettings](../../../semantic_kernel/connectors/memory/weaviate/weaviate_settings.py) |
5153

52-
## Other settings used:
54+
## Other settings used
5355

5456
| Provider | Service | Constructor Settings | Environment Variable | Required? | Settings Class |
5557
| --- | --- | --- | --- | --- | --- |

‎python/semantic_kernel/connectors/ai/README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,14 +3,14 @@
33
This directory contains the implementation of the AI connectors (aka AI services) that are used to interact with AI models.
44

55
Depending on the modality, the AI connector can inherit from one of the following classes:
6+
67
- [`ChatCompletionClientBase`](./chat_completion_client_base.py) for chat completion tasks.
78
- [`TextCompletionClientBase`](./text_completion_client_base.py) for text completion tasks.
89
- [`AudioToTextClientBase`](./audio_to_text_client_base.py) for audio to text tasks.
910
- [`TextToAudioClientBase`](./text_to_audio_client_base.py) for text to audio tasks.
1011
- [`TextToImageClientBase`](./text_to_image_client_base.py) for text to image tasks.
1112
- [`EmbeddingGeneratorBase`](./embeddings/embedding_generator_base.py) for text embedding tasks.
1213

13-
1414
All base clients inherit from the [`AIServiceClientBase`](../../services/ai_service_client_base.py) class.
1515

1616
## Existing AI connectors

‎python/semantic_kernel/connectors/ai/bedrock/README.md‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,27 @@
1111

1212
Follow this [guide](https://boto3.amazonaws.com/v1/documentation/api/latest/guide/quickstart.html#configuration) to configure your environment to use the Bedrock API.
1313

14+
Please configure the `aws_access_key_id`, `aws_secret_access_key`, and `region` otherwise you will need to create custom clients for the services. For example:
15+
16+
```python
17+
runtime_client=boto.client(
18+
"bedrock-runtime",
19+
aws_access_key_id="your_access_key",
20+
aws_secret_access_key="your_secret_key",
21+
region_name="your_region",
22+
[...other parameters you may need...]
23+
)
24+
client=boto.client(
25+
"bedrock",
26+
aws_access_key_id="your_access_key",
27+
aws_secret_access_key="your_secret_key",
28+
region_name="your_region",
29+
[...other parameters you may need...]
30+
)
31+
32+
bedrock_chat_completion_service = BedrockChatCompletion(runtime_client=runtime_client, client=client)
33+
```
34+
1435
## Supports
1536

1637
### Region

‎python/semantic_kernel/connectors/ai/bedrock/services/bedrock_base.py‎

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,8 @@
44
from functools import partial
55
from typing import Any, ClassVar
66

7+
import boto3
8+
79
from semantic_kernel.kernel_pydantic import KernelBaseModel
810
from semantic_kernel.utils.async_utils import run_in_executor
911

@@ -19,6 +21,26 @@ class BedrockBase(KernelBaseModel, ABC):
1921
# Client: Use for model management
2022
bedrock_client: Any
2123

24+
def __init__(
25+
self,
26+
*,
27+
runtime_client: Any | None = None,
28+
client: Any | None = None,
29+
**kwargs: Any,
30+
) -> None:
31+
"""Initialize the Amazon Bedrock Base Class.
32+
33+
Args:
34+
runtime_client: The Amazon Bedrock runtime client to use.
35+
client: The Amazon Bedrock client to use.
36+
**kwargs: Additional keyword arguments.
37+
"""
38+
super().__init__(
39+
bedrock_runtime_client=runtime_client or boto3.client("bedrock-runtime"),
40+
bedrock_client=client or boto3.client("bedrock"),
41+
**kwargs,
42+
)
43+
2244
async def get_foundation_model_info(self, model_id: str) -> dict[str, Any]:
2345
"""Get the foundation model information."""
2446
response = await run_in_executor(

‎python/semantic_kernel/connectors/ai/bedrock/services/bedrock_chat_completion.py‎

Lines changed: 2 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -5,8 +5,6 @@
55
from functools import partial
66
from typing import TYPE_CHECKING, Any, ClassVar
77

8-
import boto3
9-
108
if sys.version_info >= (3, 12):
119
from typing import override # pragma: no cover
1210
else:
@@ -95,8 +93,8 @@ def __init__(
9593
super().__init__(
9694
ai_model_id=bedrock_settings.chat_model_id,
9795
service_id=service_id or bedrock_settings.chat_model_id,
98-
bedrock_runtime_client=runtime_client or boto3.client("bedrock-runtime"),
99-
bedrock_client=client or boto3.client("bedrock"),
96+
runtime_client=runtime_client,
97+
client=client,
10098
)
10199

102100
# region Overriding base class methods

‎python/semantic_kernel/connectors/ai/bedrock/services/bedrock_text_completion.py‎

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,6 @@
66
from functools import partial
77
from typing import TYPE_CHECKING, Any
88

9-
import boto3
109
from pydantic import ValidationError
1110

1211
if sys.version_info >= (3, 12):
@@ -73,8 +72,8 @@ def __init__(
7372
super().__init__(
7473
ai_model_id=bedrock_settings.text_model_id,
7574
service_id=service_id or bedrock_settings.text_model_id,
76-
bedrock_runtime_client=runtime_client or boto3.client("bedrock-runtime"),
77-
bedrock_client=client or boto3.client("bedrock"),
75+
runtime_client=runtime_client,
76+
client=client,
7877
)
7978

8079
# region Overriding base class methods

‎python/semantic_kernel/connectors/ai/bedrock/services/bedrock_text_embedding.py‎

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,6 @@
66
from functools import partial
77
from typing import TYPE_CHECKING, Any
88

9-
import boto3
109
from numpy import array, ndarray
1110
from pydantic import ValidationError
1211

@@ -70,8 +69,8 @@ def __init__(
7069
super().__init__(
7170
ai_model_id=bedrock_settings.embedding_model_id,
7271
service_id=service_id or bedrock_settings.embedding_model_id,
73-
bedrock_runtime_client=runtime_client or boto3.client("bedrock-runtime"),
74-
bedrock_client=client or boto3.client("bedrock"),
72+
runtime_client=runtime_client,
73+
client=client,
7574
)
7675

7776
@override

‎python/tests/integration/completions/chat_completion_test_base.py‎

Lines changed: 19 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -67,8 +67,6 @@
6767
["ONNX_GEN_AI_CHAT_MODEL_FOLDER"], raise_if_not_set=False
6868
) # Tests are optional for ONNX
6969
anthropic_setup: bool = is_service_setup_for_testing(["ANTHROPIC_API_KEY", "ANTHROPIC_CHAT_MODEL_ID"])
70-
# When testing Bedrock, after logging into AWS CLI this has been set, so we can use it to check if the service is setup
71-
bedrock_setup: bool = is_service_setup_for_testing(["AWS_DEFAULT_REGION"], raise_if_not_set=False)
7270

7371

7472
# A mock plugin that contains a function that returns a complex object.
@@ -90,7 +88,9 @@ class ChatCompletionTestBase(CompletionTestBase):
9088
"""Base class for testing completion services."""
9189

9290
@override
93-
@pytest.fixture(scope="function")
91+
@pytest.fixture(
92+
scope="function"
93+
) # This needs to be scoped to function to avoid resources getting cleaned up after each test
9494
def services(self) -> dict[str, tuple[ServiceType | None, type[PromptExecutionSettings] | None]]:
9595
azure_openai_setup = True
9696
azure_openai_settings = AzureOpenAISettings.create()
@@ -152,27 +152,27 @@ def services(self) -> dict[str, tuple[ServiceType | None, type[PromptExecutionSe
152152
OnnxGenAIPromptExecutionSettings,
153153
),
154154
"bedrock_amazon_titan": (
155-
BedrockChatCompletion(model_id="amazon.titan-text-premier-v1:0") if bedrock_setup else None,
155+
self._try_create_bedrock_chat_completion_client("amazon.titan-text-premier-v1:0"),
156156
BedrockChatPromptExecutionSettings,
157157
),
158158
"bedrock_ai21labs": (
159-
BedrockChatCompletion(model_id="ai21.jamba-1-5-mini-v1:0") if bedrock_setup else None,
159+
self._try_create_bedrock_chat_completion_client("ai21.jamba-1-5-mini-v1:0"),
160160
BedrockChatPromptExecutionSettings,
161161
),
162162
"bedrock_anthropic_claude": (
163-
BedrockChatCompletion(model_id="anthropic.claude-3-5-sonnet-20240620-v1:0") if bedrock_setup else None,
163+
self._try_create_bedrock_chat_completion_client("anthropic.claude-3-5-sonnet-20240620-v1:0"),
164164
BedrockChatPromptExecutionSettings,
165165
),
166166
"bedrock_cohere_command": (
167-
BedrockChatCompletion(model_id="cohere.command-r-v1:0") if bedrock_setup else None,
167+
self._try_create_bedrock_chat_completion_client("cohere.command-r-v1:0"),
168168
BedrockChatPromptExecutionSettings,
169169
),
170170
"bedrock_meta_llama": (
171-
BedrockChatCompletion(model_id="meta.llama3-70b-instruct-v1:0") if bedrock_setup else None,
171+
self._try_create_bedrock_chat_completion_client("meta.llama3-70b-instruct-v1:0"),
172172
BedrockChatPromptExecutionSettings,
173173
),
174174
"bedrock_mistralai": (
175-
BedrockChatCompletion(model_id="mistral.mistral-small-2402-v1:0") if bedrock_setup else None,
175+
self._try_create_bedrock_chat_completion_client("mistral.mistral-small-2402-v1:0"),
176176
BedrockChatPromptExecutionSettings,
177177
),
178178
}
@@ -218,3 +218,13 @@ async def get_chat_completion_response(
218218
if parts:
219219
return sum(parts[1:], parts[0])
220220
raise AssertionError("No response")
221+
222+
def _try_create_bedrock_chat_completion_client(self, model_id: str) -> BedrockChatCompletion | None:
223+
try:
224+
return BedrockChatCompletion(model_id=model_id)
225+
except Exception as ex:
226+
from conftest import logger
227+
228+
logger.warning(ex)
229+
# Returning None so that the test that uses this service will be skipped
230+
return None

0 commit comments

Comments
 (0)