Skip to content

Commit 865d67e

Browse files
westey-mmoonbox3dmytrostrukcrickmandependabot[bot]
authored
Python: Preb1 merge from main 1 (#10805)
### Description Merging changes from main and resolving merge conflicts ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [ ] The code builds clean without any errors or warnings - [ ] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [ ] All unit tests pass, and I have added new tests where possible - [ ] I didn't break anyone 😄 --------- Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: Vincent Biret <[email protected]> Co-authored-by: Evan Mattson <[email protected]> Co-authored-by: Dmytro Struk <[email protected]> Co-authored-by: Chris <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Tao Chen <[email protected]> Co-authored-by: Roger Barreto <[email protected]> Co-authored-by: Mark Wallace <[email protected]> Co-authored-by: Estefanía Tenorio <[email protected]> Co-authored-by: davidpene <[email protected]> Co-authored-by: ThDuquennoy <[email protected]> Co-authored-by: Thomas DUQUENNOY <[email protected]> Co-authored-by: Eduard van Valkenburg <[email protected]> Co-authored-by: Evan Mattson <[email protected]> Co-authored-by: Rob Emanuele <[email protected]> Co-authored-by: K. Andrew Parker <[email protected]> Co-authored-by: SergeyMenshykh <[email protected]> Co-authored-by: jenfoxbot <[email protected]> Co-authored-by: Ben Thomas <[email protected]> Co-authored-by: Ben Thomas <[email protected]> Co-authored-by: Adit Sheth <[email protected]> Co-authored-by: Adit Sheth <[email protected]> Co-authored-by: ふぁー <[email protected]> Co-authored-by: Vincent Biret <[email protected]> Co-authored-by: David A. Torres <[email protected]> Co-authored-by: Genevieve Warren <[email protected]> Co-authored-by: Atiqur Rahman Foyshal <[email protected]> Co-authored-by: Md. Atiqur Rahman Foyshal <[email protected]> Co-authored-by: Fabian Williams <[email protected]> Co-authored-by: [email protected] <[email protected]> Co-authored-by: Ram.Type-0 <[email protected]> Co-authored-by: Tommy Falgout <[email protected]> Co-authored-by: Gary Tang <[email protected]> Co-authored-by: Eirik Tsarpalis <[email protected]> Co-authored-by: Tommaso Stocchi <[email protected]> Co-authored-by: Chris Rickman <[email protected]> Co-authored-by: Devis Lucato <[email protected]> Co-authored-by: K. Andrew Parker <[email protected]> Co-authored-by: Jose Luis Latorre Millas <[email protected]> Co-authored-by: Carsten Lemm <[email protected]> Co-authored-by: Stephen Toub <[email protected]> Co-authored-by: Musale Martin <[email protected]> Co-authored-by: Ross Smith <[email protected]>
1 parent 10cde20 commit 865d67e

1,091 files changed

Lines changed: 50849 additions & 17275 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/_typos.toml‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,10 @@ extend-exclude = [
2323
"PopulationByCountry.csv",
2424
"PopulationByAdmin1.csv",
2525
"WomensSuffrage.txt",
26-
"SK-dotnet.sln.DotSettings"
26+
"SK-dotnet.sln.DotSettings",
27+
"**/azure_ai_search_hotel_samples/README.md",
28+
"**/Demos/ProcessFrameworkWithAspire/ProcessFramework.Aspire/ProcessFramework.Aspire.ProcessOrchestrator/Program.cs",
29+
"**/Demos/ProcessFrameworkWithAspire/**/*.http"
2730
]
2831

2932
[default.extend-words]

‎.github/workflows/dotnet-build-and-test.yml‎

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -148,11 +148,17 @@ jobs:
148148

149149
# Generate test reports and check coverage
150150
- name: Generate test reports
151-
uses: danielpalme/[email protected].3
151+
uses: danielpalme/[email protected].4
152152
with:
153153
reports: "./TestResults/Coverage/**/coverage.cobertura.xml"
154154
targetdir: "./TestResults/Reports"
155-
reporttypes: "JsonSummary"
155+
reporttypes: "HtmlInline;JsonSummary"
156+
157+
- name: Upload coverage report artifact
158+
uses: actions/upload-artifact@v4
159+
with:
160+
name: CoverageReport-${{ matrix.os }}-${{ matrix.dotnet }}-${{ matrix.configuration }} # Artifact name
161+
path: ./TestResults/Reports # Directory containing files to upload
156162

157163
- name: Check coverage
158164
shell: pwsh
Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
name: Create Issue when Needs Port label is added
2+
on:
3+
issues:
4+
types: [labeled]
5+
pull_request_target:
6+
types: [labeled]
7+
8+
jobs:
9+
create_issue:
10+
if: contains(github.event.pull_request.labels.*.name, 'needs_port_to_dotnet') || contains(github.event.pull_request.labels.*.name, 'needs_port_to_python') || contains(github.event.issue.labels.*.name, 'needs_port_to_dotnet') || contains(github.event.issue.labels.*.name, 'needs_port_to_python')
11+
name: "Create Issue"
12+
continue-on-error: true
13+
runs-on: ubuntu-latest
14+
permissions:
15+
issues: write
16+
pull-requests: read
17+
env:
18+
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
19+
GH_REPO: ${{ github.repository }}
20+
21+
steps:
22+
- name: Create dotnet issue
23+
if: contains(github.event.pull_request.labels.*.name, 'needs_port_to_dotnet') || contains(github.event.issue.labels.*.name, 'needs_port_to_dotnet')
24+
run: |
25+
new_issue_url=$(gh issue create \
26+
--title "Port python feature: ${{ github.event.issue.title || github.event.pull_request.title }}" \
27+
--label ".NET" \
28+
--body "# Original issue
29+
${{ github.event.issue.html_url || github.event.pull_request.html_url }}
30+
## Description
31+
${{ github.event.issue.body || github.event.pull_request.body }}")
32+
- name: Create python issue
33+
if: contains(github.event.pull_request.labels.*.name, 'needs_port_to_python') || contains(github.event.issue.labels.*.name, 'needs_port_to_python')
34+
run: |
35+
new_issue_url=$(gh issue create \
36+
--title "Port dotnet feature: ${{ github.event.issue.title || github.event.pull_request.title }}" \
37+
--label "python" \
38+
--body "# Original issue
39+
${{ github.event.issue.html_url || github.event.pull_request.html_url }}
40+
## Description
41+
${{ github.event.issue.body || github.event.pull_request.body }}")

‎.github/workflows/python-build.yml‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,11 +4,16 @@ on:
44
release:
55
types: [published]
66

7+
permissions:
8+
contents: read
9+
id-token: "write"
10+
711
jobs:
812
python-build-assets:
913
if: github.event_name == 'release' && startsWith(github.event.release.tag_name, 'python-')
1014
name: Python Build Assets and add to Release
1115
runs-on: ubuntu-latest
16+
environment: "integration"
1217
permissions:
1318
contents: write
1419
env:
@@ -21,6 +26,7 @@ jobs:
2126
version: "0.5.x"
2227
enable-cache: true
2328
cache-suffix: ${{ runner.os }}-${{ matrix.python-version }}
29+
cache-dependency-glob: "**/uv.lock"
2430
- name: Check version
2531
run: |
2632
echo "Building and uploading Python package version: ${{ github.event.release.tag_name }}"
@@ -31,3 +37,14 @@ jobs:
3137
with:
3238
files: |
3339
python/dist/*
40+
- name: Azure Login
41+
uses: azure/login@v2
42+
with:
43+
client-id: ${{ secrets.AZURE_CLIENT_ID }}
44+
tenant-id: ${{ secrets.AZURE_TENANT_ID }}
45+
subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}
46+
- name: Start DevOps pipeline
47+
uses: azure/cli@v2
48+
with:
49+
inlineScript: |
50+
az pipelines run --id ${{ vars.ADO_PYTHON_RELEASE_ID }} --org ${{ vars.ADO_ORG }} --project ${{ vars.ADO_PROJECT_NAME }} --parameters tag=${{ github.event.release.tag_name }} delay=0

‎.github/workflows/python-integration-tests.yml‎

Lines changed: 113 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -55,10 +55,10 @@ env:
5555
MISTRALAI_EMBEDDING_MODEL_ID: ${{ vars.MISTRALAI_EMBEDDING_MODEL_ID }}
5656
ANTHROPIC_API_KEY: ${{secrets.ANTHROPIC_API_KEY}}
5757
ANTHROPIC_CHAT_MODEL_ID: ${{ vars.ANTHROPIC_CHAT_MODEL_ID }}
58-
OLLAMA_CHAT_MODEL_ID: "${{ vars.OLLAMA_CHAT_MODEL_ID || '' }}" # llava-phi3
59-
OLLAMA_CHAT_MODEL_ID_IMAGE: "${{ vars.OLLAMA_CHAT_MODEL_ID_IMAGE || '' }}" # llava-phi3
60-
OLLAMA_CHAT_MODEL_ID_TOOL_CALL: "${{ vars.OLLAMA_CHAT_MODEL_ID_TOOL_CALL || '' }}" # llama3.2
61-
OLLAMA_TEXT_MODEL_ID: "${{ vars.OLLAMA_TEXT_MODEL_ID || '' }}" # llava-phi3
58+
OLLAMA_CHAT_MODEL_ID: "${{ vars.OLLAMA_CHAT_MODEL_ID || '' }}" # llama3.2:1b
59+
OLLAMA_CHAT_MODEL_ID_IMAGE: "${{ vars.OLLAMA_CHAT_MODEL_ID_IMAGE || '' }}" # moondream
60+
OLLAMA_CHAT_MODEL_ID_TOOL_CALL: "${{ vars.OLLAMA_CHAT_MODEL_ID_TOOL_CALL || '' }}" # llama3.2:1b
61+
OLLAMA_TEXT_MODEL_ID: "${{ vars.OLLAMA_TEXT_MODEL_ID || '' }}" # llama3.2:1b
6262
OLLAMA_EMBEDDING_MODEL_ID: "${{ vars.OLLAMA_EMBEDDING_MODEL_ID || '' }}" # nomic-embed-text
6363
GOOGLE_AI_GEMINI_MODEL_ID: ${{ vars.GOOGLE_AI_GEMINI_MODEL_ID }}
6464
GOOGLE_AI_EMBEDDING_MODEL_ID: ${{ vars.GOOGLE_AI_EMBEDDING_MODEL_ID }}
@@ -69,6 +69,8 @@ env:
6969
REDIS_CONNECTION_STRING: ${{ vars.REDIS_CONNECTION_STRING }}
7070
AZURE_COSMOS_DB_NO_SQL_URL: ${{ vars.AZURE_COSMOS_DB_NO_SQL_URL }}
7171
AZURE_COSMOS_DB_NO_SQL_KEY: ${{ secrets.AZURE_COSMOS_DB_NO_SQL_KEY }}
72+
BEDROCK_AGENT_AGENT_RESOURCE_ROLE_ARN: ${{ secrets.BEDROCK_AGENT_AGENT_RESOURCE_ROLE_ARN }}
73+
BEDROCK_AGENT_FOUNDATION_MODEL: ${{ vars.BEDROCK_AGENT_FOUNDATION_MODEL }}
7274

7375
jobs:
7476
paths-filter:
@@ -118,6 +120,7 @@ jobs:
118120
version: "0.5.x"
119121
enable-cache: true
120122
cache-suffix: ${{ runner.os }}-${{ matrix.python-version }}
123+
cache-dependency-glob: "**/uv.lock"
121124
- name: Install dependencies
122125
run: |
123126
uv sync --all-extras --dev
@@ -147,10 +150,92 @@ jobs:
147150
run: |
148151
uv run pytest -v --log-cli-level=INFO --durations=20 -n logical --dist loadfile --dist worksteal -m "not ollama" ./tests/integration/completions ./tests/integration/embeddings ./tests/samples ./tests/integration/cross_language
149152
153+
python-merge-gate-multi-modality:
154+
name: Python Pre-Merge Integration Tests - Multi-Modality
155+
needs: paths-filter
156+
if: github.event_name != 'pull_request' && github.event_name != 'schedule' && needs.paths-filter.outputs.pythonChanges == 'true'
157+
strategy:
158+
max-parallel: 1
159+
fail-fast: false
160+
matrix:
161+
python-version: ["3.11"]
162+
os: [ubuntu-latest]
163+
defaults:
164+
run:
165+
working-directory: python
166+
runs-on: ${{ matrix.os }}
167+
environment: "integration"
168+
env:
169+
UV_PYTHON: ${{ matrix.python-version }}
170+
steps:
171+
- uses: actions/checkout@v4
172+
- name: Set up uv
173+
uses: astral-sh/setup-uv@v5
174+
with:
175+
version: "0.5.x"
176+
enable-cache: true
177+
cache-suffix: ${{ runner.os }}-${{ matrix.python-version }}
178+
- name: Install dependencies
179+
run: |
180+
uv sync --all-extras --dev
181+
- name: Azure CLI Login
182+
if: github.event_name != 'pull_request'
183+
uses: azure/login@v2
184+
with:
185+
client-id: ${{ secrets.AZURE_CLIENT_ID }}
186+
tenant-id: ${{ secrets.AZURE_TENANT_ID }}
187+
subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}
188+
- name: Run Integration Tests
189+
id: run_tests_multi_modality
190+
shell: bash
191+
run: |
192+
uv run pytest -v --log-cli-level=INFO --durations=20 -n logical --dist loadfile --dist worksteal ./tests/integration/audio_to_text ./tests/integration/text_to_audio ./tests/integration/text_to_image
193+
194+
python-merge-gate-agents:
195+
name: Python Pre-Merge Integration Tests - Agents
196+
needs: paths-filter
197+
if: github.event_name != 'pull_request' && github.event_name != 'schedule' && needs.paths-filter.outputs.pythonChanges == 'true'
198+
strategy:
199+
max-parallel: 1
200+
fail-fast: false
201+
matrix:
202+
python-version: ["3.11"]
203+
os: [ubuntu-latest]
204+
defaults:
205+
run:
206+
working-directory: python
207+
runs-on: ${{ matrix.os }}
208+
environment: "integration"
209+
env:
210+
UV_PYTHON: ${{ matrix.python-version }}
211+
steps:
212+
- uses: actions/checkout@v4
213+
- name: Set up uv
214+
uses: astral-sh/setup-uv@v5
215+
with:
216+
version: "0.5.x"
217+
enable-cache: true
218+
cache-suffix: ${{ runner.os }}-${{ matrix.python-version }}
219+
- name: Install dependencies
220+
run: |
221+
uv sync --all-extras --dev
222+
- name: Configure AWS Credentials
223+
uses: aws-actions/configure-aws-credentials@v4
224+
with:
225+
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
226+
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
227+
aws-region: ${{ vars.AWS_REGION }}
228+
- name: Run Integration Tests
229+
id: run_tests_agents
230+
shell: bash
231+
run: |
232+
uv run pytest -v --log-cli-level=INFO --durations=20 -n logical --dist loadfile --dist worksteal ./tests/integration/agents
233+
150234
python-merge-gate-ollama:
151235
name: Python Pre-Merge Integration Tests - Ollama
152236
needs: paths-filter
153-
if: github.event_name != 'pull_request' && github.event_name != 'schedule' && needs.paths-filter.outputs.pythonChanges == 'true'
237+
# Ollama tests are very unstable at the moment. It often fails to pull models from the Ollama server. Thus, this job is disabled for now.
238+
if: false && github.event_name != 'pull_request' && github.event_name != 'schedule' && needs.paths-filter.outputs.pythonChanges == 'true'
154239
strategy:
155240
max-parallel: 1
156241
fail-fast: false
@@ -173,6 +258,7 @@ jobs:
173258
version: "0.5.x"
174259
enable-cache: true
175260
cache-suffix: ${{ runner.os }}-${{ matrix.python-version }}
261+
cache-dependency-glob: "**/uv.lock"
176262
- name: Install dependencies
177263
run: |
178264
uv sync --all-extras --dev
@@ -237,6 +323,7 @@ jobs:
237323
version: "0.5.x"
238324
enable-cache: true
239325
cache-suffix: ${{ runner.os }}-${{ matrix.python-version }}
326+
cache-dependency-glob: "**/uv.lock"
240327
- name: Install dependencies
241328
run: |
242329
uv sync --all-extras --dev
@@ -268,7 +355,6 @@ jobs:
268355
run:
269356
working-directory: python
270357
runs-on: ${{ matrix.os }}
271-
environment: "integration"
272358
env:
273359
UV_PYTHON: ${{ matrix.python-version }}
274360
MEMORY_CONCEPT_SAMPLE: "true"
@@ -295,6 +381,7 @@ jobs:
295381
version: "0.5.x"
296382
enable-cache: true
297383
cache-suffix: ${{ runner.os }}-${{ matrix.python-version }}
384+
cache-dependency-glob: "**/uv.lock"
298385
- name: Install dependencies
299386
run: |
300387
uv sync --all-extras --dev
@@ -305,7 +392,8 @@ jobs:
305392
ollama serve &
306393
sleep 5
307394
- name: Pull model in Ollama
308-
if: matrix.os == 'ubuntu-latest'
395+
# Ollama tests are very unstable at the moment. It often fails to pull models from the Ollama server. Thus, Ollama is disabled for now.
396+
if: false && matrix.os == 'ubuntu-latest'
309397
run: |
310398
ollama pull ${{ vars.OLLAMA_CHAT_MODEL_ID }}
311399
ollama pull ${{ vars.OLLAMA_CHAT_MODEL_ID_IMAGE }}
@@ -343,14 +431,16 @@ jobs:
343431
id: run_tests_completions
344432
timeout-minutes: 10
345433
shell: bash
434+
# Ollama tests are very unstable at the moment. It often fails to pull models from the Ollama server. Thus, Ollama is disabled for now.
346435
run: |
347-
uv run pytest -v -n logical --dist loadfile --dist worksteal ./tests/integration/completions
436+
uv run pytest -v -n logical --dist loadfile --dist worksteal -m "not ollama" ./tests/integration/completions
348437
- name: Run Integration Tests - Embeddings
349438
id: run_tests_embeddings
350439
timeout-minutes: 5
351440
shell: bash
441+
# Ollama tests are very unstable at the moment. It often fails to pull models from the Ollama server. Thus, Ollama is disabled for now.
352442
run: |
353-
uv run pytest -v -n logical --dist loadfile --dist worksteal ./tests/integration/embeddings
443+
uv run pytest -v -n logical --dist loadfile --dist worksteal -m "not ollama" ./tests/integration/embeddings
354444
- name: Run Integration Tests - Memory
355445
id: run_tests_memory
356446
timeout-minutes: 5
@@ -375,6 +465,18 @@ jobs:
375465
shell: bash
376466
run: |
377467
uv run pytest -v -n logical --dist loadfile --dist worksteal ./tests/samples
468+
- name: Run Integration Tests - Agents
469+
id: run_tests_agents
470+
timeout-minutes: 5
471+
shell: bash
472+
run: |
473+
uv run pytest -v -n logical --dist loadfile --dist worksteal ./tests/integration/agents
474+
- name: Run Integration Tests - Multi-Modality
475+
id: run_tests_multi_modality
476+
timeout-minutes: 5
477+
shell: bash
478+
run: |
479+
uv run pytest -v -n logical --dist loadfile --dist worksteal ./tests/integration/audio_to_text ./tests/integration/text_to_audio ./tests/integration/text_to_image
378480
379481
# This final job is required to satisfy the merge queue. It must only run (or succeed) if no tests failed
380482
python-integration-tests-check:
@@ -388,6 +490,8 @@ jobs:
388490
python-merge-gate-ai-services,
389491
python-merge-gate-ollama,
390492
python-merge-gate-memory,
493+
python-merge-gate-agents,
494+
python-merge-gate-multi-modality,
391495
python-integration-tests,
392496
]
393497
steps:

‎.github/workflows/python-lint.yml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,7 @@ jobs:
3030
version: "0.5.x"
3131
enable-cache: true
3232
cache-suffix: ${{ runner.os }}-${{ matrix.python-version }}
33+
cache-dependency-glob: "**/uv.lock"
3334
- name: Install the project
3435
run: uv sync --all-extras --dev
3536
- uses: pre-commit/[email protected]
Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
name: Python Start Release on ADO
2+
3+
on:
4+
workflow_dispatch:
5+
inputs:
6+
tag:
7+
description: "Tag to release"
8+
required: true
9+
10+
permissions:
11+
contents: read
12+
id-token: "write"
13+
14+
jobs:
15+
python-build-assets:
16+
name: Trigger ADO Pipeline for Python Release
17+
runs-on: ubuntu-latest
18+
environment: "integration"
19+
steps:
20+
- name: Azure Login
21+
uses: azure/login@v2
22+
with:
23+
client-id: ${{ secrets.AZURE_CLIENT_ID }}
24+
tenant-id: ${{ secrets.AZURE_TENANT_ID }}
25+
subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}
26+
- name: Start DevOps pipeline
27+
uses: azure/cli@v2
28+
with:
29+
inlineScript: |
30+
az pipelines run --id ${{ vars.ADO_PYTHON_RELEASE_ID }} --org ${{ vars.ADO_ORG }} --project ${{ vars.ADO_PROJECT_NAME }} --parameters tag=${{ inputs.tag }} delay=0

‎.github/workflows/python-test-coverage.yml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,7 @@ jobs:
3232
version: "0.5.x"
3333
enable-cache: true
3434
cache-suffix: ${{ runner.os }}-${{ env.UV_PYTHON }}
35+
cache-dependency-glob: "**/uv.lock"
3536
- name: Install the project
3637
run: uv sync --all-extras --dev
3738
- name: Test with pytest

0 commit comments

Comments
 (0)