Skip to content

Commit 977c3ad

Browse files
Python: replace pre-commit with prek, add PEP 723 script deps, clean up dev dependencies (#3748)
* python: replace pre-commit with prek, add PEP 723 script deps, clean up dev dependencies - Replace pre-commit with prek (Rust-native, faster pre-commit alternative) - Move supported hooks to repo: builtin for zero-clone speed - Add new builtin hooks: trailing-whitespace, check-merge-conflict, detect-private-key, check-added-large-files - Update all hook versions to latest (pre-commit-hooks v6, pyupgrade v3.21.2, bandit 1.9.3, uv-pre-commit 0.10.0) - Add PEP 723 inline script metadata to 34 samples with external deps - Remove autogen-agentchat/autogen-ext from dev deps (now declared per-sample) - Remove unused dev deps: pytest-env, tomli-w - Add agent-framework-core>=1.0.0b260130 lower bound to all 21 packages - Update CI workflow to use j178/prek-action - Update docs: DEV_SETUP.md, AGENTS.md, CODING_STANDARD.md, SAMPLE_GUIDELINES.md * updated lock * python: fix prek config paths for local execution and CI workflow Remove global 'files: ^python/' filter and strip python/ prefix from all path patterns in .pre-commit-config.yaml so prek finds files when run from the python/ directory. Update CI workflow to use --cd python instead of --config path. Include trailing whitespace fixes and dev dependency cleanup. * python: move helper scripts to scripts/ folder and exclude from checks * python: exclude AGENTS.md from prek markdown code lint * python: exclude AGENTS.md and azure_ai_search sample from markdown lint * fix m365 sample * python: ignore CPY rule for samples with PEP 723 headers * fix in dev_setup * python: replace aiofiles with regular open in samples * python: suppress reportUnusedImport in markdown code block checker * python: use samples pyright config for markdown code block checker Write a temp pyrightconfig.json matching pyrightconfig.samples.json rules (typeCheckingMode=off, only reportMissingImports and reportAttributeAccessIssue). Filter output to only fail on these rules since syntax-level errors (top-level await, undefined vars) are expected in README documentation snippets. * python: use markdown-code-lint with fixed globs instead of prek file list The prek-markdown-code-lint task received all changed files including non-README markdown and files with pre-existing broken imports. Replace with the standard markdown-code-lint task which uses the correct glob patterns (README.md, packages/**/README.md, samples/**/*.md). * python: exclude READMEs with pre-existing broken imports from markdown lint * python: fix broken README code snippets instead of excluding them - ag-ui: replace TextContent (removed) with content.type == 'text' - durabletask: fix import path to durabletask.worker.TaskHubGrpcWorker - orchestrations: use constructor params instead of .participants() method - observability: mark deprecated code blocks as plain text, filter reportMissingImports to agent_framework modules only - remove README excludes from markdown-code-lint task * add revision to gaia download * feat(python): parallelize checks across packages Run (package × task) cross-product in parallel using ThreadPoolExecutor and subprocesses. Key changes: - Add scripts/task_runner.py with shared parallel execution engine - Update run_tasks_in_packages_if_exists.py to accept multiple tasks - Update run_tasks_in_changed_packages.py with --files flag and parallel support - Add check-packages poe task (fmt+lint+pyright+mypy in parallel) - Add prek-markdown-code-lint and prek-samples-check with change detection - Split CI code quality workflow into parallel prek and mypy jobs - Update DEV_SETUP.md to document new parallel behavior Core package changes still trigger checks on all packages. * feat(ci): split code quality into 4 parallel jobs Split the single prek job into parallel jobs: - pre-commit-hooks: lightweight hooks (SKIP=poe-check) - package-checks: fmt/lint/pyright/mypy via check-packages - samples-markdown: samples-lint, samples-syntax, markdown-code-lint - mypy: change-detected mypy checks All 4 jobs run concurrently (×2 Python versions = 8 runners). * feat(ci): use only Python 3.10 for code quality checks * refactor(python): add future annotations and remove quoted types Add `from __future__ import annotations` to 93 package files that used quoted string annotations, then run pyupgrade --py310-plus to remove the now-unnecessary quotes. Fixes #3578
1 parent ad0dac3 commit 977c3ad

177 files changed

Lines changed: 1374 additions & 1011 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/workflows/python-code-quality.yml‎

Lines changed: 99 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -12,13 +12,13 @@ env:
1212
UV_CACHE_DIR: /tmp/.uv-cache
1313

1414
jobs:
15-
pre-commit:
16-
name: Checks
15+
pre-commit-hooks:
16+
name: Pre-commit Hooks
1717
if: "!cancelled()"
1818
strategy:
1919
fail-fast: false
2020
matrix:
21-
python-version: ["3.10", "3.14"]
21+
python-version: ["3.10"]
2222
runs-on: ubuntu-latest
2323
continue-on-error: true
2424
defaults:
@@ -37,16 +37,106 @@ jobs:
3737
python-version: ${{ matrix.python-version }}
3838
os: ${{ runner.os }}
3939
env:
40-
# Configure a constant location for the uv cache
4140
UV_CACHE_DIR: /tmp/.uv-cache
4241
- uses: actions/cache@v5
4342
with:
44-
path: ~/.cache/pre-commit
45-
key: pre-commit|${{ matrix.python-version }}|${{ hashFiles('python/.pre-commit-config.yaml') }}
46-
- uses: pre-commit/[email protected]
47-
name: Run Pre-Commit Hooks
43+
path: ~/.cache/prek
44+
key: prek|${{ matrix.python-version }}|${{ hashFiles('python/.pre-commit-config.yaml') }}
45+
- uses: j178/prek-action@v1
46+
name: Run Pre-commit Hooks (excluding poe-check)
47+
env:
48+
SKIP: poe-check
4849
with:
49-
extra_args: --config python/.pre-commit-config.yaml --all-files
50+
extra-args: --cd python --all-files
51+
52+
package-checks:
53+
name: Package Checks
54+
if: "!cancelled()"
55+
strategy:
56+
fail-fast: false
57+
matrix:
58+
python-version: ["3.10"]
59+
runs-on: ubuntu-latest
60+
continue-on-error: true
61+
defaults:
62+
run:
63+
working-directory: ./python
64+
env:
65+
UV_PYTHON: ${{ matrix.python-version }}
66+
steps:
67+
- uses: actions/checkout@v6
68+
with:
69+
fetch-depth: 0
70+
- name: Set up python and install the project
71+
id: python-setup
72+
uses: ./.github/actions/python-setup
73+
with:
74+
python-version: ${{ matrix.python-version }}
75+
os: ${{ runner.os }}
76+
env:
77+
UV_CACHE_DIR: /tmp/.uv-cache
78+
- name: Run fmt, lint, pyright in parallel across packages
79+
run: uv run poe check-packages
80+
81+
samples-markdown:
82+
name: Samples & Markdown
83+
if: "!cancelled()"
84+
strategy:
85+
fail-fast: false
86+
matrix:
87+
python-version: ["3.10"]
88+
runs-on: ubuntu-latest
89+
continue-on-error: true
90+
defaults:
91+
run:
92+
working-directory: ./python
93+
env:
94+
UV_PYTHON: ${{ matrix.python-version }}
95+
steps:
96+
- uses: actions/checkout@v6
97+
with:
98+
fetch-depth: 0
99+
- name: Set up python and install the project
100+
id: python-setup
101+
uses: ./.github/actions/python-setup
102+
with:
103+
python-version: ${{ matrix.python-version }}
104+
os: ${{ runner.os }}
105+
env:
106+
UV_CACHE_DIR: /tmp/.uv-cache
107+
- name: Run samples lint
108+
run: uv run poe samples-lint
109+
- name: Run samples syntax check
110+
run: uv run poe samples-syntax
111+
- name: Run markdown code lint
112+
run: uv run poe markdown-code-lint
113+
114+
mypy:
115+
name: Mypy Checks
116+
if: "!cancelled()"
117+
strategy:
118+
fail-fast: false
119+
matrix:
120+
python-version: ["3.10"]
121+
runs-on: ubuntu-latest
122+
continue-on-error: true
123+
defaults:
124+
run:
125+
working-directory: ./python
126+
env:
127+
UV_PYTHON: ${{ matrix.python-version }}
128+
steps:
129+
- uses: actions/checkout@v6
130+
with:
131+
fetch-depth: 0
132+
- name: Set up python and install the project
133+
id: python-setup
134+
uses: ./.github/actions/python-setup
135+
with:
136+
python-version: ${{ matrix.python-version }}
137+
os: ${{ runner.os }}
138+
env:
139+
UV_CACHE_DIR: /tmp/.uv-cache
50140
- name: Run Mypy
51141
env:
52142
GITHUB_BASE_REF: ${{ github.event.pull_request.base.ref || github.base_ref || 'main' }}

‎python/.pre-commit-config.yaml‎

Lines changed: 26 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -1,59 +1,68 @@
1-
files: ^python/
21
fail_fast: true
2+
exclude: ^scripts/
33
repos:
4-
- repo: https://github.com/pre-commit/pre-commit-hooks
5-
rev: v5.0.0
4+
- repo: builtin
65
hooks:
76
- id: check-toml
87
name: Check TOML files
98
files: \.toml$
10-
exclude: ^python/packages/lab/cookiecutter-agent-framework-lab/
9+
exclude: ^packages/lab/cookiecutter-agent-framework-lab/
1110
- id: check-yaml
1211
name: Check YAML files
1312
files: \.yaml$
1413
- id: check-json
1514
name: Check JSON files
1615
files: \.json$
17-
exclude: ^.*\.vscode\/.*|^python/demos/samples/chatkit-integration/frontend/(tsconfig.*\.json|package-lock\.json)$
16+
exclude: ^.*\.vscode\/.*|^demos/samples/chatkit-integration/frontend/(tsconfig.*\.json|package-lock\.json)$
1817
- id: end-of-file-fixer
1918
name: Fix End of File
2019
files: \.py$
21-
exclude: ^python/packages/lab/cookiecutter-agent-framework-lab/
20+
exclude: ^packages/lab/cookiecutter-agent-framework-lab/
2221
- id: mixed-line-ending
2322
name: Check Mixed Line Endings
2423
files: \.py$
25-
exclude: ^python/packages/lab/cookiecutter-agent-framework-lab/
24+
exclude: ^packages/lab/cookiecutter-agent-framework-lab/
25+
- id: trailing-whitespace
26+
name: Trim Trailing Whitespace
27+
exclude: ^packages/lab/cookiecutter-agent-framework-lab/
28+
- id: check-merge-conflict
29+
name: Check Merge Conflicts
30+
- id: detect-private-key
31+
name: Detect Private Keys
32+
- id: check-added-large-files
33+
name: Check Added Large Files
34+
- repo: https://github.com/pre-commit/pre-commit-hooks
35+
rev: v6.0.0
36+
hooks:
2637
- id: check-ast
2738
name: Check Valid Python Samples
2839
types: ["python"]
29-
exclude: ^python/packages/lab/cookiecutter-agent-framework-lab/
40+
exclude: ^packages/lab/cookiecutter-agent-framework-lab/
3041
- repo: https://github.com/asottile/pyupgrade
31-
rev: v3.20.0
42+
rev: v3.21.2
3243
hooks:
3344
- id: pyupgrade
3445
name: Upgrade Python syntax
3546
args: [--py310-plus]
36-
exclude: ^python/packages/lab/cookiecutter-agent-framework-lab/
47+
exclude: ^packages/lab/cookiecutter-agent-framework-lab/
3748
- repo: local
3849
hooks:
3950
- id: poe-check
4051
name: Run checks through Poe
41-
entry: uv --directory ./python run poe pre-commit-check
52+
entry: uv run poe prek-check
4253
language: system
43-
files: ^python/
4454
- repo: https://github.com/PyCQA/bandit
45-
rev: 1.8.5
55+
rev: 1.9.3
4656
hooks:
4757
- id: bandit
4858
name: Bandit Security Checks
49-
args: ["-c", "python/pyproject.toml"]
59+
args: ["-c", "pyproject.toml"]
5060
additional_dependencies: ["bandit[toml]"]
5161
- repo: https://github.com/astral-sh/uv-pre-commit
5262
# uv version.
53-
rev: 0.7.18
63+
rev: 0.10.0
5464
hooks:
5565
# Update the uv lockfile
5666
- id: uv-lock
5767
name: Update uv lockfile
58-
files: python/pyproject.toml
59-
args: [--project, python]
68+
files: pyproject.toml

‎python/.vscode/tasks.json‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@
99
"command": "uv",
1010
"args": [
1111
"run",
12-
"pre-commit",
12+
"prek",
1313
"run",
1414
"-a"
1515
],

‎python/AGENTS.md‎

Lines changed: 2 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -61,22 +61,9 @@ from agent_framework.azure import AzureOpenAIChatClient, AzureAIAgentClient
6161
- **Comments**: Avoid excessive comments; prefer clear code
6262
- **Formatting**: Format only files you changed, not the entire codebase
6363

64-
## Sample Structure
64+
## Samples
6565

66-
1. Copyright header: `# Copyright (c) Microsoft. All rights reserved.`
67-
2. Required imports
68-
3. Module docstring: `"""This sample demonstrates..."""`
69-
4. Helper functions
70-
5. Main function(s) demonstrating functionality
71-
6. Entry point: `if __name__ == "__main__": asyncio.run(main())`
72-
73-
When modifying samples, update associated README files in the same or parent folders.
74-
75-
### Samples Syntax Checking
76-
77-
Run `uv run poe samples-syntax` to check samples for syntax errors and missing imports from `agent_framework`. This uses a relaxed pyright configuration that validates imports without strict type checking.
78-
79-
Some samples depend on external packages (e.g., `azure.ai.agentserver.agentframework`, `microsoft_agents`) that are not installed in the dev environment. These are excluded in `pyrightconfig.samples.json`. When adding or modifying these excluded samples, add them to the exclude list and manually verify they have no import errors from `agent_framework` packages by temporarily removing them from the exclude list and running the check.
66+
See [samples/SAMPLE_GUIDELINES.md](samples/SAMPLE_GUIDELINES.md) for sample structure, external dependency handling (PEP 723), and syntax checking instructions.
8067

8168
## Package Documentation
8269

‎python/CODING_STANDARD.md‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -264,6 +264,13 @@ After the package has been released and gained a measure of confidence:
264264
2. Add the package to the `[all]` extra in `packages/core/pyproject.toml`
265265
3. Create a provider folder in `agent_framework/` with lazy loading `__init__.py`
266266

267+
### Versioning and Core Dependency
268+
269+
All non-core packages declare a lower bound on `agent-framework-core` (e.g., `"agent-framework-core>=1.0.0b260130"`). Follow these rules when bumping versions:
270+
271+
- **Core version changes**: When `agent-framework-core` is updated with breaking or significant changes and its version is bumped, update the `agent-framework-core>=...` lower bound in every other package's `pyproject.toml` to match the new core version.
272+
- **Non-core version changes**: Non-core packages (connectors, extensions) can have their own versions incremented independently while keeping the existing core lower bound pinned. Only raise the core lower bound if the non-core package actually depends on new core APIs.
273+
267274
### Installation Options
268275

269276
Connectors are distributed as separate packages and are not imported by default in the core package. Users install the specific connectors they need:

‎python/DEV_SETUP.md‎

Lines changed: 25 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -64,11 +64,11 @@ uv venv --python $PYTHON_VERSION
6464
uv sync --dev
6565
# Install all the tools and dependencies
6666
uv run poe install
67-
# Install pre-commit hooks
68-
uv run poe pre-commit-install
67+
# Install prek hooks
68+
uv run poe prek-install
6969
```
7070

71-
Alternatively, you can reinstall the venv, pacakges, dependencies and pre-commit hooks with a single command (but this requires poe in the current env), this is especially useful if you want to switch python versions:
71+
Alternatively, you can reinstall the venv, pacakges, dependencies and prek hooks with a single command (but this requires poe in the current env), this is especially useful if you want to switch python versions:
7272

7373
```bash
7474
uv run poe setup -p 3.13
@@ -144,7 +144,7 @@ To run the same checks that run during a commit and the GitHub Action `Python Co
144144
uv run poe check
145145
```
146146

147-
Ideally you should run these checks before committing any changes, when you install using the instructions above the pre-commit hooks should be installed already.
147+
Ideally you should run these checks before committing any changes, when you install using the instructions above the prek hooks should be installed already.
148148

149149
## Code Coverage
150150

@@ -196,10 +196,10 @@ and then you can run the following tasks:
196196
uv sync --all-extras --dev
197197
```
198198

199-
After this initial setup, you can use the following tasks to manage your development environment. It is advised to use the following setup command since that also installs the pre-commit hooks.
199+
After this initial setup, you can use the following tasks to manage your development environment. It is advised to use the following setup command since that also installs the prek hooks.
200200

201201
#### `setup`
202-
Set up the development environment with a virtual environment, install dependencies and pre-commit hooks:
202+
Set up the development environment with a virtual environment, install dependencies and prek hooks:
203203
```bash
204204
uv run poe setup
205205
# or with specific Python version
@@ -220,36 +220,36 @@ uv run poe venv
220220
uv run poe venv --python 3.12
221221
```
222222

223-
#### `pre-commit-install`
224-
Install pre-commit hooks:
223+
#### `prek-install`
224+
Install prek hooks:
225225
```bash
226-
uv run poe pre-commit-install
226+
uv run poe prek-install
227227
```
228228

229229
### Code Quality and Formatting
230230

231-
Each of the following tasks are designed to run against both the main `agent-framework` package and the extension packages, ensuring consistent code quality across the project.
231+
Each of the following tasks run against both the main `agent-framework` package and the extension packages in parallel, ensuring consistent code quality across the project.
232232

233233
#### `fmt` (format)
234-
Format code using ruff:
234+
Format code using ruff (runs in parallel across all packages):
235235
```bash
236236
uv run poe fmt
237237
```
238238

239239
#### `lint`
240-
Run linting checks and fix issues:
240+
Run linting checks and fix issues (runs in parallel across all packages):
241241
```bash
242242
uv run poe lint
243243
```
244244

245245
#### `pyright`
246-
Run Pyright type checking:
246+
Run Pyright type checking (runs in parallel across all packages):
247247
```bash
248248
uv run poe pyright
249249
```
250250

251251
#### `mypy`
252-
Run MyPy type checking:
252+
Run MyPy type checking (runs in parallel across all packages):
253253
```bash
254254
uv run poe mypy
255255
```
@@ -270,16 +270,22 @@ uv run poe markdown-code-lint
270270

271271
### Comprehensive Checks
272272

273+
#### `check-packages`
274+
Run all package-level quality checks (format, lint, pyright, mypy) in parallel across all packages. This runs the full cross-product of (package × check) concurrently:
275+
```bash
276+
uv run poe check-packages
277+
```
278+
273279
#### `check`
274-
Run all quality checks (format, lint, pyright, mypy, test, markdown lint):
280+
Run all quality checks including package checks, samples, tests and markdown lint:
275281
```bash
276282
uv run poe check
277283
```
278284

279285
### Testing
280286

281287
#### `test`
282-
Run unit tests with coverage by invoking the `test` task in each package sequentially:
288+
Run unit tests with coverage by invoking the `test` task in each package in parallel:
283289
```bash
284290
uv run poe test
285291
```
@@ -325,10 +331,10 @@ Publish packages to PyPI:
325331
uv run poe publish
326332
```
327333

328-
## Pre-commit Hooks
334+
## Prek Hooks
329335

330-
Pre-commit hooks run automatically on commit and execute a subset of the checks on changed files only. You can also run all checks using pre-commit directly:
336+
Prek hooks run automatically on commit and execute a subset of the checks on changed files only. Package-level checks (fmt, lint, pyright) run in parallel but only for packages with changed files. Markdown and sample checks are skipped when no relevant files were changed. If the `core` package is changed, all packages are checked. You can also run all checks using prek directly:
331337

332338
```bash
333-
uv run pre-commit run -a
339+
uv run prek run -a
334340
```

‎python/devsetup.sh‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -6,5 +6,5 @@ uv venv --python $PYTHON_VERSION
66
uv sync --dev
77
# Install all the tools and dependencies
88
uv run poe install
9-
# Install pre-commit hooks
10-
uv run poe pre-commit-install
9+
# Install prek hooks
10+
uv run poe prek-install

0 commit comments

Comments
 (0)