Skip to content

Commit 24fef03

Browse files
Adam Cozzettecopybara-github
Adam Cozzette
authored and
copybara-github
committed
Add support and partial CI coverage for Python 3.12
There's a test run in test_python.yml that is non-trivial to get working with Python 3.12 due to some refactoring of our Docker images that would be needed. But this change updates everything else to add coverage for Python 3.12. The main changes necessary to get the builds working were to upgrade some Pip packages via requirements.txt, including in a patch to `rules_fuzzing` that I plan to upstream soon. I also had to take an explicit dependency on `setuptools`. I removed tox.ini, since it was outdated and we have not been actively maintaining it. PiperOrigin-RevId: 580548224
1 parent 2bcfbd8 commit 24fef03

10 files changed

Lines changed: 22 additions & 36 deletions

File tree

‎.github/workflows/test_python.yml‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,8 @@ jobs:
1717
fail-fast: false # Don't cancel all jobs if one fails.
1818
matrix:
1919
type: [ Pure, C++]
20-
version: ["3.8", "3.9", "3.10", "3.11" ]
20+
# TODO: b/309627662 - Add coverage for Python 3.12.
21+
version: ["3.8", "3.9", "3.10", "3.11"]
2122
include:
2223
- type: Pure
2324
targets: //python/... //python:python_version_test
@@ -55,7 +56,7 @@ jobs:
5556
matrix:
5657
type: [ Pure, C++]
5758
# TODO Consider expanding this set of versions.
58-
version: [ "3.11" ]
59+
version: [ "3.12" ]
5960
include:
6061
- type: Pure
6162
targets: //python/... //python:python_version_test

‎.github/workflows/test_upb.yml‎

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -100,7 +100,6 @@ jobs:
100100
with:
101101
cache: pip
102102
cache-dependency-path: 'python/requirements.txt'
103-
python-version: '3.11' # 3.12 doesn't have setuptools
104103
- name: Run tests
105104
uses: protocolbuffers/protobuf-ci/bazel@v2
106105
with:
@@ -171,23 +170,24 @@ jobs:
171170
# coverage.
172171
- { os: ubuntu-latest, python-version: "3.8", architecture: x64, type: 'binary' }
173172
- { os: macos-11, python-version: "3.8", architecture: x64, type: 'binary' }
174-
- { os: ubuntu-latest, python-version: "3.11", architecture: x64, type: 'binary' }
175-
- { os: macos-12, python-version: "3.11", architecture: x64, type: 'binary' }
173+
- { os: ubuntu-latest, python-version: "3.12", architecture: x64, type: 'binary' }
174+
- { os: macos-12, python-version: "3.12", architecture: x64, type: 'binary' }
176175
- { os: ubuntu-latest, python-version: "3.8", architecture: x64, type: 'source' }
177176
- { os: macos-11, python-version: "3.8", architecture: x64, type: 'source' }
178-
- { os: ubuntu-latest, python-version: "3.11", architecture: x64, type: 'source' }
179-
- { os: macos-12, python-version: "3.11", architecture: x64, type: 'source' }
177+
- { os: ubuntu-latest, python-version: "3.12", architecture: x64, type: 'source' }
178+
- { os: macos-12, python-version: "3.12", architecture: x64, type: 'source' }
180179

181-
# Windows uses the full API up until Python 3.10, so each of these
182-
# jobs tests a distinct binary wheel.
180+
# Windows uses the full API up until Python 3.10.
183181
- { os: windows-2019, python-version: "3.8", architecture: x86, type: 'binary' }
184182
- { os: windows-2019, python-version: "3.9", architecture: x86, type: 'binary' }
185183
- { os: windows-2019, python-version: "3.10", architecture: x86, type: 'binary' }
186184
- { os: windows-2019, python-version: "3.11", architecture: x86, type: 'binary' }
185+
- { os: windows-2019, python-version: "3.12", architecture: x86, type: 'binary' }
187186
- { os: windows-2019, python-version: "3.8", architecture: x64, type: 'binary' }
188187
- { os: windows-2019, python-version: "3.9", architecture: x64, type: 'binary' }
189188
- { os: windows-2019, python-version: "3.10", architecture: x64, type: 'binary' }
190189
- { os: windows-2019, python-version: "3.11", architecture: x64, type: 'binary' }
190+
- { os: windows-2019, python-version: "3.12", architecture: x64, type: 'binary' }
191191
runs-on: ${{ matrix.os }}
192192
if: ${{ github.event_name != 'pull_request_target' }}
193193
defaults:
@@ -248,7 +248,7 @@ jobs:
248248
strategy:
249249
fail-fast: false # Don't cancel all jobs if one fails.
250250
matrix:
251-
python-version: ["3.8", "3.11"]
251+
python-version: ["3.8", "3.12"]
252252
runs-on: ubuntu-latest
253253
if: ${{ github.event_name != 'pull_request_target' }}
254254
steps:

‎python/build_targets.bzl‎

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -437,7 +437,6 @@ def build_targets(name):
437437
"README.md",
438438
"google/__init__.py",
439439
"setup.cfg",
440-
"tox.ini",
441440
],
442441
strip_prefix = "",
443442
visibility = ["//python/dist:__pkg__"],
@@ -463,7 +462,6 @@ def build_targets(name):
463462
"python_version_test.py",
464463
"setup.cfg",
465464
"setup.py",
466-
"tox.ini",
467465
],
468466
strip_prefix = strip_prefix.from_root(""),
469467
visibility = ["//pkg:__pkg__"],

‎python/dist/BUILD.bazel‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,7 @@
55
# license that can be found in the LICENSE file or at
66
# https://developers.google.com/open-source/licenses/bsd
77

8+
load("@pip_deps//:requirements.bzl", "requirement")
89
load("@rules_pkg//:mappings.bzl", "pkg_files", "strip_prefix")
910
load("@rules_pkg//:pkg.bzl", "pkg_tar")
1011
load("@rules_python//python:packaging.bzl", "py_wheel")
@@ -281,6 +282,7 @@ genrule(
281282
srcs = [":source_tarball"],
282283
outs = ["protobuf-%s.tar.gz" % PROTOBUF_PYTHON_VERSION],
283284
cmd = """
285+
export PYTHONPATH=$$PWD/external/pip_deps_setuptools/site-packages
284286
set -eux
285287
tar -xzvf $(location :source_tarball)
286288
cd protobuf/
@@ -292,6 +294,7 @@ genrule(
292294
"@system_python//:none": ["@platforms//:incompatible"],
293295
"//conditions:default": [],
294296
}),
297+
tools = [requirement("setuptools")],
295298
)
296299

297300
py_wheel(
@@ -310,6 +313,7 @@ py_wheel(
310313
"Programming Language :: Python :: 3.9",
311314
"Programming Language :: Python :: 3.10",
312315
"Programming Language :: Python :: 3.11",
316+
"Programming Language :: Python :: 3.12",
313317
],
314318
distribution = "protobuf",
315319
extra_distinfo_files = {
@@ -365,6 +369,7 @@ py_wheel(
365369
"Programming Language :: Python :: 3.9",
366370
"Programming Language :: Python :: 3.10",
367371
"Programming Language :: Python :: 3.11",
372+
"Programming Language :: Python :: 3.12",
368373
],
369374
distribution = "protobuf",
370375
extra_distinfo_files = {

‎python/dist/setup.py‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -58,6 +58,7 @@ def GetVersion():
5858
'Programming Language :: Python :: 3.9',
5959
'Programming Language :: Python :: 3.10',
6060
'Programming Language :: Python :: 3.11',
61+
'Programming Language :: Python :: 3.12',
6162
],
6263
namespace_packages=['google'],
6364
packages=find_packages(),

‎python/protobuf_distutils/setup.py‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,7 @@
3434
'Programming Language :: Python :: 3.9',
3535
'Programming Language :: Python :: 3.10',
3636
'Programming Language :: Python :: 3.11',
37+
'Programming Language :: Python :: 3.12',
3738
'Topic :: Software Development :: Code Generators',
3839
],
3940
description=(

‎python/requirements.txt‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1,2 @@
1-
numpy<=1.24.4
1+
numpy<=1.26.1
2+
setuptools<=68.2.2

‎python/setup.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -389,7 +389,6 @@ def HasLibraryDirsOpt():
389389
])
390390
os.environ['PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION'] = 'cpp'
391391

392-
# Keep this list of dependencies in sync with tox.ini.
393392
install_requires = []
394393

395394
setup(
@@ -415,6 +414,7 @@ def HasLibraryDirsOpt():
415414
'Programming Language :: Python :: 3.9',
416415
'Programming Language :: Python :: 3.10',
417416
'Programming Language :: Python :: 3.11',
417+
'Programming Language :: Python :: 3.12',
418418
],
419419
namespace_packages=['google'],
420420
packages=find_packages(

‎python/tox.ini‎

Lines changed: 0 additions & 21 deletions
This file was deleted.

‎third_party/rules_fuzzing.patch‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -22,4 +22,4 @@ index 01482d4..4b36f4f 100644
2222
-absl-py==0.11.0 --hash=sha256:b3d9eb5119ff6e0a0125f6dabf2f9fae02f8acae7be70576002fac27235611c5
2323
-six==1.15.0 --hash=sha256:8b74bedcbbbaca38ff6d7491d76f2b06b3592611af620f8426e82dddb04a5ced
2424
+absl-py==2.0.0 --hash=sha256:9a28abb62774ae4e8edbe2dd4c49ffcd45a6a848952a5eccc6a49f3f0fc1e2f3
25-
+six==1.15.0
25+
+six==1.16.0 --hash=sha256:8abb2f1d86890a2dfb989f9a77cfcfd3e47c2a354b01111771326f8aa26e0254

0 commit comments

Comments
 (0)