Skip to content

Comments

do_cmake: use Python 3.12 on ubuntu >= 24#61827

Merged
idryomov merged 1 commit intoceph:mainfrom
tchaikov:python3.12-on-ubuntu-noble
Feb 15, 2025
Merged

do_cmake: use Python 3.12 on ubuntu >= 24#61827
idryomov merged 1 commit intoceph:mainfrom
tchaikov:python3.12-on-ubuntu-noble

Conversation

@tchaikov
Copy link
Contributor

@tchaikov tchaikov commented Feb 14, 2025

the "official" Python shipped along with Ubuntu 24.04 (Noble Numbat) is Python 3.12. And some of our building hosts have been upgraded to Ubuntu 24.04. But we are still using Python 3.10 on Ubuntu >= 22, this breaks the build. And CMake fails like:

CMake Error at /usr/share/cmake-3.28/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
  Could NOT find Python3 (missing: Python3_EXECUTABLE Python3_INCLUDE_DIRS
  Python3_LIBRARIES Interpreter Development Development.Module
  Development.Embed) (Required is exact version "3.10")

      Reason given by package:
          Interpreter: Wrong version for the interpreter "/bin/python3"

Call Stack (most recent call first):
  /usr/share/cmake-3.28/Modules/FindPackageHandleStandardArgs.cmake:600 (_FPHSA_FAILURE_MESSAGE)
  cmake/modules/FindPython/Support.cmake:3863 (find_package_handle_standard_args)
  cmake/modules/FindPython3.cmake:545 (include)
  CMakeLists.txt:597 (find_package)

This build failure should happen on all building hosts of Ubuntu >= 24.

In this change, we use Python 3.12 on Ubuntu >= 24

Contribution Guidelines

  • To sign and title your commits, please refer to Submitting Patches to Ceph.

  • If you are submitting a fix for a stable branch (e.g. "quincy"), please refer to Submitting Patches to Ceph - Backports for the proper workflow.

  • When filling out the below checklist, you may click boxes directly in the GitHub web UI. When entering or editing the entire PR message in the GitHub web UI editor, you may also select a checklist item by adding an x between the brackets: [x]. Spaces and capitalization matter when checking off items this way.

Checklist

  • Tracker (select at least one)
    • References tracker ticket
    • Very recent bug; references commit where it was introduced
    • New feature (ticket optional)
    • Doc update (no ticket needed)
    • Code cleanup (no ticket needed)
  • Component impact
    • Affects Dashboard, opened tracker ticket
    • Affects Orchestrator, opened tracker ticket
    • No impact that needs to be tracked
  • Documentation (select at least one)
    • Updates relevant documentation
    • No doc update is appropriate
  • Tests (select at least one)
Show available Jenkins commands
  • jenkins retest this please
  • jenkins test classic perf
  • jenkins test crimson perf
  • jenkins test signed
  • jenkins test make check
  • jenkins test make check arm64
  • jenkins test submodules
  • jenkins test dashboard
  • jenkins test dashboard cephadm
  • jenkins test api
  • jenkins test docs
  • jenkins render docs
  • jenkins test ceph-volume all
  • jenkins test ceph-volume tox
  • jenkins test windows
  • jenkins test rook e2e

the "official" Python shipped along with Ubuntu 24.04 (Noble Numbat) is
Python 3.12. And some of our building have been upgraded to Ubuntu
24.04. But we are still using Python 3.10 on Ubuntu >= 22, this breaks
the build. And CMake fails like:

```
CMake Error at /usr/share/cmake-3.28/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
  Could NOT find Python3 (missing: Python3_EXECUTABLE Python3_INCLUDE_DIRS
  Python3_LIBRARIES Interpreter Development Development.Module
  Development.Embed) (Required is exact version "3.10")

      Reason given by package:
          Interpreter: Wrong version for the interpreter "/bin/python3"

Call Stack (most recent call first):
  /usr/share/cmake-3.28/Modules/FindPackageHandleStandardArgs.cmake:600 (_FPHSA_FAILURE_MESSAGE)
  cmake/modules/FindPython/Support.cmake:3863 (find_package_handle_standard_args)
  cmake/modules/FindPython3.cmake:545 (include)
  CMakeLists.txt:597 (find_package)
```

This build failure should also happen on developers who build Ceph on
Ubuntu >= 24.

In this change, we use Python 3.12 on Ubuntu >= 24

Signed-off-by: Kefu Chai <[email protected]>
@tchaikov tchaikov requested a review from ljflores February 14, 2025 10:59
@tchaikov tchaikov mentioned this pull request Feb 14, 2025
14 tasks
@idryomov
Copy link
Contributor

Python 3.12 gets it much further, but the build fails on several instances of:

FAILED: src/seastar/CMakeFiles/seastar.dir/gen/src/proto/metrics2.pb.cc.o 
/usr/bin/ccache /usr/bin/clang++-14 -DBOOST_ASIO_DISABLE_THREAD_KEYWORD_EXTENSION -DBOOST_ASIO_HAS_IO_URING -DBOOST_ASIO_NO_TS_EXECUTORS -DBOOST_NO_CXX98_FUNCTION_BASE -DHAVE_CONFIG_H -DSEASTAR_API_LEVEL=6 -DSEASTAR_BROKEN_SOURCE_LOCATION -DSEASTAR_DEBUG -DSEASTAR_DEBUG_PROMISE -DSEASTAR_DEBUG_SHARED_PTR -DSEASTAR_DEFAULT_ALLOCATOR -DSEASTAR_DEFERRED_ACTION_REQUIRE_NOEXCEPT -DSEASTAR_HAS_MEMBARRIER -DSEASTAR_HAVE_ASAN_FIBER_SUPPORT -DSEASTAR_HAVE_NUMA -DSEASTAR_HAVE_SYSTEMTAP_SDT -DSEASTAR_LOGGER_TYPE_STDOUT -DSEASTAR_PTHREAD_ATTR_SETAFFINITY_NP -DSEASTAR_SCHEDULING_GROUPS_COUNT=16 -DSEASTAR_SHUFFLE_TASK_QUEUE -DSEASTAR_SSTRING -DSEASTAR_STRERROR_R_CHAR_P -DSEASTAR_THREAD_STACK_GUARDS -DSEASTAR_TYPE_ERASE_MORE -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -D_REENTRANT -D_THREAD_SAFE -D__CEPH__ -D__STDC_FORMAT_MACROS -D__linux__ -I/home/jenkins-build/build/workspace/ceph-pull-requests/build/src/include -I/home/jenkins-build/build/workspace/ceph-pull-requests/src -I/home/jenkins-build/build/workspace/ceph-pull-requests/src/seastar/include -I/home/jenkins-build/build/workspace/ceph-pull-requests/build/src/seastar/gen/include -I/home/jenkins-build/build/workspace/ceph-pull-requests/build/src/seastar/gen/src -I/home/jenkins-build/build/workspace/ceph-pull-requests/src/seastar/src -isystem /home/jenkins-build/build/workspace/ceph-pull-requests/build/boost/include -isystem /home/jenkins-build/build/workspace/ceph-pull-requests/build/include -isystem /home/jenkins-build/build/workspace/ceph-pull-requests/src/jaegertracing/opentelemetry-cpp/api/include -isystem /home/jenkins-build/build/workspace/ceph-pull-requests/src/jaegertracing/opentelemetry-cpp/exporters/jaeger/include -isystem /home/jenkins-build/build/workspace/ceph-pull-requests/src/jaegertracing/opentelemetry-cpp/ext/include -isystem /home/jenkins-build/build/workspace/ceph-pull-requests/src/jaegertracing/opentelemetry-cpp/sdk/include -isystem /home/jenkins-build/build/workspace/ceph-pull-requests/src/xxHash -isystem /home/jenkins-build/build/workspace/ceph-pull-requests/src/fmt/include -g -Werror -std=gnu++2b -fPIC -fno-builtin-malloc -fno-builtin-calloc -fno-builtin-realloc -fno-builtin-free -Wall -fno-strict-aliasing -fsigned-char -Wtype-limits -Wignored-qualifiers -Wpointer-arith -Werror=format-security -Winit-self -Wno-unknown-pragmas -Wnon-virtual-dtor -Wno-ignored-qualifiers -ftemplate-depth-1024 -Wpessimizing-move -Wredundant-move -Wno-inconsistent-missing-override -Wno-mismatched-tags -Wno-unused-private-field -Wno-address-of-packed-member -Wno-unused-function -Wno-unused-local-typedef -Wno-varargs -Wno-gnu-designator -Wno-missing-braces -Wno-parentheses -Wno-deprecated-register -DCEPH_DEBUG_MUTEX -D_GLIBCXX_ASSERTIONS -fdiagnostics-color=auto -U_FORTIFY_SOURCE -Wno-error=unused-result "-Wno-error=#warnings" -fstack-clash-protection -UNDEBUG -Werror -Wimplicit-fallthrough -Wdeprecated -Wno-error=deprecated -fvisibility=hidden -gz -DSEASTAR_NO_EXCEPTION_HACK -Wno-error -Wno-sign-compare -Wno-attributes -Wno-pessimizing-move -Wno-non-virtual-dtor -fsanitize=address -fsanitize=undefined -fno-sanitize=vptr -MD -MT src/seastar/CMakeFiles/seastar.dir/gen/src/proto/metrics2.pb.cc.o -MF src/seastar/CMakeFiles/seastar.dir/gen/src/proto/metrics2.pb.cc.o.d -o src/seastar/CMakeFiles/seastar.dir/gen/src/proto/metrics2.pb.cc.o -c /home/jenkins-build/build/workspace/ceph-pull-requests/build/src/seastar/gen/src/proto/metrics2.pb.cc
In file included from /home/jenkins-build/build/workspace/ceph-pull-requests/build/src/seastar/gen/src/proto/metrics2.pb.cc:4:
In file included from /home/jenkins-build/build/workspace/ceph-pull-requests/build/src/seastar/gen/src/proto/metrics2.pb.h:8:
In file included from /usr/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/string:55:
/usr/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/basic_string.tcc:599:34: error: expected expression
      auto __r = std::move(__op)(auto(__p), auto(__n));
                                 ^
/usr/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/basic_string.tcc:599:45: error: expected expression
      auto __r = std::move(__op)(auto(__p), auto(__n));
                                            ^
2 errors generated.
FAILED: src/seastar/CMakeFiles/seastar.dir/src/core/fair_queue.cc.o 
/usr/bin/ccache /usr/bin/clang++-14 -DBOOST_ASIO_DISABLE_THREAD_KEYWORD_EXTENSION -DBOOST_ASIO_HAS_IO_URING -DBOOST_ASIO_NO_TS_EXECUTORS -DBOOST_NO_CXX98_FUNCTION_BASE -DHAVE_CONFIG_H -DSEASTAR_API_LEVEL=6 -DSEASTAR_BROKEN_SOURCE_LOCATION -DSEASTAR_DEBUG -DSEASTAR_DEBUG_PROMISE -DSEASTAR_DEBUG_SHARED_PTR -DSEASTAR_DEFAULT_ALLOCATOR -DSEASTAR_DEFERRED_ACTION_REQUIRE_NOEXCEPT -DSEASTAR_HAS_MEMBARRIER -DSEASTAR_HAVE_ASAN_FIBER_SUPPORT -DSEASTAR_HAVE_NUMA -DSEASTAR_HAVE_SYSTEMTAP_SDT -DSEASTAR_LOGGER_TYPE_STDOUT -DSEASTAR_PTHREAD_ATTR_SETAFFINITY_NP -DSEASTAR_SCHEDULING_GROUPS_COUNT=16 -DSEASTAR_SHUFFLE_TASK_QUEUE -DSEASTAR_SSTRING -DSEASTAR_STRERROR_R_CHAR_P -DSEASTAR_THREAD_STACK_GUARDS -DSEASTAR_TYPE_ERASE_MORE -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -D_REENTRANT -D_THREAD_SAFE -D__CEPH__ -D__STDC_FORMAT_MACROS -D__linux__ -I/home/jenkins-build/build/workspace/ceph-pull-requests/build/src/include -I/home/jenkins-build/build/workspace/ceph-pull-requests/src -I/home/jenkins-build/build/workspace/ceph-pull-requests/src/seastar/include -I/home/jenkins-build/build/workspace/ceph-pull-requests/build/src/seastar/gen/include -I/home/jenkins-build/build/workspace/ceph-pull-requests/build/src/seastar/gen/src -I/home/jenkins-build/build/workspace/ceph-pull-requests/src/seastar/src -isystem /home/jenkins-build/build/workspace/ceph-pull-requests/build/boost/include -isystem /home/jenkins-build/build/workspace/ceph-pull-requests/build/include -isystem /home/jenkins-build/build/workspace/ceph-pull-requests/src/jaegertracing/opentelemetry-cpp/api/include -isystem /home/jenkins-build/build/workspace/ceph-pull-requests/src/jaegertracing/opentelemetry-cpp/exporters/jaeger/include -isystem /home/jenkins-build/build/workspace/ceph-pull-requests/src/jaegertracing/opentelemetry-cpp/ext/include -isystem /home/jenkins-build/build/workspace/ceph-pull-requests/src/jaegertracing/opentelemetry-cpp/sdk/include -isystem /home/jenkins-build/build/workspace/ceph-pull-requests/src/xxHash -isystem /home/jenkins-build/build/workspace/ceph-pull-requests/src/fmt/include -g -Werror -std=gnu++2b -fPIC -fno-builtin-malloc -fno-builtin-calloc -fno-builtin-realloc -fno-builtin-free -Wall -fno-strict-aliasing -fsigned-char -Wtype-limits -Wignored-qualifiers -Wpointer-arith -Werror=format-security -Winit-self -Wno-unknown-pragmas -Wnon-virtual-dtor -Wno-ignored-qualifiers -ftemplate-depth-1024 -Wpessimizing-move -Wredundant-move -Wno-inconsistent-missing-override -Wno-mismatched-tags -Wno-unused-private-field -Wno-address-of-packed-member -Wno-unused-function -Wno-unused-local-typedef -Wno-varargs -Wno-gnu-designator -Wno-missing-braces -Wno-parentheses -Wno-deprecated-register -DCEPH_DEBUG_MUTEX -D_GLIBCXX_ASSERTIONS -fdiagnostics-color=auto -U_FORTIFY_SOURCE -Wno-error=unused-result "-Wno-error=#warnings" -fstack-clash-protection -UNDEBUG -Werror -Wimplicit-fallthrough -Wdeprecated -Wno-error=deprecated -fvisibility=hidden -gz -DSEASTAR_NO_EXCEPTION_HACK -Wno-error -Wno-sign-compare -Wno-attributes -Wno-pessimizing-move -Wno-non-virtual-dtor -fsanitize=address -fsanitize=undefined -fno-sanitize=vptr -MD -MT src/seastar/CMakeFiles/seastar.dir/src/core/fair_queue.cc.o -MF src/seastar/CMakeFiles/seastar.dir/src/core/fair_queue.cc.o.d -o src/seastar/CMakeFiles/seastar.dir/src/core/fair_queue.cc.o -c /home/jenkins-build/build/workspace/ceph-pull-requests/src/seastar/src/core/fair_queue.cc
In file included from /home/jenkins-build/build/workspace/ceph-pull-requests/src/seastar/src/core/fair_queue.cc:26:
In file included from /usr/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/chrono:45:
In file included from /usr/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/sstream:40:
In file included from /usr/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/istream:40:
In file included from /usr/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/ios:44:
In file included from /usr/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/ios_base.h:41:
In file included from /usr/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/locale_classes.h:40:
In file included from /usr/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/string:55:
/usr/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/basic_string.tcc:599:34: error: expected expression
      auto __r = std::move(__op)(auto(__p), auto(__n));
                                 ^
/usr/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/basic_string.tcc:599:45: error: expected expression
      auto __r = std::move(__op)(auto(__p), auto(__n));
                                            ^
In file included from /home/jenkins-build/build/workspace/ceph-pull-requests/src/seastar/src/core/fair_queue.cc:26:
/usr/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/chrono:2360:48: error: call to consteval function 'std::chrono::hh_mm_ss::_S_fractional_width' is not a constant expression
        static constexpr unsigned fractional_width = {_S_fractional_width()};
                                                      ^
/usr/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/chrono:2360:48: note: undefined function '_S_fractional_width' cannot be used in a constant expression
/usr/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/chrono:2315:2: note: declared here
        _S_fractional_width()
        ^
3 errors generated.

@tchaikov
Copy link
Contributor Author

looks like a bug in clang-14, see https://www.mail-archive.com/[email protected]/msg1918839.html for a bug report with similar symptom. probably, it's time to move on to a new clang. hopefully, #61740 can land sooner.

@idryomov
Copy link
Contributor

jenkins test make check

@idryomov
Copy link
Contributor

jenkins test api

1 similar comment
@idryomov
Copy link
Contributor

jenkins test api

@idryomov idryomov merged commit 92ef6ae into ceph:main Feb 15, 2025
13 of 17 checks passed
@tchaikov tchaikov deleted the python3.12-on-ubuntu-noble branch February 15, 2025 12:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants