Feature/test installed package by ClausKlein · Pull Request #173 · bemanproject/exemplar

ClausKlein · 2025-06-01T20:53:56Z

Description

Use ctest --build-and-test to check the installed version

Related Issues

Motivation and Context

Every exported cmake config package must be checked for its usability.

Testing

cmake -S . -B build -DCMAKE_CXX_STANDARD=20 -DCMAKE_BUILD_TYPE=Release --fresh
ninja -c build install
ninja -c build test

Meta

see https://discourse.cmake.org/t/is-beman-exemplar-beman-exemplar-version-cmake-a-vallid-package-name/14188

If all approvals are obtained and the PR is green, any Beman member can merge the PR.

wusatosi

I'm very much for testing installation.
This clashes with #122 on functionality, but that PR has not been updated in a while.
@nickelpro can you check if this approach is good?

ClausKlein · 2025-06-02T10:36:52Z

see #172

nickelpro · 2025-06-02T13:18:43Z

Yep, you beat me to it, version file name is wrong

ClausKlein · 2025-06-03T15:56:26Z

@camio @wusatosi What is planed with this feature?

ClausKlein · 2025-07-01T20:23:11Z

@camio @dietmarkuehl @ednolan Is it possible to review and merge this MR?

It was ready since weeks, I do not want to rebase it again!

dietmarkuehl

I'm not cmake expert at all! Mostly the changes look OK to me. I believe such a test is present on beman.execution and it takes more time than all other tests together, significantly slowing down my development: I build often and something which takes multiple seconds is annoying.

I don't see a way to disabe this test without modifying the CMakeLists.txt (I haven't tried it, though).

ClausKlein · 2025-07-06T08:59:58Z

I don't see a way to disabe this test without modifying the CMakeLists.txt (I haven't tried it, though).

Good point!
We may disable it if BEMAN_EXEMPLAR_INSTALL_CONFIG_FILE_PACKAGE is set to FLASE.

ednolan · 2025-07-06T16:16:44Z

FYI, Claus: You can test cookiecutter locally by running exemplar/cookiecutter/check_cookiecutter.sh before pushing.

ClausKlein · 2025-07-06T17:03:45Z

@ednolan after rebase ci fails, what dit you do?

ednolan · 2025-07-06T17:17:35Z

I merged a refactor of the CI system.

It made various changes, but I think the change that appears to have broken things here is that, previously, CI would build with Ninja Multi-Config, and it would always build Debug before building Release. Now, it still uses Ninja Multi-Config, but it only builds either just Debug or just Release.

I don't know why that would make a difference, but I think that's the culprit because it looks like all the CI failures here are in one of two categories:

Release builds with GCC or Clang
Debug builds with MSVC

I'm continuing to look into this.

ClausKlein · 2025-07-06T17:29:03Z

It made various changes, but I think the change that appears to have broken things here is that, previously, CI would build with Ninja Multi-Config, and it would always build Debug before building Release. Now, it still uses Ninja Multi-Config, but it only builds either just Debug or just Release.

I got it.

We need
CTEST_OUTPUT_ON_FAILURE

on CI set!

ednolan · 2025-07-06T17:31:47Z

I've managed to reproduce thie issue locally by launching one of the new images with the following command:

docker run -it --rm -v $PWD:$PWD -u $(id -u):$(id -g) ghcr.io/bemanproject/infra-containers-gcc:15.1.0 bash

And running the following commands:

cmake -G'Ninja Multi-Config' -B build -S . -DCMAKE_CXX_STANDARD=20 -DCMAKE_TOOLCHAIN_FILE=./infra/cmake/gnu-toolchain.cmake -DCMAKE_PROJECT_TOP_LEVEL_INCLUDES=./infra/cmake/use-fetch-content.cmake
cmake --build build --config Release --parallel --verbose
cmake --build build --config Release --target all_verify_interface_header_sets
cmake --install build --config Release --prefix /tmp/beman.package
ctest --test-dir build --build-config Release
ctest --test-dir build --build-config Release --rerun-failed --output-on-failure

This produced the following output:

Test project /home/eddie/sync/cpp/tests/ClausKlein/exemplar/build
    Start 5: install-to-stagedir
1/2 Test #5: install-to-stagedir ..............***Failed    0.01 sec
CMake Error at /home/eddie/sync/cpp/tests/ClausKlein/exemplar/build/src/beman/exemplar/cmake_install.cmake:47 (file):
  file INSTALL cannot find
  "/home/eddie/sync/cpp/tests/ClausKlein/exemplar/build/src/beman/exemplar/Debug/libbeman.exemplar.a":
  No such file or directory.
Call Stack (most recent call first):
  /home/eddie/sync/cpp/tests/ClausKlein/exemplar/build/cmake_install.cmake:47 (include)



    Start 6: find-package-test
2/2 Test #6: find-package-test ................***Failed    0.02 sec
Internal cmake changing into directory: /home/eddie/sync/cpp/tests/ClausKlein/exemplar/build/tests/beman/exemplar/find-package-test
======== CMake output     ======
CMake Error at CMakeLists.txt:21 (find_package):
  By not providing "Findbeman.exemplar.cmake" in CMAKE_MODULE_PATH this
  project has asked CMake to find a package configuration file provided by
  "beman.exemplar", but CMake did not find one.

  Could not find a package configuration file provided by "beman.exemplar"
  (requested version 2.1.1) with any of the following names:

    beman.exemplarConfig.cmake
    beman.exemplar-config.cmake

  Add the installation prefix of "beman.exemplar" to CMAKE_PREFIX_PATH or set
  "beman.exemplar_DIR" to a directory containing one of the above files.  If
  "beman.exemplar" provides a separate development package or SDK, be sure it
  has been installed.


Configuring incomplete, errors occurred!
======== End CMake output ======
Error: cmake execution failed


0% tests passed, 2 tests failed out of 2

Total Test time (real) =   0.03 sec

The following tests FAILED:
          5 - install-to-stagedir (Failed)
          6 - find-package-test (Failed)
Errors while running CTest

Note specifically in:

  file INSTALL cannot find
  "/home/eddie/sync/cpp/tests/ClausKlein/exemplar/build/src/beman/exemplar/Debug/libbeman.exemplar.a":

It's looking for a directory called Debug despite the fact that I specified Release.

I'll push an update to CI that enables CTEST_OUTPUT_ON_FAILURE.

nickelpro · 2025-07-06T22:31:31Z

      "hidden": true,
      "generator": "Ninja",
      "binaryDir": "${sourceDir}/build/${presetName}",
+      "installDir": "${sourceDir}/stagedir",


Now that the tests don't assume an installDir being set by the preset, we should not be mandating the installDir here

nickelpro · 2025-07-06T22:31:58Z

    {
      "name": "_root-build",
      "hidden": true,
+      "targets": [


Ditto, mandating targets is no longer required so we shouldn't

nickelpro · 2025-07-06T22:44:00Z

+    VERSION 2.1.1
+)
+
+if(PROJECT_IS_TOP_LEVEL)


The example code should not be used for testing. This code serves no purpose for a user who is copy-pasting the example into their own source directory. It serves only to confuse them.

The test should be a five line project that calls cmake_minimum_required() -> project() -> find_package(REQUIRED) -> if(NOT TARGET). It should not be an example. It should exist solely for the purpose of testing the imports.

nickelpro · 2025-07-06T22:46:56Z

 /compile_commands.json
-/build
+/CMakeFiles
+/stagedir


These shouldn't exist outside the build directory anyway, no need to add them here

ClausKlein · 2025-07-07T10:58:31Z

I am the point to stop my support of this project. It takes to much time and to much discussions for me!

ClausKlein requested review from bretbrownjr, camio, dietmarkuehl, neatudarius, steve-downey and wusatosi as code owners June 1, 2025 20:53

ClausKlein requested a review from a team June 1, 2025 20:53

ClausKlein marked this pull request as draft June 1, 2025 20:54

ClausKlein mentioned this pull request Jun 1, 2025

started to update the project to the current Beman standard bemanproject/execution#151

Closed

wusatosi reviewed Jun 1, 2025

View reviewed changes

Comment thread CMakeLists.txt Outdated

wusatosi reviewed Jun 1, 2025

View reviewed changes

ClausKlein force-pushed the feature/test-installed-package branch from 6d2cffa to 9b3db9d Compare June 2, 2025 06:08

ClausKlein marked this pull request as ready for review June 2, 2025 06:26

ClausKlein commented Jun 2, 2025

View reviewed changes

Comment thread examples/CMakeLists.txt Outdated

ClausKlein commented Jun 2, 2025

View reviewed changes

Comment thread examples/CMakeLists.txt

ClausKlein force-pushed the feature/test-installed-package branch from 55f0336 to b35342d Compare June 2, 2025 10:32

ClausKlein requested a review from wusatosi June 2, 2025 10:40

ClausKlein force-pushed the feature/test-installed-package branch from cc3f595 to 01a5be1 Compare June 3, 2025 16:01

nickelpro reviewed Jun 3, 2025

View reviewed changes

Comment thread examples/CMakeLists.txt

nickelpro reviewed Jun 3, 2025

View reviewed changes

Comment thread examples/CMakeLists.txt Outdated

nickelpro reviewed Jun 3, 2025

View reviewed changes

Comment thread examples/CMakeLists.txt Outdated

nickelpro reviewed Jun 3, 2025

View reviewed changes

Comment thread cmake/appleclang-toolchain.cmake

ClausKlein mentioned this pull request Jun 3, 2025

Add Test for Basic CMake Config-File Package #122

Closed

ClausKlein requested a review from nickelpro June 3, 2025 17:33

ClausKlein force-pushed the feature/test-installed-package branch 3 times, most recently from 45f6a25 to 5b8132b Compare June 4, 2025 16:56

ClausKlein added 3 commits July 1, 2025 17:36

sync cookiecutter CMakeLists.txt

809bace

sync cookiecutter files with current project

e2af89a

Useless cookiecutter merge with current project

d273e04

dietmarkuehl reviewed Jul 5, 2025

View reviewed changes

ClausKlein and others added 2 commits July 6, 2025 11:18

Test installed version only if installing is enabled

d44330d

Merge branch 'main' into feature/test-installed-package

cb4a036

github-actions Bot reviewed Jul 6, 2025

View reviewed changes

Comment thread .gitignore Outdated

ClausKlein added 6 commits July 6, 2025 15:28

Add missing newline

4a2cc14

Update gersemi to v0.20.0

4d70f99

sync the cookiecutter with newline too

272f56f

pre-commit autoupdate

bdd6729

back to gersemi v0.20.0

69a5ffc

It is a nightmare with cookiecutter

e75561e

ClausKlein requested review from JeffGarland and dietmarkuehl July 6, 2025 14:34

ClausKlein added 2 commits July 6, 2025 18:49

Merge branch 'main' into feature/test-installed-package

a3bc8a2

Merge branch 'main' into feature/test-installed-package

025f2bb

Use --config for install-to-stagedir

aa8bc04

ClausKlein and others added 2 commits July 6, 2025 19:32

And again the cookicutter ...

a4fcc94

Merge branch 'main' into feature/test-installed-package

11cf1af

nickelpro reviewed Jul 6, 2025

View reviewed changes

ClausKlein closed this Jul 7, 2025

Conversation

ClausKlein commented Jun 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Related Issues

Motivation and Context

Testing

Meta

Uh oh!

Uh oh!

wusatosi left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

ClausKlein commented Jun 2, 2025

Uh oh!

nickelpro commented Jun 2, 2025

Uh oh!

ClausKlein commented Jun 3, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ClausKlein commented Jul 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dietmarkuehl left a comment

Choose a reason for hiding this comment

Uh oh!

ClausKlein commented Jul 6, 2025

Uh oh!

Uh oh!

ednolan commented Jul 6, 2025

Uh oh!

ClausKlein commented Jul 6, 2025

Uh oh!

ednolan commented Jul 6, 2025

Uh oh!

ClausKlein commented Jul 6, 2025

Uh oh!

ednolan commented Jul 6, 2025

Uh oh!

nickelpro Jul 6, 2025

Choose a reason for hiding this comment

Uh oh!

nickelpro Jul 6, 2025

Choose a reason for hiding this comment

Uh oh!

nickelpro Jul 6, 2025

Choose a reason for hiding this comment

Uh oh!

nickelpro Jul 6, 2025

Choose a reason for hiding this comment

Uh oh!

ClausKlein Jul 7, 2025

Choose a reason for hiding this comment

Uh oh!

ClausKlein commented Jul 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

I am the point to stop my support of this project. It takes to much time and to much discussions for me!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

ClausKlein commented Jun 1, 2025 •

edited

Loading

ClausKlein commented Jul 1, 2025 •

edited

Loading

ClausKlein commented Jul 7, 2025 •

edited

Loading