Use in-tree Cabal library for `cabal-install` tests with custom setup. by mpickering · Pull Request #9671 · haskell/cabal

mpickering · 2024-01-30T12:27:54Z

Allow using different Cabal library versions for cabal-install tests with custom setup.

The idea here is to pass a `--package-db` flag to `cabal-install` which
contains just `Cabal` and `Cabal-syntax` of the specific version. This
allows `cabal-install` tests to use the in-tree `Cabal` version,
something which you can easily run into and get very confused about when
writing tests.

There are a few options which can be passed to `cabal-tests` executable
to control which Cabal library you will test against.

1. --boot-cabal-lib specifies to use the Cabal library bundled with the
   test compiler, this is the default and existing behaviour of the
   testsuite.
2. --intree-cabal-lib=<root_dir> specifies to use Cabal and Cabal-syntax
   from a specific directory, and `--test-tmp` indicates where to put
   the package database they are built with.
3. --specific-cabal-lib=<VERSION> specifies to use a specific Cabal
   version from hackage (ie 3.10.2.0) and installs the package database
   into --test-tmp=<DIR>

The end result is that changes in the Cabal library can be tested with
cabal-install tests in the testsuite.

There have been a number of confusing issues with people writing tests
for changes in the Cabal library which never ran because of
cabal-install tests always used the boot Cabal library (see #9425
for one).

Fixes #9681

Mikolaj · 2024-02-22T20:16:05Z

This implements #9681.

Mikolaj · 2024-02-22T20:16:44Z

BTW is this ready for review (I don't see the label)?

mpickering · 2024-02-23T09:54:09Z

I have been concentrating so far on getting CI working for the normal CI configurations, which it now does.

There are some further things I want to work on:

Adding some more configurations of testing different Cabal versions to CI (ie 9.8.1 with Cabal 3.10.1.0 rather than 3.11.0.0)
Going through the testsuite and checking that tests which query GHC version actually mean to query the GHC version or instead mean to query the Cabal library version.

But I think it would be good to merge this firstly.

Mikolaj · 2024-02-23T09:55:20Z

I see. Great. That's normally signalled by setting the review_needed labels, which is why I ask. Let me set it instead.

Mikolaj

That looks great. However, is there a possibility this breaks somebody's workflow? If so, can we mitigate that, give some porting hints, deprecate things in advance? If not, why?

mpickering · 2024-02-23T11:22:29Z

I think by default, no there isn't. Because if you run the tests locally then the same behaviour will happen as before (using the boot Cabal version rather than the in-tree one). Perhaps that is confusing in itself.

You have to explicitly pass a flag to cabal-tests in order to get it to use a different Cabal version than the one you already have. You are right that I still need to document this feature in the documentation as well. I will try to do that this afternoon.

mpickering · 2024-02-23T15:51:43Z

I added some documentation now so I think this first step is ready to merge after #9671 is merged.

ulysses4ever · 2024-02-23T15:54:32Z

synopsis: Clarify the semantics of the -package-db flag

Is it possible to update the PR title and description in accordance with the actual changes?

ulysses4ever · 2024-02-23T15:55:26Z

I added some documentation now so I think this first step is ready to merge after #9671 is merged.

This is #9671.

mpickering · 2024-02-25T17:03:40Z

I have rebased and this patch is ready to merge.

alt-romes

This is a great improvement to the testsuite. I've been very confused myself regarding changes to Cabal not working as intended in the testsuite because of this (and going mad...). Thanks!

ulysses4ever

Terrific, thanks!

Mikolaj · 2024-02-29T19:29:15Z

I'm afraid there are conflicts to solve before this can auto-merge.

…s with custom setup. The idea here is to pass a `--package-db` flag to `cabal-install` which contains just `Cabal` and `Cabal-syntax` of the specific version. This allows `cabal-install` tests to use the in-tree `Cabal` version, something which you can easily run into and get very confused about when writing tests. There are a few options which can be passed to `cabal-tests` executable to control which Cabal library you will test against. 1. --boot-cabal-lib specifies to use the Cabal library bundled with the test compiler, this is the default and existing behaviour of the testsuite. 2. --intree-cabal-lib=<root_dir> specifies to use Cabal and Cabal-syntax from a specific directory, and `--test-tmp` indicates where to put the package database they are built with. 3. --specific-cabal-lib=<VERSION> specifies to use a specific Cabal version from hackage (ie 3.10.2.0) and installs the package database into --test-tmp=<DIR> The end result is that changes in the Cabal library can be tested with cabal-install tests in the testsuite. There have been a number of confusing issues with people writing tests for changes in the Cabal library which never ran because of cabal-install tests always used the boot Cabal library (see haskell#9425 for one). Fixes haskell#9681

Kleidukos · 2024-05-13T14:52:57Z

@Mergifyio backport 3.12

mergify · 2024-05-13T14:53:04Z

backport 3.12

✅ Backports have been created

Details

#9999 Use in-tree Cabal library for cabal-install tests with custom setup. (backport #9671) has been created for branch 3.12

Use in-tree Cabal library for `cabal-install` tests with custom setup. (backport #9671)

mpickering force-pushed the wip/cabal-install-Cabal branch from e73d236 to 93bd6bf Compare February 1, 2024 17:42

mpickering changed the title ~~WIP: Use in-tree Cabal library for cabal-install tests with custom setup.~~ Use in-tree Cabal library for cabal-install tests with custom setup. Feb 1, 2024

mpickering force-pushed the wip/cabal-install-Cabal branch from 93bd6bf to 8ee4abe Compare February 2, 2024 09:43

mpickering force-pushed the wip/cabal-install-Cabal branch 8 times, most recently from 30e8104 to 0a364fb Compare February 22, 2024 15:06

Mikolaj added the attention: needs-review label Feb 23, 2024

Mikolaj requested changes Feb 23, 2024

View reviewed changes

mpickering force-pushed the wip/cabal-install-Cabal branch 2 times, most recently from a873ea8 to feb0f68 Compare February 25, 2024 17:03

mpickering requested a review from ulysses4ever February 26, 2024 09:56

alt-romes approved these changes Feb 27, 2024

View reviewed changes

ulysses4ever approved these changes Feb 27, 2024

View reviewed changes

alt-romes added merge me Tell Mergify Bot to merge and removed attention: needs-review labels Feb 27, 2024

mergify bot added the merge delay passed Applied (usually by Mergify) when PR approved and received no updates for 2 days label Feb 29, 2024

Mikolaj approved these changes Feb 29, 2024

View reviewed changes

mpickering force-pushed the wip/cabal-install-Cabal branch from feb0f68 to 716b109 Compare March 1, 2024 11:28

mergify bot merged commit a0774dc into haskell:master Mar 1, 2024

geekosaur mentioned this pull request Mar 1, 2024

backport: Find build-tool installed programs before programs in path #9767

Merged

philderbeast mentioned this pull request Mar 7, 2024

Ignore validate generated files #9786

Closed

mergify bot mentioned this pull request May 13, 2024

Use in-tree Cabal library for cabal-install tests with custom setup. (backport #9671) #9999

Merged

Kleidukos added the attention: needs-backport 3.12 label May 13, 2024

mergify bot added a commit that referenced this pull request May 13, 2024

Merge pull request #9999 from haskell/mergify/bp/3.12/pr-9671

d7ce36f

Use in-tree Cabal library for `cabal-install` tests with custom setup. (backport #9671)

ulysses4ever mentioned this pull request May 29, 2024

CI: add GHC 9.10 (backport #9914) #10058

Merged

2 tasks

geekosaur removed the attention: needs-backport 3.12 label Jun 7, 2024

Conversation

mpickering commented Jan 30, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Mikolaj commented Feb 22, 2024

Uh oh!

Mikolaj commented Feb 22, 2024

Uh oh!

mpickering commented Feb 23, 2024

Uh oh!

Mikolaj commented Feb 23, 2024

Uh oh!

Mikolaj left a comment

Choose a reason for hiding this comment

Uh oh!

mpickering commented Feb 23, 2024

Uh oh!

mpickering commented Feb 23, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ulysses4ever commented Feb 23, 2024

Uh oh!

ulysses4ever commented Feb 23, 2024

Uh oh!

mpickering commented Feb 25, 2024

Uh oh!

alt-romes left a comment

Choose a reason for hiding this comment

Uh oh!

ulysses4ever left a comment

Choose a reason for hiding this comment

Uh oh!

Mikolaj commented Feb 29, 2024

Uh oh!

Kleidukos commented May 13, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mergify bot commented May 13, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Backports have been created

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

mpickering commented Jan 30, 2024 •

edited

Loading

mpickering commented Feb 23, 2024 •

edited

Loading

Kleidukos commented May 13, 2024 •

edited

Loading

mergify bot commented May 13, 2024 •

edited

Loading