Skip to content

extend Version class so that 2.0 > 1.develop > 1.1 and develop > master > head > trunk > 9999#1983

Merged
becker33 merged 7 commits intospack:developfrom
davydden:feature/infinity_version_2
May 2, 2019
Merged

extend Version class so that 2.0 > 1.develop > 1.1 and develop > master > head > trunk > 9999#1983
becker33 merged 7 commits intospack:developfrom
davydden:feature/infinity_version_2

Conversation

@davydden
Copy link
Copy Markdown
Member

@davydden davydden commented Oct 10, 2016

as per @adamjstewart request, an alternative to #1979 which defines a list of infinity-like versions.

@adamjstewart : how do I use that __gt__ operator in places like

for a, b in zip(self.version, other.version)
    return a < b

?

Closes #1975, Closes #8421

@davydden davydden added the WIP label Oct 10, 2016
@davydden davydden changed the title Feature/infinity version 2 extend Version class so that 1.develop > 1.1 (alternative) Oct 10, 2016
@adamjstewart
Copy link
Copy Markdown
Member

@adamjstewart : how do I use that __gt__ operator in places like

for a, b in zip(self.version, other.version)
    return a < b

If you define __lt__, it is implicitly used when you call a < b. So basically, we would need to define all of the rich comparison operators. See https://docs.python.org/2/reference/datamodel.html#object.__lt__ for details.

This is the way Python does ordering internally. Python used to define a single behemoth __cmp__ that returned:

 1 if self  > other
 0 if self == other
-1 if self  < other

but this was deprecated and completely removed in Python 3 because it was deemed not flexible enough: https://www.python.org/dev/peps/pep-0207/

@adamjstewart
Copy link
Copy Markdown
Member

You'll notice that the Version class was already overriding __lt__ for this reason. __lt__ is actually what is called when you try to sort a list, which is why it's the most important. __gt__ is called when finding the max. But if you directly want to do a comparison like self.version > Version('1.2.3'), I think you have to implement __gt__. Please correct me if I'm wrong and it implicitly calls not __lt__ if __gt__ isn't defined.

@adamjstewart
Copy link
Copy Markdown
Member

So you should probably invert my __gt__ and replace __lt__ as it is now incorrect.

@davydden
Copy link
Copy Markdown
Member Author

Almost made it work, just need to figure out 1 failing test...

@davydden davydden force-pushed the feature/infinity_version_2 branch from b8f10dd to 4caf6c0 Compare October 10, 2016 19:09
@davydden davydden removed the WIP label Oct 10, 2016
@davydden
Copy link
Copy Markdown
Member Author

@tgamblin :

I actually like @davydden's and @alalazo's idea to make develop function the same way as a version component as it does as a standalone version (sorry if I'm getting the credit wrong here).

Here is an alternative version which has several infinity-like versions: inf > 2.0 > 1.inf > 1.1 where inf is one of 'develop', 'dev', 'latest', 'master', 'head', 'trunk'. This is essentially @adamjstewart 's tier system.

@davydden davydden changed the title extend Version class so that 1.develop > 1.1 (alternative) extend Version class so that 1.develop > 1.1 (multiple infinity-like versions) Oct 10, 2016
@davydden davydden force-pushed the feature/infinity_version_2 branch from 4caf6c0 to fcaebad Compare October 10, 2016 19:33
@citibeth
Copy link
Copy Markdown
Member

As I said before.... if one magic version number is a problem, then 6 is a bigger problem. I know this is easy to implement, that is not the point.

@davydden
Copy link
Copy Markdown
Member Author

if one magic version number is a problem, then 6 is a bigger problem. I know this is easy to implement, that is not the point.

how exactly is it a bigger problem?

@davydden
Copy link
Copy Markdown
Member Author

@tgamblin @citibeth @alalazo @adamjstewart : I know that we are still to go back to discusion of version handling in Spack after v.0.1.0, but I suggest we merge this PR because in its current form it does not add anything else but 2.0 > 1.develop > 1.0 plus a minor refactoring of Version class to make it easier to read.

@citibeth
Copy link
Copy Markdown
Member

In all our discussion on versions, I realized that versions are a pretty trivial problem: only a finite set of version numbers are used for each package, and they are package-specific. That is... one package might have a version 1.3 while another one does not. So we're left with defining a (currently) complete order on a small and finite set of versions. Should be a trivial problem.

The insight from this was there is no need to spill so much ink over this issue, to get one "universal" way of numbering versions that will work for every package --- because there will never be such a thing. Versions are package-specific, and that needs to be reflected in the API.

I suggested a way to factor out versions that, IMHO, would be not very disruptive; see below from #1997 . I think it is the right first step to do on versions. After that, we can begin defining version schemes beyond the current default; for example, one that uses "develop=infinity" (or make it as a variant on the current scheme). Packages that want to use these new versions schemes can.

In short... I'd like to see versions get factored out, then I'd be happy to see this PR merged as an additional version scheme.

  1. Since version schemes seem to be package-specific, that should be reflected in the Spack API. This will give us the flexibility we need to implement different versioning schemes, partial orders, etc: For example:
    a) Versions should maintain a reference to the Package to which they are related.
    b) New versions are constructed off of a Package (eg: Package.new_version() method).
    c) Comparison of Versions between packages throws an exception.
    d) Comparison of Versions belonging to the same package is delegated to a method on the Package. Eg:
class Version(object):
    def compare(self, other):
        if self.package != other.package:
            raise Exception(....)
        return self.package.compare_versions(self, other)

Of course, delegating this to packages doesn't solve the problem of how version schemes should work for 99% of packages that will use the standard scheme. But it does give us flexibility for the corner cases. More importantly, it allows the standard scheme to do package-specific things (based on, for example, what it finds in package.py or the Git repo).

@citibeth
Copy link
Copy Markdown
Member

This PR will break the current behavior of Spack; as far as I can tell, isdevelop('1.develop') will be False, it should be True.

One other thing about factoring out versions... there should be a Version.isrelease() method, which is used in concretization. (Currently, the method is called isdevelop(); the name and boolean sense should be reversed). This allows the concretizer to choose only release versions, in the absense of a specific request from the user to install a non-release version.

@citibeth citibeth removed the ready label Oct 28, 2016
@davydden
Copy link
Copy Markdown
Member Author

This PR will break the current behavior of Spack; as far as I can tell, isdevelop('1.develop') will be False, it should be True.

there is a unit test which pass 😄 so no, it does not break anything at all!

there should be a Version.isrelease() method

probably, but it's a different story.

@davydden davydden added the ready label Oct 28, 2016
@citibeth
Copy link
Copy Markdown
Member

OK, now I see that. But it will also give isdevelop('1.mydevelopmentnightmare.3')==True as well, which is inconsistent with the rest of the PR.

Please... factor out version schemes, and then submit this as a new scheme. There have been too many unintended consequences due to "easy" changes in Version.

@citibeth citibeth removed the ready label Oct 28, 2016
@davydden
Copy link
Copy Markdown
Member Author

davydden commented Oct 28, 2016

But it will also give isdevelop('1.mydevelopmentnightmare.3')==True as well,

good point, fixed.

Please... factor out version schemes, and then submit this as a new scheme

I am sorry, but i do not have time or will to do so. Here I suggest modifications which I find useful, maintainers can always reject the PR if they don't see it fitting the goals of Spack.

@s-sajid-ali s-sajid-ali mentioned this pull request Aug 1, 2019
@adamjstewart adamjstewart mentioned this pull request Sep 1, 2019
@adamjstewart adamjstewart mentioned this pull request Sep 9, 2019
SteVwonder added a commit to SteVwonder/spack that referenced this pull request Nov 2, 2019
SteVwonder added a commit to SteVwonder/spack that referenced this pull request Nov 2, 2019
SteVwonder added a commit to SteVwonder/spack that referenced this pull request Nov 4, 2019
adamjstewart pushed a commit that referenced this pull request Nov 4, 2019
* flux: add `url_for_version` to support their C4 repo model

Flux uses a fork of ZeroMQ's Collective Code Construction Contract
(https://github.com/flux-framework/rfc/blob/master/spec_1.adoc).
This model requires a repository fork for every stable release that has
patch releases.  For example, 0.8.0 and 0.9.0 are both tags within the
main repository, but 0.8.1 and 0.9.5 would be releases on the v0.8 and
v0.9 forks, respectively.

* flux: add latest versions

* flux: remove master from `[email protected]:,master` statements

Now that #1983 has been merged, master > 0.X.0.

* flux-core: remove extraneous `99` patch version in `when` range

Replace `when=@:0.11.99` with `when=@:0.11` since the intention is to
include all patch versions of `0.11`.

* flux-core: fix `setup_build_environment` after changes in #13411

In #13411, `setup_environment` was split into `setup_build_environment`
and `setup_run_environment`, with the `spack_env` and `run_env`
arguments being changed to `env`.  Somehow the flux package was the only
one to not have its `spack_env` references in the function changed to
`env`.

* flux: add runtime environment variables that Flux checks

with older versions of Flux (i.e, 0.0:0.13), FLUX_CONNECTOR_PATH must be
set by spack to prevent failures in certain
scenarios (flux-framework/flux-core#2456).

the flux binary also sets some other environment variables, which can be
listed by running `flux -v start`.  I added a few of those just to be
sure that the Spack-installed paths are used, rather than
system-installed ones.

* flux: add optional testing dependencies to maximize test coverage

Install optional dependencies to ensure that only spack-installed
software is detected and that all tests are run when `spack install
--test` is used.

Flux's test suite will test for the existance of valgrind, jq, and any
MPI installation.  If it detects them (even if they are system-installed
and outside the spack environment), it will run optional tests against
them.  I noticed on my machine that the valgrind tests were running
against the system-install valgrind.

* flux-sched: switch to new `setup_run_environment` API
@adamjstewart adamjstewart mentioned this pull request Mar 13, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Sort non-numeric versions higher than numeric versions How to handle versions like develop and master

6 participants