Skip to content

Parser fix#2769

Merged
tgamblin merged 16 commits intodevelopfrom
parser-fix
Jan 16, 2017
Merged

Parser fix#2769
tgamblin merged 16 commits intodevelopfrom
parser-fix

Conversation

@becker33
Copy link
Copy Markdown
Member

@becker33 becker33 commented Jan 6, 2017

Eliminates the need for escape quotes.

TODO: Fix multiple specs. Ideally add (more) unit tests.

@becker33 becker33 added the WIP label Jan 6, 2017
@becker33 becker33 requested a review from tgamblin January 6, 2017 23:47
@adamjstewart
Copy link
Copy Markdown
Member

This won't change the hashes will it?

@tgamblin
Copy link
Copy Markdown
Member

tgamblin commented Jan 7, 2017

@becker33: You forgot to mention that this disallows the use of - for variants. I am still mulling that one over -- not convinced it's necessary to do what is in the title. I'm tempted not to remove it as I don't think it actually hurts anything. You can actually parse all of these unambiguously even if dash is allowed (suppose the packages have a variant called g):

pkg cflags=-O3
pkg +g
pkg -g
pkg cflags="-O3 -g" +g
pkg cflags="-O3 -g" -g
pkg cflags=-O3 +g
pkg cflags=-O3 -g

pkg-with-dashes cflags=-O3
pkg-with-dashes +g
pkg-with-dashes -g
pkg-with-dashes cflags="-O3 -g" +g
pkg-with-dashes cflags="-O3 -g" -g
pkg-with-dashes cflags=-O3 +g
pkg-with-dashes cflags=-O3 -g

@tgamblin
Copy link
Copy Markdown
Member

tgamblin commented Jan 7, 2017

@becker33: can we remove the dash part and just do the escape quotes?

@becker33
Copy link
Copy Markdown
Member Author

becker33 commented Jan 7, 2017

@tgamblin I've added the - back into the syntax.

@adamjstewart This should not change any hashes, it only affects the parser.

This should be good to go now code-wise, but I'm leaving the WIP tag on until I can write some unit tests for it. I want to test multiple spec parsing more fully and possibly add some more tests for odd spacing choices.

@becker33 becker33 added ready and removed WIP labels Jan 9, 2017
@becker33
Copy link
Copy Markdown
Member Author

becker33 commented Jan 9, 2017

@tgamblin Should I change install so that it can handle more than one package at once?

@tgamblin
Copy link
Copy Markdown
Member

tgamblin commented Jan 9, 2017

@becker33: yes please.

alalazo referenced this pull request Jan 9, 2017
* Removes the extra argument from Package.do_install while maintaining the changes in behavior pulled in #1603

* install : removed -i and -d shorthands (breaks backward compatibility)

* Change ':' to ','
@becker33 becker33 added WIP and removed ready labels Jan 9, 2017
@becker33 becker33 added ready and removed WIP labels Jan 9, 2017
@becker33
Copy link
Copy Markdown
Member Author

I'll create a separate PR to update documentation to match the new parser.

@adamjstewart
Copy link
Copy Markdown
Member

Might as well put documentation fixes in the same PR.

Copy link
Copy Markdown
Member

@tgamblin tgamblin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Nice work @becker33!

@tgamblin tgamblin merged commit a091eec into develop Jan 16, 2017
@tgamblin
Copy link
Copy Markdown
Member

@becker33: still doesn't handle this (these should be the same, assuming /abc123 is a hash for libelf):

libelf /abc123
libelf/abc123
/abc123

This parser thinks the first two are two separate specs. But we can handle that in another PR.

@becker33
Copy link
Copy Markdown
Member Author

We currently accept the third. If we accept the first two, we cannot accept the third and maintain our support for multiple specs. (Or at least we cannot allow hashes in any but the first spec).

@tgamblin
Copy link
Copy Markdown
Member

@becker33: how so?

@adamjstewart
Copy link
Copy Markdown
Member

Because the hash actually contains the name of the package, I'm assuming. saying libelf /abc123 is essentially saying libelf [email protected]%[email protected] arch=linux-fedora25-x86_64. If we allow multiple specs, we would have to disallow specifying the same package twice. Otherwise, what if someone says netcdf /fdsafa where netcdf defaults to +mpi and /fdsafa is netcdf~mpi? Should we build netcdf+mpi and netcdf~mpi? It introduces ambiguity.

@tgamblin
Copy link
Copy Markdown
Member

tgamblin commented Jan 17, 2017

I'm proposing that netcdf /fdsafa should be an error if fdsafa isn't a netcdf hash. The parser knows this; it has to look the hash up anyway. It's just a constraint violation, same as if you typed:

    libelf +foo ~foo

currently the / is special because it always introduces a new spec. I'm proposing (and this was the original intent of the parser) that it should constrain the any incomplete spec being parsed, like the other sigils do.

It would then be special in that it could both do that and introduce a new spec:

python                     # 1 spec
/abc123                    # 1 spec
python /abc123             # 1 spec
/456789                    # 1 spec
python /abc123 /456789     # 2 specs
python /456789 /abc123     # 2 specs
/abc123 /456789            # 2 specs
/456789 /abc123            # 2 specs
/456789 /abc123 python     # 3 specs

@tgamblin
Copy link
Copy Markdown
Member

You may ask: why do this? Because of this:

$ spack uninstall libelf
==> Error: libelf matches multiple packages:

-- darwin-elcapitan-x86_64 / [email protected] ------------------
yroox5q [email protected]%clang

5iqi6kr [email protected]%clang


==> Error: You can either:
    a) Use a more specific spec, or
    b) use spack uninstall -a to uninstall ALL matching specs.

$ spack uninstall libelf /5iqi6kr

With what I'm proposing, you can just make the previous line more specific. With the current syntax, you have to delete libelf and replace with /5iqi6kr.

@adamjstewart
Copy link
Copy Markdown
Member

$ spack uninstall libelf /5iqi6kr

A lot of users, including myself, have run into this trap and been confused why this didn't work. I agree that if possible, it would be nice to support this.

@tgamblin
Copy link
Copy Markdown
Member

@becker33: This is possible. I can help out w/the parser if you have questions.

@becker33
Copy link
Copy Markdown
Member Author

@tgamblin How is your third line of examples two specs? Are you suggesting that it be two specs if \abc123 is not a spec that satisfies python and one spec if it does satisfy python? That seems overly ambiguous to me.

All the rest of your examples seem very doable, since the hash starts a new spec if it is in the first position or if it follows another hash

@becker33
Copy link
Copy Markdown
Member Author

In general I agree that the use case of adding the hash to an existing command for clarity is compelling.

@tgamblin
Copy link
Copy Markdown
Member

@becker33:

How is your third line of examples two specs? Are you suggesting that it be two specs if \abc123 is not a spec that satisfies python and one spec if it does satisfy python? That seems overly ambiguous to me.

Ugh that's a bug. Edited. Sorry! If the hash doesn't match the preceding package name and constraints, it should just raise an exception and say the spec is a contradiction.

@becker33
Copy link
Copy Markdown
Member Author

Okay, I agree with that specification.

How would you feel about adding // to specify a hash unrelated to whatever precedes it?

@tgamblin
Copy link
Copy Markdown
Member

@becker33: I'm not sure it's worth it. How about we implement the simple thing first and see whether users run into issues where they really need to start a new spec now, without deleting or reordering... if they do we'll think about how to address it.

@becker33
Copy link
Copy Markdown
Member Author

Ok, I'll create a new PR for this.

becker33 added a commit that referenced this pull request Jan 20, 2017
@becker33 becker33 mentioned this pull request Jan 20, 2017
tgamblin pushed a commit that referenced this pull request Jan 26, 2017
- Allows hashes to be specified after other parts of the spec
- Does not allow other parts of the spec to be specified after the hash
- The hash must either end input or be followed by another separate spec
- The next spec cannot be an anonymous spec (it must start with a package name or a hash)

See #2769 (after it was merged) for further discussion of this interface addition. That discussion resulted in these requirements:

```
python                     # 1 spec
/abc123                    # 1 spec
python /abc123             # 1 spec
/456789                    # 1 spec
python /abc123 /456789     # 2 specs
python /456789 /abc123     # 2 specs
/abc123 /456789            # 2 specs
/456789 /abc123            # 2 specs
/456789 /abc123 python     # 3 specs
```

assuming `abc123` and `456789` are both hashes of different python specs.
diaena pushed a commit to diaena/spack that referenced this pull request May 26, 2017
- Allows hashes to be specified after other parts of the spec
- Does not allow other parts of the spec to be specified after the hash
- The hash must either end input or be followed by another separate spec
- The next spec cannot be an anonymous spec (it must start with a package name or a hash)

See spack#2769 (after it was merged) for further discussion of this interface addition. That discussion resulted in these requirements:

```
python                     # 1 spec
/abc123                    # 1 spec
python /abc123             # 1 spec
/456789                    # 1 spec
python /abc123 /456789     # 2 specs
python /456789 /abc123     # 2 specs
/abc123 /456789            # 2 specs
/456789 /abc123            # 2 specs
/456789 /abc123 python     # 3 specs
```

assuming `abc123` and `456789` are both hashes of different python specs.
amklinv pushed a commit that referenced this pull request Jul 17, 2017
- Allows hashes to be specified after other parts of the spec
- Does not allow other parts of the spec to be specified after the hash
- The hash must either end input or be followed by another separate spec
- The next spec cannot be an anonymous spec (it must start with a package name or a hash)

See #2769 (after it was merged) for further discussion of this interface addition. That discussion resulted in these requirements:

```
python                     # 1 spec
/abc123                    # 1 spec
python /abc123             # 1 spec
/456789                    # 1 spec
python /abc123 /456789     # 2 specs
python /456789 /abc123     # 2 specs
/abc123 /456789            # 2 specs
/456789 /abc123            # 2 specs
/456789 /abc123 python     # 3 specs
```

assuming `abc123` and `456789` are both hashes of different python specs.
healther pushed a commit to electronicvisions/spack that referenced this pull request Jul 26, 2017
- Allows hashes to be specified after other parts of the spec
- Does not allow other parts of the spec to be specified after the hash
- The hash must either end input or be followed by another separate spec
- The next spec cannot be an anonymous spec (it must start with a package name or a hash)

See spack#2769 (after it was merged) for further discussion of this interface addition. That discussion resulted in these requirements:

```
python                     # 1 spec
/abc123                    # 1 spec
python /abc123             # 1 spec
/456789                    # 1 spec
python /abc123 /456789     # 2 specs
python /456789 /abc123     # 2 specs
/abc123 /456789            # 2 specs
/456789 /abc123            # 2 specs
/456789 /abc123 python     # 3 specs
```

assuming `abc123` and `456789` are both hashes of different python specs.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants