Skip to content

Allow version to accept the '=' token without activating lexer switch#32257

Merged
becker33 merged 50 commits intospack:developfrom
psakievich:parser-work
Sep 14, 2022
Merged

Allow version to accept the '=' token without activating lexer switch#32257
becker33 merged 50 commits intospack:developfrom
psakievich:parser-work

Conversation

@psakievich
Copy link
Copy Markdown
Contributor

@psakievich psakievich commented Aug 19, 2022

Closes #32184. That PR seems to have been corrupted due to an unfortunate rebase.

This PR will address the issue of GitVerions with assigned refs that inspired the creation of #32184 but it will also make the parser behave in a more targeted manner when dealing with the = token. This is done by giving higher precedent, targeted tokens for the cases where we want to glob several commands together (ldlibs, compiler flags and targets).

This should make the parser behave more cleanly when the = token is encountered since it is only the targeted tokens that will trigger the switch in behavior.

After back and forth it was decided that we should change the lexing specifically for versions to allow a version token to accept the entire string. This adds additional complexity because there are two layers of lexing:

  1. Shell lexing
  2. Spack's SpecLexer

These two lexers treat spaces differently and so additional logic is required when there are spaces around the @. Another side effect is that different combinations of tokens can ultimately parse the same spec. A fun exercise is to go through the git history of this PR and see how I discover this over time.

The net result should be the same as #32437 minus the extra @ in the token. If this is acceptable then I will change #32437 to just be the fixes for submodules with GitRef versions and the other small bug fixes.

@becker33 and @tgamblin

@spackbot-app spackbot-app bot added core PR affects Spack core functionality tests General test capability(ies) labels Aug 19, 2022
@psakievich
Copy link
Copy Markdown
Contributor Author

@spackbot fix style

@spackbot-app
Copy link
Copy Markdown

spackbot-app bot commented Aug 22, 2022

Let me see if I can fix that for you!

@spackbot-app
Copy link
Copy Markdown

spackbot-app bot commented Aug 22, 2022

I was able to run spack style --fix for you!

spack style --fix
==> Running style checks on spack
  selected: isort, mypy, black, flake8
==> Modified files
  lib/spack/spack/spec.py
  lib/spack/spack/test/spec_syntax.py
==> Running isort checks
  isort checks were clean
==> Running mypy checks
Success: no issues found in 556 source files
  mypy checks were clean
==> Running black checks
All done! ✨ 🍰 ✨
2 files left unchanged.
  black checks were clean
==> Running flake8 checks
  flake8 checks were clean
==> spack style checks were clean
Keep in mind that I cannot fix your flake8 or mypy errors, so if you have any you'll need to fix them and update the pull request. If I was able to push to your branch, if you make further changes you will need to pull from your updated branch before pushing again.

I wasn't able to make any further changes, but please see the message above for remaining issues you can fix locally!

@psakievich psakievich changed the title Improve precision of the parser when dealing with '=' tokens Allow version to accept the '=' token without activating lexer switch Sep 2, 2022
@psakievich psakievich marked this pull request as ready for review September 2, 2022 04:52
@psakievich psakievich closed this Sep 4, 2022
@psakievich psakievich reopened this Sep 4, 2022
@psakievich
Copy link
Copy Markdown
Contributor Author

Pinging @becker33 and @tgamblin

@psakievich
Copy link
Copy Markdown
Contributor Author

@spackbot fix style

@spackbot-app
Copy link
Copy Markdown

spackbot-app bot commented Sep 9, 2022

Let me see if I can fix that for you!

@spackbot-app
Copy link
Copy Markdown

spackbot-app bot commented Sep 9, 2022

I was able to run spack style --fix for you!

spack style --fix
==> Running style checks on spack
  selected: isort, mypy, black, flake8
==> Modified files
  lib/spack/spack/spec.py
  lib/spack/spack/test/spec_syntax.py
==> Running isort checks
  isort checks were clean
==> Running mypy checks
Success: no issues found in 558 source files
  mypy checks were clean
==> Running black checks
reformatted lib/spack/spack/test/spec_syntax.py
All done! ✨ 🍰 ✨
1 file reformatted, 1 file left unchanged.
  black checks were clean
==> Running flake8 checks
  flake8 checks were clean
==> spack style checks were clean
Keep in mind that I cannot fix your flake8 or mypy errors, so if you have any you'll need to fix them and update the pull request. If I was able to push to your branch, if you make further changes you will need to pull from your updated branch before pushing again.

I've updated the branch with isort fixes.

@psakievich
Copy link
Copy Markdown
Contributor Author

@spackbot re-run pipeline

@spackbot-app
Copy link
Copy Markdown

spackbot-app bot commented Sep 12, 2022

I've started that pipeline for you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

core PR affects Spack core functionality tests General test capability(ies)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants