I think there's one other modification we've made to argparse: dfcef57

Is something similar present in Python 3 or does this need to be hacked somehow?

Is something similar present in Python 3 or does this need to be hacked somehow?

Can you point me to some place where this modification will be evident? I don't think Python 3 adds something specific to this, but I checked a few commands with options having multiple choices and didn't notice a difference in formatting on my platform.

Not sure, @tgamblin added this patch during the big help-message refactor several years ago.

I think this is ready to be reviewed and eventually merged then. I don't think we should substitute a standard library to have a different layout of some multi-valued option. We might extend some of the argparse classes later, if we spot what the issue is here.

@adamjstewart did you try using this branch? The minute I check it out I get:

(spackle):spack> spack help
Traceback (most recent call last):
  File "/Users/gamblin2/src/spack/bin/spack", line 73, in <module>
    import spack.main  # noqa
  File "/Users/gamblin2/Workspace/spack/lib/spack/spack/main.py", line 13, in <module>
    import argparse
ImportError: bad magic number in 'argparse': b'\x03\xf3\r\n'

Ok, this is b/c there is a stale argparse.pyc in the external directory. In the past when we have removed things from there, we've done something about the stale .pyc files. If they're not removed, import argparse will pick up the pyc *even if there is no argparse.py.

The commit @adamjstewart is there to make this look nice:

develop:

(spackle):spack> spack config --scope foobar
usage: spack config [-h] [--scope {defaults,system,site,user}[/PLATFORM]]
                    SUBCOMMAND ...
spack config: error: argument --scope: invalid choice: 'foobar' choose from:
    _builtin      defaults         site         system         user
    command_line  defaults/darwin  site/darwin  system/darwin  user/darwin

This PR:

(spackle):spack> spack config --scope foobar
usage: spack config [-h] [--scope {defaults,system,site,user}[/PLATFORM]]
                    SUBCOMMAND ...
spack config: error: argument --scope: invalid choice: 'foobar' (choose from '_builtin', 'defaults', 'defaults/darwin', 'system', 'system/darwin', 'site', 'site/darwin', 'user', 'user/darwin', 'command_line')

Any thoughts on how to fix that?

Any thoughts on how to fix that?

I moved that function to our SpackArgumentParser class. It's not ideal since it's an underscore method, but the API seems stable across the Python versions we support.

For stale *.pyc files, can we rely on spack clean -p?

For stale *.pyc files, can we rely on spack clean -p?

This is probably the right way to do it but it needs to be a clear recommendation. Can we catch ImportErrors in main and notice when they refer to import argparase? I think then we can check for the .pyc file and recommend running spack clean -p if it is there (otherwise raise b/c we don't know what happened).

This is one of those checks that we can likely eventually remove, but that need to go in when the change is made.

I could reproduce the error by copying a Python2.7 argparse.pyc file in the lib/spack/external directory. Unfortunately my suggestion to run spack clean -p wouldn't work in this particular case since... every command needs argparse. I just added code in main.py that should take care of deleting the pyc file if present and a TODO to remember removing it at some point in the future.

@spackbot run pipeline

I've started that pipeline for you!

Done. The low patch coverage is due to 5c607ca but I think a test for that might be an overkill.