Skip to content

Merged 'purge' command with 'clean' and deleted 'purge' #4970

Merged
becker33 merged 3 commits intospack:developfrom
epfl-scitas:refactoring/merge_purge_and_clean
Aug 9, 2017
Merged

Merged 'purge' command with 'clean' and deleted 'purge' #4970
becker33 merged 3 commits intospack:developfrom
epfl-scitas:refactoring/merge_purge_and_clean

Conversation

@alalazo
Copy link
Copy Markdown
Member

@alalazo alalazo commented Aug 4, 2017

fixes #2942

spack purge has been merged with spack clean. Documentation has been updated accordingly. The clean and purge behavior are not mutually exclusive, and they log brief information to tty while they go.

Also, cleans up temporary files for a particular package, by deleting the
expanded/checked out source code *and* any downloaded archive. If
``fetch``, ``stage``, or ``install`` are run again after this, Spack's
build process will start from scratch.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if I would expect spack clean foo to delete the downloaded archive. Does spack clean delete the downloaded archive for everything? If not, we should keep them in line. I think this would be a good setup:

$ spack clean --stage foo  # delete stage for foo
$ spack clean foo  # same as above
$ spack clean --downloads foo  # delete downloaded archive for foo
$ spack clean --stage --downloads foo  # both of the above

It would actually be really nice if spack clean --downloads foo worked. It makes it a lot easier to test fetching for packages I've already downloaded. I've been manually deleting archives for so long.

There's also a reference above that should be updated.

When called with --stage or without arguments this removes all staged files and will be equivalent to running spack clean for every package you have fetched or staged.

This is no longer equivalent to spack clean, it is spack clean.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The functionality didn't change at all. The command:

$ spack clean --stage foo

in this branch is equivalent to:

$ spack clean foo
$ spack purge --stage

in develop (so it deletes the stage for foo, and then it deletes all stages.). Same for other commands. This is reflected in the command help (hopefully):

$ spack help clean
usage: spack clean [-h] [-s] [-d] [-m] [-a] ...

remove temporary build files and/or downloaded archives

positional arguments:
  specs             removes the build stages and tarballs for specs

optional arguments:
  -h, --help        show this help message and exit
  -s, --stage       remove all temporary build stages (default)
  -d, --downloads   remove cached downloads
  -m, --misc-cache  remove long-lived caches, like the virtual package index
  -a, --all         equivalent to -sdm

Good catch for the docs. I'll update the reference.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if that's true:

$ spack clean zlib
$ spack fetch zlib
==> Using cached archive: /Users/Adam/spack/var/spack/cache/zlib/zlib-1.2.11.tar.gz

spack clean foo doesn't delete the archived tarball. Or are we talking about different tarballs? Maybe you're talking about the one copied to the stage? I was talking about the one in the downloads mirror.

Anyway, I would like to see this behavior:

$ spack clean --stage foo  # delete stage for foo
$ spack clean foo  # same as above
$ spack clean --downloads foo  # delete downloaded archive for foo
$ spack clean --stage --downloads foo  # both of the above

As long as I can do this, I'm happy. Not sure how much work would be involved in spack clean --downloads foo though.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we keep this extension for another PR? The code here behaves exactly like the code in develop, it just merges the two commands. Example:

$ spack clean --stage zlib
==> Cleaning build stage [[email protected]%[email protected]+pic+shared arch=linux-ubuntu14-x86_64 /pgxsxv7]
==> Removing all temporary build stages

$ spack clean --all
==> Removing all temporary build stages
==> Removing cached downloads
==> Removing cached information on repositories

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll open an issue after this is merged.

@alalazo alalazo force-pushed the refactoring/merge_purge_and_clean branch from b94cefa to b1c168a Compare August 4, 2017 14:29

When called with positional arguments, cleans up temporary files only
for a particular package, by deleting the
expanded/checked out source code *and* any downloaded archive. If
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just to clarify, does spack clean foo delete the stage and the cached download archive? Or just the former?

Copy link
Copy Markdown
Member Author

@alalazo alalazo Aug 4, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If think what was meant is any downloaded archive in the stage. I didn't check, but maybe this documentation precedes the introduction of the cached download archive. Any suggestion for writing this part better is welcome.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would just remove the second half of the sentence. We've already explained what spack clean/spack clean --staged does. Here we are clarifying that spack clean foo only applies to foo, not to every package.

@alalazo alalazo force-pushed the refactoring/merge_purge_and_clean branch from b1c168a to 66bc68d Compare August 4, 2017 14:56
@adamjstewart
Copy link
Copy Markdown
Member

Oh, don't forget to update the Bash completion script!

alalazo added 3 commits August 5, 2017 15:48
'spack purge' has been merged with 'spack clean'. Documentation has been
updated accordingly. The 'clean' and 'purge' behavior are not mutually
exclusive, and they log brief information to tty while they go.
@alalazo alalazo force-pushed the refactoring/merge_purge_and_clean branch from 66bc68d to 39fc37d Compare August 5, 2017 14:54
@alalazo
Copy link
Copy Markdown
Member Author

alalazo commented Aug 5, 2017

@tgamblin If you agree with the removal of spack purge, I think this is ready to be merged.

@alalazo alalazo requested a review from davydden August 8, 2017 06:48
@alalazo
Copy link
Copy Markdown
Member Author

alalazo commented Aug 8, 2017

@davydden I was wondering if you could do a test drive of this and provide some feedback?

Copy link
Copy Markdown
Member

@davydden davydden left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like spack -h does not show clean as an option. Otherwise looks good:

$ spack clean
==> Removing all temporary build stages
$ spack clean -a
==> Removing all temporary build stages
==> Removing cached downloads
==> Removing cached information on repositories

@alalazo
Copy link
Copy Markdown
Member Author

alalazo commented Aug 8, 2017

@davydden That's because neither spack clean nor spack purge were considered basic commands. You should see it doing:

$ spack help -a

@davydden
Copy link
Copy Markdown
Member

davydden commented Aug 8, 2017

That's because neither spack clean nor spack purge were considered basic commands

did not know that! Thanks.

@tgamblin
Copy link
Copy Markdown
Member

tgamblin commented Aug 8, 2017

Should we put spack clean in the basic help?

@becker33
Copy link
Copy Markdown
Member

becker33 commented Aug 9, 2017

Not sure we should add it, but I think we could make clearer in the basic help that there are additional commands not listed.

@becker33
Copy link
Copy Markdown
Member

becker33 commented Aug 9, 2017

@alalazo in #2942 you mentioned wanting to use anonymous specs in spack clean. Did you decide these belong in separate PRs?

@alalazo
Copy link
Copy Markdown
Member Author

alalazo commented Aug 9, 2017

@becker33 Kind of. I realized that I missed an important detail back then: we don't maintain a db for the build stages as we do for installed packages.

Without having a db it's difficult to know what is in the stage and select directories based on the constraints given by the anonymous specs. We can of course maintain a simplified db in the stage, but I am not sure it is worth the effort. Thoughts?

@tgamblin

Should we put spack clean in the basic help?

Fine with me. If everybody agrees I can change the group here.

@becker33
Copy link
Copy Markdown
Member

becker33 commented Aug 9, 2017

I'd agree that it's not worth it, or at least can be a separate issue.

If we wanted to do it we could add things to the database when staged and then mark them installed when installed, but I wouldn't necessarily go for it right now.

@becker33 becker33 merged commit faeb1b7 into spack:develop Aug 9, 2017
@alalazo alalazo deleted the refactoring/merge_purge_and_clean branch August 9, 2017 17:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Should we merge spack clean and spack purge?

5 participants