Skip to content

Features/deprecate#12933

Merged
tgamblin merged 32 commits intodevelopfrom
features/deprecate
Oct 23, 2019
Merged

Features/deprecate#12933
tgamblin merged 32 commits intodevelopfrom
features/deprecate

Conversation

@becker33
Copy link
Copy Markdown
Member

@becker33 becker33 commented Sep 24, 2019

Closes #172.
Closes #7409.

spack deprecate allows for the removal of insecure packages with minimal impact to their dependents. It is intended as a very big hammer for people who really know what they're doing, that will allow one package to be symlinked into the prefix of another to provide seamless transition for rpath'd and hard-coded applications using the old version.

Take openssl as an example. Old versions are often found to have security flaws, and diligent sysadmins want to purge them from the system. Spack currently provides no way to do this without breaking every package that depends on openssl. With spack deprecate, the sysadmin will run

spack deprecate /hash-of-old-openssl /hash-of-new-openssl

and all hardcoded codes can use the new one transparently.

From the documentation in this PR:

spack deprecate allows for the removal of insecure packages with minimal impact to their dependents.

.. warning::

The spack deprecate command is designed for use only in extroardinary circumstances. This is a VERY big hammer to be used with care.

The spack deprecate command will remove one package and replace it with another by replacing the deprecated package's prefix with a link to the replacement package's prefix.

.. warning::

The spack deprecate command makes no promises about binary compatibility. It is up to the user to ensure the replacement is suitable for the deprecated package.

Spack tracks concrete deprecated specs and ensures that no future packages concretize to a deprecated spec.

The first spec given to the spack deprecate command is the package to deprecate. It is an abstract spec that must describe a single installed package. The second spec argument is the replacement spec. By default it must be an abstract spec that describes a single installed package, but with the -i/--install-replacement it can be any abstract spec that Spack will install and then use as the replacement. The -I/--no-install-replacement option will ensure the default behavior.

By default, spack deprecate will deprecate all dependencies of the deprecated spec, replacing each by the dependency of the same name in the replacement spec. The -d/--dependencies option will ensure the default, while the -D/--no-dependencies option will deprecate only the root of the deprecate spec in favor of the root of the replacement spec.

spack deprecate can use symbolic links or hard links. The default behavior is symbolic links, but the -l/--link-type flag can take options hard or soft.

@tgamblin
Copy link
Copy Markdown
Member

@becker33: is there a good reason for coverage on this not to be higher? I think we should shoot for something in the 90's.

@scheibelp scheibelp self-assigned this Oct 17, 2019
Copy link
Copy Markdown
Member

@scheibelp scheibelp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Haven't fully covered this but I have some initial requests/questions

Copy link
Copy Markdown
Member

@scheibelp scheibelp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few more requests

Copy link
Copy Markdown
Member

@scheibelp scheibelp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have a request related to parameter naming and some questions about how to support future use cases

  • replacing local packages with upstream packages (less important)
  • re-deprecating a package in order to repeatedly point it at newer versions of a package as it is released

I think for the use case mentioned in the PR description (openssl) the latter will quickly become important and so I want to make sure this lays some groundwork for that (or at least that it wouldn't make it harder to do that).

Copy link
Copy Markdown
Member

@scheibelp scheibelp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few more requests:

  • there's some remaining debug code (tty.msg(AAAAAA))
  • a few requests for changing names
  • I think deprecated/deprecatee/replacement can be consolidated into two terms for consistency

Copy link
Copy Markdown
Member

@scheibelp scheibelp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a message string missing a format option and I have a couple questions, one of them here:

Should there be any database tests for deprecation?

  • reindexing with deprecated packages
  • ensuring ref counts or whatever other behavior is expected when it comes to deprecating specs in the DB

deprecated.package.do_deprecate(deprecator, link_fn)

# Now that we've handled metadata, uninstall and replace with link
Package.uninstall_by_spec(spec, force=True, deprecator=deprecator)
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(question) One of the post-uninstall hooks is to remove associated module files. Does it make sense to uninstall the spec if a dependent still wants to do a recursive module load?

@tgamblin tgamblin merged commit cd185c3 into develop Oct 23, 2019
@tgamblin tgamblin deleted the features/deprecate branch October 25, 2019 06:48
jrmadsen pushed a commit to jrmadsen/spack that referenced this pull request Oct 30, 2019
`spack deprecate` allows for the removal of insecure packages with minimal impact to their dependents. It allows one package to be symlinked into the prefix of another to provide seamless transition for rpath'd and hard-coded applications using the old version.

Example usage:

    spack deprecate /hash-of-old-openssl /hash-of-new-openssl

The spack deprecate command is designed for use only in extroardinary circumstances.  The spack deprecate command makes no promises about binary compatibility. It is up to the user to ensure the replacement is suitable for the deprecated package.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Proposal: Add upgrade-in-place functionality How to handle security updates?

4 participants