update CRAN R packages by glennpj · Pull Request #28786 · spack/spack

glennpj · 2022-02-04T19:03:46Z

This PR does a sweep of CRAN R packages in Spack, and does the following:

add new versions of packages if available
make sure the cran attribute is set
add new packages as required

Due to adding the cran attribute, almost all R CRAN packages had
changes. Since almost all packages were being touched, I also tried to
standardize the formatting of the packages.

make sure there is a short and long description
have dependencies specified in the same order as in the DESCRIPTION
file
place when directives at the end

Having consistency makes it easier to process these in bulk. The process
is script driven but requires interaction. I am hopeful that the
consistent formatting will allow for further automation.

All of the packages in this PR were built with gcc-10.3.0 prior to
submitting the PR.

This PR does a sweep of CRAN R packages in Spack, and does the following: - add new versions of packages if available - make sure the `cran` attribute is set - add new packages as required Due to adding the `cran` attribute, almost all R CRAN packages had changes. Since almost all packages were being touched, I also tried to standardize the formatting of the packages. - make sure there is a short and long description - have dependencies specified in the same order as in the DESCRIPTION file - place `when` directives at the end Having consistency makes it easier to process these in bulk. The process is script driven but requires interaction. I am hopeful that the consistent formatting will allow for further automation. All of the packages in this PR were built with gcc-10.3.0 prior to submitting the PR.

glennpj · 2022-02-04T19:48:10Z

Audit checks will fail if the r dependency is a version less than the lowest version in Spack's r package. Since the spec is for such an ancient version, it is probably best to just remove those dependencies, as any r in spack would work.

glennpj · 2022-02-04T19:53:05Z

Looks like I need to find UTF-8 characters to pass the python-2.7 test.

The oldest R version in Spack is 3.1.2. Any dependency spec for a version older than that will fail the github audit test.

These formed after the previous edit to remove old r dependencies.

glennpj · 2022-02-04T23:05:39Z

I am not sure how to interpret this audit test failure:

PKG-DIRECTIVES: 1 issue found
1. Generic error in conflict for package "r-rgdal": 
    proj does not satisfy gdal
Error: Process completed with exit code 1.

The following is in the package:

    version('1.5-28', sha256='7f54432cfa8c0db463f68e8856c3ca0a90671dc841ac5203af049eb318e261a2')                                            
    version('1.5-19', sha256='6fbfd71b19e7b8d9dbddc91cb1eef8890c74a1e2bf8f619da165ff51bf1231b2')                                            
    version('1.5-18', sha256='53467c19bc93d8ea311458eaa281c8c456168ab75e84d76ef5cc6c00f53122df')                                            
    version('1.4-4',  sha256='2532e76e0af27d145f799d70006a5dbecb2d3be698e3d0bbf580f4c41a34c5d7')                                            
    version('1.3-9',  sha256='3e44f88d09894be4c0abd8874d00b40a4a5f4542b75250d098ffbb3ba41e2654')                                            
    version('1.2-16', sha256='017fefea4f9a6d4540d128c707197b7025b55e4aff98fc763065366b025b03c9')                                            
                                                                                                                                            
    depends_on('[email protected]:', type=('build', 'run'))                                                                                           
    depends_on('[email protected]:', type=('build', 'run'), when='@1.5:')                                                                             
    depends_on('[email protected]:', type=('build', 'run'))                                                                                        
    depends_on('[email protected]:')                                                                                                              
    depends_on('[email protected]:5', when='@:1.3-9')                                                                                              
    depends_on('[email protected]:', when='@1.4-2:')                                                                                               
                                                                                                                                            
    conflicts('proj@:5', when='gdal@3:')                                                                                                    
    conflicts('proj@6:', when='gdal@:2')

This concretizes and builds but the audit test does not like those conflicts.

Update: I am missing the ^ for the specs, but should the spack parser catch that at concretization time?

glennpj · 2022-02-17T18:27:41Z

@spackbot run pipeline

spackbot-app · 2022-02-17T18:27:44Z

I've started that pipeline for you!

adamjstewart · 2022-02-20T20:08:50Z

Audit checks will fail if the r dependency is a version less than the lowest version in Spack's r package. Since the spec is for such an ancient version, it is probably best to just remove those dependencies, as any r in spack would work.

This should only be true if the package requires a specific version like depends_on('[email protected]') and that version doesn't exist in the package. It shouldn't affect version ranges like depends_on('[email protected]:').

This reverts commit 74e7839.

glennpj · 2022-02-21T01:12:17Z

This should only be true if the package requires a specific version like depends_on('[email protected]') and that version doesn't exist in the package. It shouldn't affect version ranges like depends_on('[email protected]:').

Yep, I misread the test results. I reverted that change and fixed the package that triggered it.

adamjstewart

Scrolled through about half of them, LGTM. Some of the new docstrings look a bit long, but I'll let you decide what to use. I like the consistency. Also, not sure if it's better to use the CRAN page as the homepage or the URLs listed on that page that the developers chose. For PyPI, I usually prefer the latter.

Someday I would love to add the update script you use directly to Spack. People have asked for similar things to autogenerate packages for CMake and PyPI but those build systems are much less uniform than R.

glennpj · 2022-02-21T17:02:07Z

Also, not sure if it's better to use the CRAN page as the homepage or the URLs listed on that page that the developers chose.

I decided to use the canonical CRAN "homepage". This makes things more internally consistent with CRAN packages as well as with BioConductor packages, which have a similar scheme, although a different portal.

kinh-nguyen · 2022-02-24T13:44:42Z

Hi, it worked before but today creating a new environment and reinstall r-sf the r-units causes the error. Would it be related to this update?

==> Installing r-units-0.7-2-gypq7s4gl53lmoz6dlykn5y63c7dcpg5
==> No binary for r-units-0.7-2-gypq7s4gl53lmoz6dlykn5y63c7dcpg5 found: installing from source
==> Error: FetchError: All fetchers failed for spack-stage-r-units-0.7-2-gypq7s4gl53lmoz6dlykn5y63c7dcpg5

/home/user/spack/lib/spack/spack/package.py:1400, in do_fetch:
       1397
       1398        self.stage.create()
       1399        err_msg = None if not self.manual_download else self.download_instr
  >>   1400        start_time = time.time()
       1401        self.stage.fetch(mirror_only, err_msg=err_msg)
       1402        self._fetch_time = time.time() - start_time
       1403

adamjstewart · 2022-02-24T15:54:27Z

I can reproduce this too, no idea what could be causing this to fail.

kinh-nguyen · 2022-02-24T15:57:20Z

I think because units has a new version? I can install with my repo after adding the new one.

adamjstewart · 2022-02-24T17:39:47Z

But we should be using list_url as a backup for all packages

glennpj · 2022-02-26T00:31:23Z

It seems like issues with list_url have gotten more frequent lately. I typically work with a local mirror so probably not seeing it. Unfortunately, R relies on list_url. I will poke at this over the next few days.

JSquar · 2022-03-18T10:26:11Z

I think because units has a new version? I can install with my repo after adding the new one.

I tried to install r-terra , which also has a new version available (1.5.21), which is not integrated in the package (1.5.17), yet. But the fetch step does not consider this and tries to get sources from https://cloud.r-project.org/src/contrib/terra_1.5-17.tar.gz, which does not exist anymore, since old versions are archived in https://cran.r-project.org/src/contrib/Archive/terra/.

glennpj · 2022-03-19T04:07:49Z

@JSquar This should be fixed now that #29258 has been merged.

JSquar · 2022-03-19T20:23:51Z

@glennpj Tried it, works fine now

glennpj added new-package new-version R labels Feb 4, 2022

glennpj added 3 commits February 4, 2022 14:11

Transliterate Unicode characters to ASCII

5d5ebfa

Remove dependencies on ancient R versions

74e7839

The oldest R version in Spack is 3.1.2. Any dependency spec for a version older than that will fail the github audit test.

Remove blank lines at end of file

bc47226

These formed after the previous edit to remove old r dependencies.

glennpj force-pushed the r_updates/CRAN branch from e368c36 to bc47226 Compare February 4, 2022 21:19

Need ^ for conflicts specs

f33c469

glennpj mentioned this pull request Feb 12, 2022

update bioconductor packages to bioconductor 3.14 #28900

Merged

glennpj mentioned this pull request Feb 18, 2022

r-backports: add 1.4.1 #28995

Merged

glennpj requested a review from adamjstewart February 20, 2022 19:53

Revert "Remove dependencies on ancient R versions"

01f5c6d

This reverts commit 74e7839.

adamjstewart approved these changes Feb 21, 2022

View reviewed changes

adamjstewart merged commit 275608e into spack:develop Feb 21, 2022

glennpj deleted the r_updates/CRAN branch February 21, 2022 18:09

This was referenced Feb 21, 2022

r-broom: add 0.7.12 #28999

Closed

r-caret: add 6.0-90 #29001

Closed

r-e1071: add 1.7-9 #29002

Closed

r-proxy: add 0.4-26 #29003

Closed

r-class: add 7.3-20 #29004

Closed

r-dbi: add 1.1.2 #29035

Closed

This was referenced Feb 21, 2022

r-digest: add 0.6.29 #29036

Closed

r-dtplyr: add 1.2.1 #29059

Closed

r-emmeans: add 1.7.2 #29060

Closed

r-fansi: add 1.0.2 #29061

Closed

r-foreign: add 0.8-82 #29063

Closed

r-fs: add 1.5.2 #29064

Closed

r-future: add 1.23.0 #29065

Closed

manuelakuhn mentioned this pull request Feb 22, 2022

r-cardata: add 3.0-5 #29000

Closed

bvanessen pushed a commit to bvanessen/spack that referenced this pull request Mar 2, 2022

update CRAN R packages (spack#28786)

16cbf0b

Conversation

glennpj commented Feb 4, 2022

Uh oh!

glennpj commented Feb 4, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

glennpj commented Feb 4, 2022

Uh oh!

glennpj commented Feb 4, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

glennpj commented Feb 17, 2022

Uh oh!

spackbot-app bot commented Feb 17, 2022

Uh oh!

adamjstewart commented Feb 20, 2022

Uh oh!

glennpj commented Feb 21, 2022

Uh oh!

adamjstewart left a comment

Choose a reason for hiding this comment

Uh oh!

glennpj commented Feb 21, 2022

Uh oh!

kinh-nguyen commented Feb 24, 2022

Uh oh!

adamjstewart commented Feb 24, 2022

Uh oh!

kinh-nguyen commented Feb 24, 2022

Uh oh!

adamjstewart commented Feb 24, 2022

Uh oh!

glennpj commented Feb 26, 2022

Uh oh!

JSquar commented Mar 18, 2022

Uh oh!

glennpj commented Mar 19, 2022

Uh oh!

JSquar commented Mar 19, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

glennpj commented Feb 4, 2022 •

edited

Loading

glennpj commented Feb 4, 2022 •

edited

Loading