Skip to content

spack list: format version_json#11652

Merged
tgamblin merged 1 commit intospack:developfrom
ax3l:topic-listLatestJson
Jun 7, 2019
Merged

spack list: format version_json#11652
tgamblin merged 1 commit intospack:developfrom
ax3l:topic-listLatestJson

Conversation

@ax3l
Copy link
Copy Markdown
Member

@ax3l ax3l commented Jun 6, 2019 •
edited
Loading

List the latest version of each package in JSON encoding.
Preparation for consumption for a "spack badge" service.

spack list --format version_json
[
  {"...": "..."},
  {"name": "adios2",
   "latest_version": "2.3.1",
   "versions": ["develop", "2.3.1", "2.2.0", "2.1.0", "2.0.0"],
   "homepage": "https://www.olcf.ornl.gov/center-projects/adios/",
   "file": "https://github.com/spack/spack/blob/develop/var/spack/repos/builtin/packages/adios2/package.py",
   "maintainers": ["ax3l", "chuckatkins"],
   "dependencies": {"build": ["cmake", "pkgconfig", "mpi", "zeromq", "hdf5", "adios", "bzip2", "zfp", "python", "py-numpy", "py-mpi4py"], "link": ["mpi", "zeromq", "hdf5", "adios", "bzip2", "zfp", "python"], "run": ["python", "py-numpy", "py-mpi4py"], "test": []}},
  {"...": "..."},
  {"name": "zsh",
   "latest_version": "5.4.2",
   "versions": ["5.4.2", "5.3.1", "5.1.1"],
   "homepage": "http://www.zsh.org",
   "file": "https://github.com/spack/spack/blob/develop/var/spack/repos/builtin/packages/zsh/package.py",
   "maintainers": [],
   "dependencies": {"build": ["pcre", "ncurses"], "link": ["pcre", "ncurses"], "run": [], "test": []}},
  {"name": "zstd",
   "latest_version": "1.4.0",
   "versions": ["1.4.0", "1.3.8", "1.3.0", "1.1.2"],
   "homepage": "http://facebook.github.io/zstd/",
   "file": "https://github.com/spack/spack/blob/develop/var/spack/repos/builtin/packages/zstd/package.py",
   "maintainers": [],
   "dependencies": {"build": [], "link": [], "run": [], "test": []}}
]

Generating a package index suitable for RESTful consumption in a static HTML service:

base_dir=$(pwd)/packages/
for pkg in $(spack list --format version_json | jq -c '.[]')
do
    name="$(echo ${pkg} | jq -r '.name')";
    first_letter=${name::1}
    mkdir -p ${base_dir}${first_letter}/
    echo ${pkg} > ${base_dir}${first_letter}/${name}.json
done
$ tree packages
packages/
├── a
│   ├── abinit.json
│   ├── abyss.json
│   ├── accfft.json
│   ├── ack.json
│   ├── activeharmony.json
│   ├── acts-core.json
│   ├── adept-utils.json
│   ├── adios2.json
│   ├── adios.json
...
├── y
│   ├── yajl.json
│   ├── yambo.json
│   ├── yaml-cpp.json
│   ├── yara.json
│   ├── yasm.json
│   └── yorick.json
└── z
    ├── z3.json
    ├── zeromq.json
    ├── zfp.json
    ├── zip.json
    ├── zlib.json
    ├── zoltan.json
    ├── zsh.json
    └── zstd.json

$ du -hs packages/                                                                                                       (topic-
13M	packages/
$ tree packages/ | wc -l                                                                                                     (topic-
3260

@ax3l ax3l added the commands label Jun 6, 2019
@ax3l ax3l requested review from adamjstewart and tgamblin June 6, 2019 22:38
@ax3l ax3l force-pushed the topic-listLatestJson branch 7 times, most recently from f5ed0a4 to d525120 Compare June 6, 2019 23:16
ax3l
ax3l commented Jun 6, 2019
View reviewed changes
lib/spack/spack/cmd/list.py Outdated
' "file": "{3}"'
'}}'.format(
pkg.name,
VersionList(pkg.versions).highest(),

This comment was marked as resolved.

Sign in to view

@ax3l ax3l marked this pull request as ready for review June 6, 2019 23:18
@ax3l ax3l force-pushed the topic-listLatestJson branch 7 times, most recently from ba6a08a to 402932d Compare June 7, 2019 06:35
alalazo
alalazo reviewed Jun 7, 2019
View reviewed changes
Copy link
Copy Markdown
Member

@alalazo alalazo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a couple of minor comments

lib/spack/spack/version.py Outdated
else:
return self[-1].highest()

def highest_numeric(self, fallback_all=False):
Copy link
Copy Markdown
Member

@alalazo alalazo Jun 7, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we return the highest numeric or the preferred version?

Copy link
Copy Markdown
Member Author

@ax3l ax3l Jun 7, 2019 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea, actually highest_numeric(True) is hopefully the preferred one in default settings... Any idea how to adjust this?
Alternatively it's probably ok for a first implementation.

@ax3l ax3l force-pushed the topic-listLatestJson branch 5 times, most recently from d9226ab to 20f58a4 Compare June 7, 2019 13:39
@ax3l ax3l changed the title spack list: latest version (JSON) spack list: format version_json Jun 7, 2019
@ax3l
spack list: latest version (JSON)
7b81df5 
List the latest version of each package in JSON encoding.
Preparation for consumption for a "spack badge" service.
@ax3l ax3l force-pushed the topic-listLatestJson branch from 20f58a4 to 7b81df5 Compare June 7, 2019 15:33
@ax3l ax3l mentioned this pull request Jun 7, 2019
Merged
2 tasks
@ax3l ax3l requested a review from alalazo June 7, 2019 19:01
@ax3l ax3l requested a review from becker33 June 7, 2019 19:07
tgamblin
tgamblin approved these changes Jun 7, 2019
View reviewed changes
lib/spack/spack/cmd/list.py
) for pkg in pkgs
])
out.write(pkg_latest)
# important: no trailing comma in JSON arrays
Copy link
Copy Markdown
Member

@tgamblin tgamblin Jun 7, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so irritating

@tgamblin tgamblin merged commit 61333dc into spack:develop Jun 7, 2019
@ax3l ax3l deleted the topic-listLatestJson branch June 7, 2019 21:16
@ax3l ax3l mentioned this pull request Jun 7, 2019
Merged
carsonwoods pushed a commit to carsonwoods/spack that referenced this pull request Jun 27, 2019
@ax3l @carsonwoods
spack list: latest version (JSON) (spack#11652)
2cc61fb 
List the latest version of each package in JSON encoding.
Preparation for consumption for a "spack badge" service.
dev-zero pushed a commit to dev-zero/spack that referenced this pull request Aug 13, 2019
@ax3l @dev-zero
spack list: latest version (JSON) (spack#11652)
2f781a3 
List the latest version of each package in JSON encoding.
Preparation for consumption for a "spack badge" service.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@tgamblin tgamblin tgamblin approved these changes

@adamjstewart adamjstewart Awaiting requested review from adamjstewart

@alalazo alalazo Awaiting requested review from alalazo

@becker33 becker33 Awaiting requested review from becker33

Assignees

No one assigned

Labels

commands

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@ax3l @tgamblin @alalazo