spack logs: added new command by alalazo · Pull Request #6166 · spack/spack

alalazo · 2017-11-06T22:09:19Z

Added a new command spack log show, that mimics docker logs:

$ spack log show libszip
#
# ... dumps the file ...
#

It takes a constraint that identifies a unique spec, and dumps its build.out. The log can be piped to other linux commands (such as grep or less). Errors are handled gracefully:

$ spack log show foobar
==> Error: Spec 'foobar' matches no installed packages.

$ spack log show foobar foo
==> Error: only one spec is allowed in the query [2 given]

$ spack log show %gcc
==> Error: %gcc matches multiple packages.
  Matching packages:
    tfkam54 [email protected]%[email protected] arch=linux-ubuntu14.04-x86_64 
    ...
    eksallf [email protected]%[email protected] arch=linux-ubuntu14.04-x86_64 
  Use a more specific spec.

The command spack log-parse has been moved to spack log parse and now accepts either a file or a spec:

$ spack log parse -s zlib
0 errors

$ spack log parse -s tcl
1 errors
     655    checking for 64-bit integer type... using long
     656    checking for build with symbols... no
     657    checking for tclsh... /tmp/pytest-of-mculpo/pytest-0/test_keep_exceptions0/tmp/spack-stage/spack-stage-aUKyP3/tcl8.6.6/unix/tclsh
  >> 658    /tmp/pytest-of-mculpo/pytest-0/test_keep_exceptions0/tmp/spack-stage/spack-stage-aUKyP3/tcl8.6.6/pkgs/tdbc1.0.4/configure: line 8999: cd: /home/mculpo/PycharmProjects/spack/opt/spack/linux-ubuntu14.
            04-x86_64/gcc-4.8/tcl-8.6.6-xrem43kcgm3ij2drw3vqhnxvhzi3yonf/lib/tdbc1.0.4: No such file or directory
     659    configure: creating ./config.status
     660    config.status: creating Makefile
     661    config.status: creating pkgIndex.tcl

tgamblin

@alalazo: see my comments -- I'm not sure we need this command as it seems non-orthogonal to some existing ones that could do the same thing. See what you think.

I think it would be useful if it dumped the log, as that would allow someone to grep for errors. But beyond that maybe it should be an add-on to either spack edit or spack env (soon to be spack build-env).

tgamblin · 2017-11-07T23:48:12Z

lib/spack/spack/cmd/logs.py

+        spack.cmd.display_specs(specs, args)
+        raise SystemExit(1)
+
+    spec = specs[0]


just use spack.cmd.disambiguate_spec() and you can delete everything from here up.

Thanks for the tip.

tgamblin · 2017-11-07T23:48:58Z

lib/spack/spack/cmd/logs.py

+
+    spec = specs[0]
+    if spec.external:
+        msg = '{0} is an external. External specs have no logs.'


'{0} is an external package and has no logs.'

tgamblin · 2017-11-08T00:09:23Z

lib/spack/spack/cmd/logs.py

+        tty.die(msg.format_map(spec.short_spec))
+
+    # Construct the filename to be opened
+    filename = 'build.env' if args.environment else 'build.out'


why not make several arguments that use store_const with these string values and a single filename as dest? Then the parser will build the name for you.

tgamblin · 2017-11-08T00:11:35Z

lib/spack/spack/cmd/logs.py

+
+def setup_parser(subparser):
+    subparser.add_argument(
+        '--environment',


I would call this --build-env or --build-environment. I'm somewhat conflicted about whether this should be an argument to this command or to spack env, e.g. spack env --installed <spec>. The latter would also let you actually run a command in the old environment, which could be very useful.

Let me know if you want this command at all, or if you want just want

spack env --install <spec>

For the what is worth, the motivation that lead me putting this PR together was just being tired to repeat:

$ spack find -p <spec> ==> 1 installed packages. -- linux-ubuntu14.04-x86_64 / [email protected] --------------------------- [email protected] /home/mculpo/PycharmProjects/spack/opt/spack/linux-ubuntu14.04-x86_64/gcc-4.8/zlib-1.2.11-eksallf6cymqkp6pkz6ymzjakqt6bqkx $ less <cut'n paste of the path above>/.spack/build.out

over and over when looking at previous installations logs. Having a dump of the environment was just a low hanging fruit once opening the logs had been implemented.

How about just these:

spack env --install <spec>

spack edit --log <spec>

or just the latter if you're short on time? I think the use case you describe is a good one, we just already have spack edit

PS -- spack edit probably only need another arg and a call to disambiguate spec in edit_package to make this work

tgamblin · 2017-11-08T00:14:07Z

lib/spack/spack/cmd/logs.py

+        msg = 'log file does not exist! [{0}]'
+        tty.die(msg.format(abs_filename))
+
+    spack.editor(abs_filename)


I think if this is just going to edit the file, it should be implemented as an argument to spack edit. e.g., spack edit --build-log <spec>.

Similarly, if it is going to print the filename, there should probably be an argument to spack location.

What we don't have currently is something to just dump the log, so maybe this command should just do that? Seems like it should also be called spack show-log if it does that. But see my comments about about overlap with spack env for the environment log.

spack.editor was the fastest way I could think of to open the logs into something paged.

I would really like to be able to somehow open the file read only, but couldn't find a decent way to do it that works for every editor I am aware of. An alternative, to avoid accidental corruptions of the logs, is to copy the logs in some tmp space and open the copy + delete it on editor exit.

Note that if you want just a dump, for instance within a script, you can:

export EDITOR=cat spack logs <spec>

Does this seem reasonable to you?

I guess it seems kind of awkward? What about just:

spack show-log <spec>

alalazo · 2017-11-08T13:01:53Z

@tgamblin Just dumping the log right now. Will do a PR for:

spack env --install <spec>

later on. Also, not sure if in the end you wanted:

spack show-log <spec>

or directly a sub-command:

spack log show <spec>

Just post your preferred, I'll change the naming here.

alalazo · 2017-11-21T05:10:07Z

ping

alalazo · 2017-12-22T08:28:03Z

I think it would be useful if it dumped the log, as that would allow someone to grep for errors.

I think I addressed the comment. Let me know if there's something else to do here.

tgamblin · 2018-01-28T19:04:53Z

@alalazo I kind of like spack log show here. I think that gives us some flexibility for adding new stuff.

Speaking of which, see #7093. Should that be added here? The only thing is that spack log-parse there takes a file, not a spec. Maybe it should be merged with this as

spack log parse [-f file | spec]

alalazo · 2018-02-13T11:37:35Z

@tgamblin I agree with merging these two as:

spack log parse [-f file | spec]
spack log show [-f file | spec]

Ok if I proceed with that in this PR?

tgamblin · 2018-02-13T22:13:37Z

sure! go ahead

alalazo · 2018-04-07T16:25:24Z

@tgamblin Ready for another review!

EDIT: In the end the sub-commands work like:

spack log parse [-f file | -s spec]
spack log show spec

Added a new command `spack logs`, that mimics `docker logs`. It takes a contraint that identifies a unique spec, and dumps its build.out.

The functionality previously in `spack logs` and `spack log-parse` has been moved to sub-commands of `spack log`. `spack log parse` now accepts an installed spec as an argument and converts it into the corresponding `build.out` file to be parsed.

alalazo · 2018-05-21T11:19:07Z

ping

alalazo added new-command ready labels Nov 6, 2017

alalazo mentioned this pull request Nov 7, 2017

SC17: advanced packaging tutorial #6148

Merged

3 tasks

tgamblin requested changes Nov 8, 2017

View reviewed changes

tgamblin removed the ready label Nov 8, 2017

alalazo force-pushed the features/logs_command branch from db83941 to 16405ea Compare November 8, 2017 12:51

alalazo force-pushed the features/logs_command branch 2 times, most recently from 02c510f to 64e400a Compare April 7, 2018 13:19

alalazo and others added 4 commits April 19, 2018 16:33

spack logs: added new command

1dc475c

Added a new command `spack logs`, that mimics `docker logs`. It takes a contraint that identifies a unique spec, and dumps its build.out.

Merged spack log-parse into spack log

5755d8a

The functionality previously in `spack logs` and `spack log-parse` has been moved to sub-commands of `spack log`. `spack log parse` now accepts an installed spec as an argument and converts it into the corresponding `build.out` file to be parsed.

Improved coverage of spack log show

874b8a6

Improved coverage of spack log parse

a78be63

alalazo force-pushed the features/logs_command branch from c3537f9 to a78be63 Compare April 19, 2018 14:35

alalazo requested a review from adamjstewart April 24, 2018 05:49

alalazo mentioned this pull request May 2, 2018

Extend config.yaml to read locations where to search for commands #7967

Closed

alalazo closed this Dec 2, 2019

alalazo deleted the features/logs_command branch December 2, 2019 09:11

Conversation

alalazo commented Nov 6, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tgamblin left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

alalazo Nov 8, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

alalazo commented Nov 8, 2017

Uh oh!

alalazo commented Nov 21, 2017

Uh oh!

alalazo commented Dec 22, 2017

Uh oh!

tgamblin commented Jan 28, 2018

Uh oh!

alalazo commented Feb 13, 2018

Uh oh!

tgamblin commented Feb 13, 2018

Uh oh!

alalazo commented Apr 7, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

alalazo commented May 21, 2018

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

alalazo commented Nov 6, 2017 •

edited

Loading

alalazo Nov 8, 2017 •

edited

Loading

alalazo commented Apr 7, 2018 •

edited

Loading