Support limiting attributes and list items in value printing to support detailed errors#9606
Support limiting attributes and list items in value printing to support detailed errors#9606roberth merged 3 commits intoNixOS:masterfrom
Conversation
NixOS/nix#9606 changes how values are printed in some cases. This updates the lib tests to work with the new output format, which should be familiar to `nix repl` users.
flake.nix
Outdated
| # https://github.com/NixOS/nixpkgs/pull/271423, don't forget | ||
| # to remove the `nix.checkAllErrors = false;` line in the tests. | ||
| inputs.nixpkgs.url = "github:NixOS/nixpkgs/staging-23.05"; | ||
| inputs.nixpkgs.url = "github:9999years/nixpkgs/update-tests-for-nix-9606-staging-23.05"; |
There was a problem hiding this comment.
This can be removed when NixOS/nixpkgs#274077 is merged, just a hack to get tests passing.
The lib test suite has a few test cases that check the output of nix-instantiate --eval-only. I don't think that output is guaranteed by any compatibility policy, so I think we're good to change it, but we do need to patch the tests. Once we have the output finalized here, we can merge this PR in nixpkgs and then update the Flake input reference and remove this patch.
There was a problem hiding this comment.
For some reason updating the nixpkgs input makes nix flake check want to rebuild LLVM and GHC? So that's hitting the 1hr CI timeout, but nix flake check does pass locally.
There was a problem hiding this comment.
I don't think that output is guaranteed by any compatibility policy
I would feel the same, but in Nix that thinking tends to lead to problems for users.
We're currently already interacting with Nixpkgs a bit, so that makes merging this line a bit hard.
To get this merged, I'd prefer to keep the "ascii" syntax around, perhaps through a bool flag?
There was a problem hiding this comment.
rebuild LLVM and GHC
I would expect this to be resolved now, but I'm surprised that this happened. We do care about avoiding this.
There was a problem hiding this comment.
I'm beginning to think the test suite should just use the JSON output instead. Also an easy change to make: NixOS/nixpkgs#275264
There was a problem hiding this comment.
Any machine parsing like the test suite should definitely use the JSON output. If the output of human-interface CLI tools shouldn't be changed to improve readability then that defeats the purpose of having CLI tools
There was a problem hiding this comment.
And more broadly, if we want to be able to actually improve the experience of using Nix, we have to be comfortable saying "no, that's not part of the public interface, you'll need to change your code because we haven't made a guarantee to keep this stable and writing a shell script that depends on it doesn't change that".
It sounds like there is need for a stable output layer that can encode (e.g.) thunks and functions, so maybe we should add a second JSON output layer that has an actual public schema that we can stick to. EDIT: It sounds like this is the purpose of the XML output layer that already exists! We should leverage that.
We'd just add another layer to disambiguate types, so we'd encode a thunk as {"type": "thunk"} (possibly with auxiliary information like where it comes from, if it's a cycle, etc.), and we could encode an attrset as {"type": "attrset", "values": {...}}. This would be machine-readable (without writing a Nix parser!) and easy to keep stable.
But in the meantime there don't exist any use-cases that can't be easily patched by using the JSON output and (for checking laziness) checking the error message for "cannot convert thunk to JSON".
There was a problem hiding this comment.
Marking this resolved as we got NixOS/nixpkgs#275264 merged and I updated the Nixpkgs input, so we're all good to release now.
There was a problem hiding this comment.
If I remember correctly: This is annoying for the Tvix people (cc @sternenseemann) because they reuse the Nix language test suite as a compatibility check, and that test suite uses the result of nix-instantiate --eval.
I don't think that's really right (as you said, it's not great to rely on that output), but since it is the de facto canonical language test, there's not a lot of leeway.
All that to say, changing the pretty printer would probably require changing the test suite first to not rely on the pretty printing (but use the XML or JSON format) first.
roberth
left a comment
There was a problem hiding this comment.
Overall, this is a very nice change!
I've made some suggestions to further improve the output. Prefixed with "thought:" they are optional.
I'd like to merge this soon, but I'm not confident about the change in output format.
If you could support the existing "nix-instantiate" or "ASCII" format, that means we can merge this PR very soon.
There was a problem hiding this comment.
It seems that the purpose of this header is to resolve a cyclic dependency between value.hh and print.hh.
If you were to use a reference type in value.hh, you could use a forward declaration struct PrintOptions; in value.hh instead. That way we can keep the actual declaration close to its use and avoid the need for this header file.
There was a problem hiding this comment.
If I do that, I have to remove the default options value:
struct Value {
void print(EvalState &state, std::ostream &str, PrintOptions & options);
};Then I have to patch the callsites and I get errors like this:
src/nix-instantiate/nix-instantiate.cc:59:46: error: taking the address of a temporary object of type 'nix::PrintOptions' [-Waddress-of-temporary]
vRes.print(state, std::cout, &PrintOptions{});
I can fix it like this, but it feels a lot less "clean":
auto options = PrintOptions {};
vRes.print(state, std::cout, options);
I think our header files are way too cluttered anyways, and they would be a lot easier to work with if each type had a separate header.
NixOS/nix#9606 changes how values are printed in some cases. This updates the lib tests to work with the new output format, which should be familiar to `nix repl` users.
d77a3dc to
aafc154
Compare
|
It's easy to tell me to change a very old and stable command, because you don't necessarily need to deal with the fallout. Your pushing the responsibility to deal with the change onto the Nix team and other users. I know that's not how you intend it, but it is how it works out. |
|
This pull request has been mentioned on NixOS Discourse. There might be relevant details there: |
|
The improved Alternatively, why not make this nice, improved output for Currently, ## this one is actually fine; it has been cleverly expanded:
$ nix eval --expr '{ foo = { }; }'
{ foo = { }; }
## this one is still not good:
$ nix eval --expr '{ foo = (x: x); }'
{ foo = <LAMBDA>; }
$ nix --version
nix (Nix) 2.20.0pre20240106_8e865f3 |
Stable and ambiguous? My argument is:
|
There was a problem hiding this comment.
It seems that the limits aren't applied yet, except in the tests. I think that's ok for now, and I'll be curious to see how they affect real usage when applied. We can revisit the details later as needed.
I've left a few more comments on the code, and I feel like this is getting close to completion now.
Previously, there were two mostly-identical value printers -- one in `libexpr/eval.cc` (which didn't force values) and one in `libcmd/repl.cc` (which did force values and also printed ANSI color codes). This PR unifies both of these printers into `print.cc` and provides a `PrintOptions` struct for controlling the output, which allows for toggling whether values are forced, whether repeated values are tracked, and whether ANSI color codes are displayed. Additionally, `PrintOptions` allows tuning the maximum number of attributes, list items, and bytes in a string that will be displayed; this makes it ideal for contexts where printing too much output (e.g. all of Nixpkgs) is distracting. (As requested by @roberth in NixOS#9554 (comment)) Please read the tests for example output. Future work: - It would be nice to provide this function as a builtin, perhaps `builtins.toStringDebug` -- a printing function that never fails would be useful when debugging Nix code. - It would be nice to support customizing `PrintOptions` members on the command line, e.g. `--option to-string-max-attrs 1000`.
The Nix team has requested that this output format remain unchanged. I've added a warning to the man page explaining that `nix-instantiate --eval` output will not parse correctly in many situations.
|
Great work @9999years, thank you! |
| </expr> | ||
| ``` | ||
|
|
||
| Note that `y` is left unevaluated (the XML representation doesn’t |
There was a problem hiding this comment.
y no longer exists in the example printouts.
| > | ||
| > This option produces ambiguous output which is not suitable for machine | ||
| > consumption. For example, these two Nix expressions print the same result | ||
| > despite having different types: |
There was a problem hiding this comment.
Arguably they have the same type at this time, tThunk, maybe “despite not being equal” would be better? In any case, this is sort of the point of not forcing the value completely, nix eval will also print the same output for non-equal values/values where the type isn't the same.
Wouldn't be clearer to just say that the problem with the output is that it produces something that can be parsed as a Nix expression, but would not return the same result?
There was a problem hiding this comment.
I agree and actually it is ambiguous what the ambiguity is. It can be either
- intentional ambiguity because
--strictwasn't specified, - or ambiguity in the mind of the reader, who might "naively" (can't blame) read it is an expression instead of a mere value, while non-value expressions are not printed by this command.
rant: why lawyering about "non-value expression"
I don't think we should ever conflate values and their expression literal counterparts, but someone might take that view and I had to be explicit. "expression" should have been enough IMO. The thing represented by Expr.
We really need to define some language terminology in the manual, but I do feel that we should just improve this text first.
Motivation
Many Nix error messages are misleading or confusing due to unhelpful stack trace information. We could improve these messages by printing out relevant values (see #561, #9554, #9553). However, the value printing code in Nix doesn't have a notion of limited output, which makes it unsuitable for printing arbitrary values, which could (for example) be all of Nixpkgs.
This PR unifies and expands on the value printing logic as a prerequisite for reopening #9553 and #9554.
Context
Previously, there were two mostly-identical value printers -- one in
libexpr/eval.cc(which didn't force values) and one inlibcmd/repl.cc(which did force values and also printed ANSI color codes):nix/src/libcmd/repl.cc
Line 904 in 1b7968e
nix/src/libexpr/eval.cc
Lines 107 to 108 in 1b7968e
This PR unified both of these printers into
print.ccand provides aPrintOptionsstruct for customizing the output, which allows for toggling whether values are forced, whether repeated values are tracked, and whether ANSI color codes are displayed. However, the oldnix-instantiateoutput format has been reinstated.Additionally,
PrintOptionsallows tuning the maximum number of attributes, list items, and bytes in a string that will be displayed; this makes it ideal for contexts where printing too much output (e.g. all of Nixpkgs) is distracting. (As requested by @roberth in #9554 (comment))Please read the tests for example output.
Future work
builtins.toStringDebug-- a printing function that never fails would be useful when debugging Nix code.PrintOptionsmembers on the command line, e.g.--option to-string-max-attrs 1000.Priorities
Add 👍 to pull requests you find important.