Skip to content

workflows/eval: Add the eval summary as a comment#361061

Merged
Mic92 merged 1 commit intoNixOS:masterfrom
azuwis:push-tnyqlukqomuv
Dec 3, 2024
Merged

workflows/eval: Add the eval summary as a comment#361061
Mic92 merged 1 commit intoNixOS:masterfrom
azuwis:push-tnyqlukqomuv

Conversation

@azuwis
Copy link
Contributor

@azuwis azuwis commented Dec 2, 2024

See it in action azuwis#9

Make #360339 more discoverable.

Will post a comment like:


Eval summary

  • Added packages: 0
  • Removed packages: 0
  • Changed packages: 1
  • Rebuild Linux: 1
  • Rebuild Darwin: 0

Compared to #360962:

  1. Only post single comment, and update it on subsequent push, reduce noise for long-running pull requests
    See workflows/eval: Add the step summary as a comment #360962 (comment)
  2. Only summary and link, no details, will not affect GH search results
    See workflows/eval: Add the step summary as a comment #360962 (comment)

Things done

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
    • sandbox = relaxed
    • sandbox = true
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 25.05 Release Notes (or backporting 24.11 and 25.05 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

Add a 👍 reaction to pull requests you find important.

@github-actions github-actions bot added 6.topic: policy discussion Discuss policies to work in and around Nixpkgs 6.topic: continuous integration Affects continuous integration (CI) in Nixpkgs, including Ofborg and GitHub Actions 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux. labels Dec 2, 2024
@azuwis
Copy link
Contributor Author

azuwis commented Dec 2, 2024

@Mic92 @FliegendeWurst What do you think?

@FliegendeWurst
Copy link
Member

I think the main benefit here is the link to the evaluation results. Numbers are already provided by the labels

@azuwis azuwis marked this pull request as ready for review December 3, 2024 01:54
@Mic92 Mic92 merged commit 260aa26 into NixOS:master Dec 3, 2024
@Mic92
Copy link
Member

Mic92 commented Dec 3, 2024

works: #361344 (comment)

@azuwis azuwis deleted the push-tnyqlukqomuv branch December 3, 2024 08:27
@wolfgangwalther
Copy link
Contributor

wolfgangwalther commented Dec 3, 2024

Is this going to be posted on every force push?

If yes: strong 👎

Edit: I see --edit-last is used. That should work.

@infinisil
Copy link
Member

Hmm not sure if I like having such a comment on every single PR

@azuwis
Copy link
Contributor Author

azuwis commented Dec 3, 2024

How about only post the comment, if more than 10 packages are added/removed/changed? Like this #361365 (comment)

@wolfgangwalther
Copy link
Contributor

It would be great if this could somehow be integrated with the job requesting reviews from OWNERS. What I mean is that it would be great if the bot could post a comment before the reviews are requested and then would update it later on... that could reduce the number of notifications a bit? You'd only receive a notification for review once and not a second one for the comment a bit later.

@Mic92
Copy link
Member

Mic92 commented Dec 3, 2024

Does requesting reviews actually creates a notification? I don't think it did this for me in the past. I agree with @infinisil I noticed that I found myself having to open updated pull requests more often just to see it was only yet another bot comment: #361388

@wolfgangwalther
Copy link
Contributor

Does requesting reviews actually creates a notification?

If that was not the case, then those review requests would be useless. I get emails like this:

@nix-owners[bot] requested your review on ...

And then a few minutes later another comment about the eval result - which I really don't care about at this stage. I might care about it for my own PRs, but also... not really. I would really like to see the evaluation results on a quick glance once I look at the PR anyway, but I don't want to be notified about them.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

6.topic: continuous integration Affects continuous integration (CI) in Nixpkgs, including Ofborg and GitHub Actions 6.topic: policy discussion Discuss policies to work in and around Nixpkgs 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants