Merged
Conversation
GrahamcOfBorg
added
8.has: clean-up
Member
|
Not sure I understand why the hydra job would be necessary. Since 19.03 is still beta, we can just merge it into release-19.03 and see if the regular tests still work? |
Member
Member
Author
|
@bachp: I'll merge both, but there is something which I have missed in this patch. It doesn't affect us, but I would like to wait for an answer from upstream before merging both. |
aszlig
force-pushed
the
kernel-4.19-bump
branch
from
6b63661 to
eab4e4e
Compare
Member
Author
|
Just updated the patch to the latest version I submitted upstream. |
Our VM tests and everything related to our virtualisation infrastructure is currently broken if used with kernel 4.19 or later. The reason for this is that since 4.19, overlayfs uses the O_NOATIME flag when opening files in lowerdir and this doesn't play nice with the way we pass the Nix store to our QEMU guests. On a NixOS system, paths in the Nix store are typically owned by root but the QEMU process is usually run by an ordinary user. Using O_NOATIME on a file where you're not the owner (or superuser) will return with EPERM (Operation not permitted). This is exactly what happens in our VM tests, because we're using overlayfs in the guests to allow writes to the store. Another implication of this is that the default kernel version for NixOS 19.03 has been reverted to Linux 4.14. Work on getting this upstream is still ongoing and the patch I posted previously was incomplete, needs rework and also some more review from upstream maintainers - in summary: This will take a while. So instead of rushing in a kernel patch to nixpkgs, which will affect all users of overlayfs, not just NixOS VM tests, I opted to patch QEMU for now to ignore the O_NOATIME flag in 9p. I think this is also the least impacting change, because even if you care about whether access times are written or not, you get the same behaviour as with Linux 4.19 in conjunction with QEMU. Signed-off-by: aszlig <[email protected]> Fixes: NixOS#54509
This reverts commit 048c36c. With the patch applied for fixing the overlayfs bug in QEMU, there really shouldn't stand anything in our way to use 4.19 as the default kernel. Signed-off-by: aszlig <[email protected]>
aszlig
force-pushed
the
kernel-4.19-bump
branch
from
eab4e4e to
9a395a4
Compare
Member
Author
|
@samueldr, @lheckemann: Since this needs more time to resolve upstream, I decided to go with a QEMU patch for now, simply because it has the lowest impact on users. We can remove the patch as soon as this is fixed in the next round of stable kernels. |
GrahamcOfBorg
added
10.rebuild-darwin: 1-10
aszlig
added a commit
that referenced
this pull request
Mar 18, 2019
In Linux 4.19 there has been a major rework of the overlayfs implementation and it now opens files in lowerdir with O_NOATIME, which in turn caused issues in our VM tests because the process owner of QEMU doesn't match the file owner of the lowerdir. The crux here is that 9p propagates the O_NOATIME flag to the host and the guest kernel has no way of verifying whether that flag will lead to any problems beforehand. There is ongoing work to possibly fix this in the kernel, but it will take a while until there is a working patch and consensus. So in order to bring our default kernel back to 4.19 and of course make it possible to run newer kernels in VM tests, I'm merging a small QEMU patch as an interim solution, which we can drop once we have a working fix in the next round of stable kernels. Now we already had Linux 4.19 set as the default kernel, but that was subsequently reverted in 048c36c because the patch we have used was the revert of the commit I bisected a while ago. This patch broke overlayfs in other ways, so I'm also merging in a VM test by @bachp, which only tests whether overlayfs is working, just to be on the safe side that something like this won't happen in the future. Even though this change could be considered a moderate mass-rebuild at least for GNU/Linux, I'm merging this to master, mainly to give us some time to get it into the current 19.03 release branch (and subsequent testing window) once we got no new breaking builds from Hydra. Cc: @samueldr, @lheckemann Fixes: #54509 Fixes: #48828 Merges: #57641 Merges: #54508
aszlig
added a commit
that referenced
this pull request
Mar 21, 2019
In Linux 4.19 there has been a major rework of the overlayfs implementation and it now opens files in lowerdir with O_NOATIME, which in turn caused issues in our VM tests because the process owner of QEMU doesn't match the file owner of the lowerdir. The crux here is that 9p propagates the O_NOATIME flag to the host and the guest kernel has no way of verifying whether that flag will lead to any problems beforehand. There is ongoing work to possibly fix this in the kernel, but it will take a while until there is a working patch and consensus. So in order to bring our default kernel back to 4.19 and of course make it possible to run newer kernels in VM tests, I'm merging a small QEMU patch as an interim solution, which we can drop once we have a working fix in the next round of stable kernels. Now we already had Linux 4.19 set as the default kernel, but that was subsequently reverted in 048c36c because the patch we have used was the revert of the commit I bisected a while ago. This patch broke overlayfs in other ways, so I'm also merging in a VM test by @bachp, which only tests whether overlayfs is working, just to be on the safe side that something like this won't happen in the future. Even though this change could be considered a moderate mass-rebuild at least for GNU/Linux, I'm merging this to master, mainly to give us some time to get it into the current 19.03 release branch (and subsequent testing window) once we got no new breaking builds from Hydra. Cc: @samueldr, @lheckemann Fixes: #54509 Fixes: #48828 Merges: #57641 Merges: #54508 (cherry picked from commit 12efcc2)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This should bring back kernel 4.19 as our default kernel by applying a patch that fixes our overlayfs regression (#54509).
I'm basing this against
masterfirst and we can backport it to stable later once we got it tested well enough™.Tested this only against the
kernel-latestNixOS test so far, so we should probably add a Hydra job to make sure this doesn't introduce additional breakages.