Improve support for relative path inputs#10089
Merged
edolstra merged 25 commits intoNixOS:masterfrom Jan 16, 2025
Merged
Conversation
github-actions
bot
added
documentation
new-cli
|
🎉 All dependencies have been resolved ! |
thufschmitt
added
the
idea approved
Member
|
Discussed during the Nix maintainers meeting on 2024-03-04.
Details
|
|
This pull request has been mentioned on NixOS Discourse. There might be relevant details there: |
edolstra
added a commit
to edolstra/nix
that referenced
this pull request
Apr 15, 2024
…re path This is needed for the path:// input scheme (until it returns a FSInputAccessor to the original path, but that's blocked by NixOS#10089) and the Mercurial input scheme.
edolstra
added a commit
to edolstra/nix
that referenced
this pull request
Apr 19, 2024
…re path This is needed for the path:// input scheme (until it returns a FSInputAccessor to the original path, but that's blocked by NixOS#10089) and the Mercurial input scheme.
edolstra
force-pushed
the
relative-flakes
branch
from
47ccb1d to
690281f
Compare
Subflakes are flakes in the same tree, accessed in flake inputs via
relative paths (e.g. `inputs.foo.url = "path:./subdir"`). Previously
these didn't work very well because they would be separately copied to
the store, which is inefficient and makes references to parent
directories tricky or impossible. Furthermore, they had their own NAR
hash in the lock file, which is superfluous since the parent is
already locked.
Now subflakes are accessed via the accessor of the calling flake. This
avoids the unnecessary copy and makes it possible for subflakes to
depend on flakes in a parent directory (so long as they're in the same
tree).
Lock file nodes for relative flake inputs now have a new `parent` field:
{
"locked": {
"path": "./subdir",
"type": "path"
},
"original": {
"path": "./subdir",
"type": "path"
},
"parent": [
"foo",
"bar"
]
}
which denotes that `./subdir` is to be interpreted relative to the
directory of the `bar` input of the `foo` input of the root flake.
Extracted from the lazy-trees branch.
`parentNode.sourceInfo.outPath` does not include the subdir of the parent flake, while `parentNode.outPath` does. So we need to use the latter.
edolstra
added a commit
to edolstra/nix
that referenced
this pull request
Sep 11, 2024
Relative path flakes ("subflakes") are basically fundamentally
broken, since they produce lock file entries like
"locked": {
"lastModified": 1,
"narHash": "sha256-/2tW9SKjQbRLzfcJs5SHijli6l3+iPr1235zylGynK8=",
"path": "./flakeC",
"type": "path"
},
that don't specify what "./flakeC" is relative to. They *sometimes*
worked by accident because the `narHash` field allowed
`fetchToStore()` to get the store path of the subflake *if* it
happened to exist in the local store or in a substituter.
Subflakes are properly fixed in NixOS#10089 (which adds a "parent" field to
the lock file). Rather than come up with some crazy hack to make them
work in the interim, let's just disable the only test that depends on
the broken behaviour for now.
…Jobs.tests.functional_user
Contributor
|
I'm pretty sure I was in the subdirectory, the top level flake didn't have any nixosConfigurations. I'll try to see what's going on tomorrow, thanks for the link to the tests. |
Contributor
|
@roberth, alright, I was doing something wrong, it works. Sorry for bothering. However I think I found out why I was confused, I think it's an issue with
{
inputs.nixpkgs.url = "github:nixos/nixpkgs/nixos-unstable";
outputs = inputs: {
nixosConfigurations.test = inputs.nixpkgs.lib.nixosSystem {
modules = [
{
nixpkgs.system = "aarch64-linux";
fileSystems."/".device = "/dev/null";
boot.loader.systemd-boot.enable = true;
environment.etc.my-test.text = builtins.readFile ../test;
}
];
};
};
}All of the following is executed in Test 1. $ nix repl
nix-repl> :lf .
nix-repl> nixosConfigurations.test.config.environment.etc.my-test.text
"Test\n"This works as I expect it to work. Test 2. This works as I expect it to work. Test 3. I expected Can you confirm if this is a bug, at least conceptually? |
|
This pull request has been mentioned on NixOS Discourse. There might be relevant details there: |
|
This pull request has been mentioned on NixOS Discourse. There might be relevant details there: |
2 tasks
aiotter
added a commit
to aiotter/neovim
that referenced
this pull request
Mar 28, 2025
Flake can now be referenced by the relative path! NixOS/nix#10089
2 tasks
roberth
reviewed
Apr 14, 2025
Comment on lines
+44
to
+54
| parentNode = allNodes.${getInputByPath lockFile.root node.parent}; | ||
|
|
||
| sourceInfo = | ||
| if overrides ? ${key} | ||
| then | ||
| overrides.${key}.sourceInfo | ||
| else if node.locked.type == "path" && builtins.substring 0 1 node.locked.path != "/" | ||
| then | ||
| parentNode.sourceInfo // { | ||
| outPath = parentNode.outPath + ("/" + node.locked.path); | ||
| } |
Member
There was a problem hiding this comment.
This introduces an unnecessary dependency from the current sourceInfo onto the result of outputs of the parent flake. It should only be strict in the fetchTree result of the parent, not the outputs.
The full chain (all WHNF) is something like sourceInfo (current) -> parentNode -> allNodes.${...} -> if (line 81) -> result -> outputs // ... -> outputs.
Reproducer:
mkdir -p ~/tmp/repro-10089/example
cd ~/tmp/repro-10089
git init
echo '{ outputs = _: { }; }' >flake.nix
cat >example/flake.nix <<EOF
{
inputs.parent.url = ../.;
outputs = { self, parent, nixpkgs, ... }: builtins.seq parent {};
}
EOF
git add -N .
cd example
# might succeed:
nix build
# fails:
nix build
->
error: stack overflow; max-call-depth exceeded
The fact that it succeeds the first time suggests that a freshly locked lock is treated in a significantly different way regarding flake directories, which may be a separate problem if there's other differences than just this failing.
nix build --recreate-lock-file --no-write-lock-file always works.
This was referenced Apr 14, 2025
MattSturgeon
added a commit
to MattSturgeon/flake-compat
that referenced
this pull request
May 12, 2025
Based on the upstream implementation added in Nix 2.26 See NixOS/nix#10089
MattSturgeon
added a commit
to MattSturgeon/flake-compat
that referenced
this pull request
May 12, 2025
Based on the upstream implementation added in Nix 2.26 See NixOS/nix#10089
MattSturgeon
pushed a commit
to MattSturgeon/flake-compat
that referenced
this pull request
May 12, 2025
Cherry-picked from NixOS/nix#10089
MattSturgeon
pushed a commit
to MattSturgeon/flake-compat
that referenced
this pull request
May 12, 2025
Cherry-picked from NixOS/nix#10089
MattSturgeon
pushed a commit
to MattSturgeon/flake-compat
that referenced
this pull request
May 12, 2025
Cherry-picked from NixOS/nix#10089
MattSturgeon
pushed a commit
to MattSturgeon/flake-compat
that referenced
this pull request
May 19, 2025
(cherry picked from PR NixOS/nix#10089)
|
This pull request has been mentioned on NixOS Discourse. There might be relevant details there: |
8 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
10 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
We want it to be possible to refer to flake inputs that reside in the same repo via relative paths (e.g.
inputs.foo.url = "path:./subdir"). Previously this didn't work very well because those inputs would be separately copied to the store, which is inefficient and makes references to parent directories tricky or impossible. Furthermore, they had their own NAR hash in the lock file, which is superfluous since the parent is already locked. In fact, if relative path inputs worked at all, it was by accident.Now relative path inputs are accessed via the accessor of the calling flake. This avoids the unnecessary copy and makes it possible for relative path inputs to depend on flakes in a parent directory (so long as they're in the same tree).
Lock file nodes for relative path inputs now have a new
parentfield:which denotes that
./subdiris to be interpreted relative to the directory of thebarinput of thefooinput of the root flake.Extracted from the lazy-trees branch.
Context
Priorities and Process
Add 👍 to pull requests you find important.
The Nix maintainer team uses a GitHub project board to schedule and track reviews.