-
-
Notifications
You must be signed in to change notification settings - Fork 4.7k
Fixes binary proxy to support spaces in source path #12524
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
good catch, thanks. Merged in 2.8 here 54119de |
|
Hello guys, Just for your information this minor change broke many PHP builds in Nix because the changes modifies the content of the I wonder what kind of guardrails we could setup to avoid this kind of situation in the future. Do you have any idea? |
|
Let me provide some context about Nix and why stability in the What is Nix?Nix is a purely functional package manager and build system. It’s best known for powering the NixOS Linux distribution but can also run on other Linux systems and macOS. For PHP projects, Nix can build and package applications (including their Composer-managed dependencies). The Why is output stability important?In Nix, the hash of a build output (here, the Because each PHP application embeds the hash of its The recent change in this PR (for very good reasons actually) modified the generated proxy script, which in turn altered the contents of the Why does this matter?
How to prevent this in the futureTo avoid similar issues, the That would help downstream systems like Nix anticipate necessary hash updates and coordinate transitions more smoothly. For example, when the PR to bump Composer in Nix would be created, all the updated hashes would need to be updated as well, all at once. Edit: I dedicated a bit of time tonight to build a test that would check that the output is stable, find it at #12574 Out of curiosity, I ran ShellCheck outputIf these small fixes are ever integrated into Composer (which would be great!), it would means that all the PHP packages in Nix would be broken and will need a manual intervention to update the vendor hash. If possible, a short note in the Composer changelog would help us to know if the resulting output of composer has been updated from a version to another. Hope this helps, let me know if this is not the case. |
It seems like this assumption is missing a piece: the version of Composer used. |
You're right that the Composer version could in theory influence the output, but in practice that hasn’t been the case and that’s precisely why Nix doesn’t need to account for it. We’ve upgraded Composer within the same major version (v2) many times in the past without any reproducibility issues or changes to the generated vendor directory. As long as the Composer team preserves backward-compatible behaviour within a major version, Nix can safely assume that a given composer.json + composer.lock combination will always produce identical outputs. That’s why this situation is quite uncommon. IIRC, it’s the first time we’ve observed such a change affecting reproducibility. |
|
This issue led to the creation of a PHP package that implements the NAR format designed by Eelco Dolstra (the original Nix author). By creating an external package that implements this specification, it can now be reused everywhere else. I think I made it in an efficient way, if you think it can be improved, feel free to contribute to the project. The project is available at https://github.com/loophp/path-hasher In #12574, I updated the code to use that library. I am not expecting this pull request to be merged (but perhaps I am wrong?) but rather open a discussion on how we could avoid issue in the future when it comes to the PHP code generated by Composer. Also, that PR is incomplete because it doesn't take into account the Composer proxy binaries mechanism wrapper. |
That was just luck. I am totally with you on producing deterministic output for a given version, but things within the vendor/composer/* and vendor/bin/* dirs are definitely going to keep changing every now and then. We cannot bump the major version every time. |
I ran across an issue while using Drush and calling a command that starts a subprocess (such as
updatedb) and found that the binary proxy does not handle spaces in the source path.In my environment, I have my Composer/Drupal install at a patch which has a space in the name (i.e.
/home/jenkins/workspace/Job Name). I attempted to callvendor/bin/drush updatedband got an error similar to:The
drushcommand started correctly, but when it attempted to start a subprocess ('/home/jenkins/workspace/Job Name/vendor/bin/drush' updatedb:status --no-cache-clear --strict=0 --uri=default), it could not find drush.I found the root cause to be in the binary proxy where it calls
realpath:selfwas becoming/home/jenkins/workspace/Job.