Skip to content

treewide: replace substituteAll with replaceVars (#1)#365440

Merged
philiptaron merged 1 commit intoNixOS:masterfrom
wolfgangwalther:refactor-tractor-1
Dec 16, 2024
Merged

treewide: replace substituteAll with replaceVars (#1)#365440
philiptaron merged 1 commit intoNixOS:masterfrom
wolfgangwalther:refactor-tractor-1

Conversation

@wolfgangwalther
Copy link
Contributor

Some more gifts from the refactor tractor.

Driving towards #237216

Those are a sub-set of really simple-to-replace cases. I ran the following script:

  find pkgs -type f -iname '*.nix' -print0 | \
    xargs -0 -n100 perl -0777 -i -pe \
      's/substituteAll {([^}]*?)\n[ ]*src =([^;]+);/replaceVars\2 \{\1/gs'

  git ls-files -m | xargs sed -i -E \
    -e 's/\{ substituteAll\b/{ replaceVars/g' \
    -e 's/, substituteAll\b/, replaceVars/g' \
    -e 's/substituteAll,/replaceVars,/g' \
    -e 's/substituteAll;/replaceVars;/g'

but then only selected a smaller part of the changes with the goal of avoiding too many rebuilds, so that this can target master and easily be tested with nixpkgs-review.

Everything that fails to build is left out and will come at the end with more specific fixes.

@philiptaron

Things done

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
    • sandbox = relaxed
    • sandbox = true
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 25.05 Release Notes (or backporting 24.11 and 25.05 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

Add a 👍 reaction to pull requests you find important.

@github-actions github-actions bot added 6.topic: qt/kde Object-oriented framework for GUI creation 6.topic: emacs Text editor 6.topic: games Gaming on NixOS labels Dec 15, 2024
@wolfgangwalther
Copy link
Contributor Author

nixpkgs-review result

Generated using nixpkgs-review.

Command: nixpkgs-review


x86_64-linux

⏩ 4 packages marked as broken and skipped:
  • postgresql13JitPackages.pgvecto-rs
  • postgresql13Packages.pgvecto-rs
  • postgresql17JitPackages.pgvecto-rs
  • postgresql17Packages.pgvecto-rs
❌ 5 packages failed to build:
  • obs-studio-plugins.obs-transition-table
  • python311Packages.heudiconv
  • python311Packages.heudiconv.dist
  • python312Packages.heudiconv
  • python312Packages.heudiconv.dist
✅ 189 packages built:
  • activitywatch
  • ansible-cmdb
  • ansible-cmdb.dist
  • apptainer
  • apptainer-overriden-nixos
  • authenticator
  • auto-cpufreq
  • auto-cpufreq.dist
  • aw-server-rust
  • bashblog
  • bcache-tools
  • bonnmotion
  • bottles
  • bottles-unwrapped
  • budgie-control-center
  • budgie-control-center.debug
  • budgie-session
  • budgie-session.debug
  • budgie-session.man
  • calyx-vpn
  • cargo-benchcmp
  • chawan
  • cheese
  • cheese.devdoc
  • cheese.man
  • cjdns
  • cosmocc
  • cosmopolitan
  • cosmopolitan.dist
  • coz
  • curseradio
  • curseradio.dist
  • cytoscape
  • davfs2
  • dcm2niix
  • deja-dup
  • dmensamenu
  • dmensamenu.dist
  • ebpf-verifier
  • edgedb
  • emacsPackages.lsp-bridge
  • evolution-ews
  • evolutionWithPlugins
  • febio
  • febio-studio
  • flottbot
  • fprintd
  • fprintd-tod
  • fprintd-tod.devdoc
  • fprintd.devdoc
  • fractal
  • fxlinuxprintutil
  • gammu
  • glycin-loaders
  • gnome-decoder
  • gnome-music
  • govulncheck
  • grilo-plugins
  • heroic
  • home-assistant-component-tests.sms
  • hplipWithPlugin
  • hypnotix
  • ibus-engines.hangul
  • inochi-creator
  • kazam
  • kazam.dist
  • keybase
  • leocad
  • libelfin
  • libpam-wrapper
  • librum
  • libsForQt5.kdenlive
  • libvpl
  • likwid
  • loupe
  • luaformatter
  • mangohud
  • mangohud.doc
  • mangohud.man
  • mangojuice
  • mysql-workbench
  • nekoray
  • networkmanager-fortisslvpn
  • networkmanager-iodine
  • networkmanager-l2tp
  • networkmanager-openconnect
  • networkmanager-openvpn
  • networkmanager-vpnc
  • nifskope
  • nitter
  • novnc
  • obs-studio
  • obs-studio-plugins.advanced-scene-switcher
  • obs-studio-plugins.droidcam-obs
  • obs-studio-plugins.input-overlay
  • obs-studio-plugins.looking-glass-obs
  • obs-studio-plugins.obs-3d-effect
  • obs-studio-plugins.obs-backgroundremoval
  • obs-studio-plugins.obs-command-source
  • obs-studio-plugins.obs-composite-blur
  • obs-studio-plugins.obs-freeze-filter
  • obs-studio-plugins.obs-gradient-source
  • obs-studio-plugins.obs-gstreamer
  • obs-studio-plugins.obs-hyperion
  • obs-studio-plugins.obs-livesplit-one
  • obs-studio-plugins.obs-move-transition
  • obs-studio-plugins.obs-multi-rtmp
  • obs-studio-plugins.obs-mute-filter
  • obs-studio-plugins.obs-ndi
  • obs-studio-plugins.obs-nvfbc
  • obs-studio-plugins.obs-pipewire-audio-capture
  • obs-studio-plugins.obs-replay-source
  • obs-studio-plugins.obs-rgb-levels-filter
  • obs-studio-plugins.obs-scale-to-sound
  • obs-studio-plugins.obs-shaderfilter
  • obs-studio-plugins.obs-source-clone
  • obs-studio-plugins.obs-source-record
  • obs-studio-plugins.obs-source-switcher
  • obs-studio-plugins.obs-teleport
  • obs-studio-plugins.obs-text-pthread
  • obs-studio-plugins.obs-tuna
  • obs-studio-plugins.obs-vaapi
  • obs-studio-plugins.obs-vertical-canvas
  • obs-studio-plugins.obs-vintage-filter
  • obs-studio-plugins.obs-vkcapture
  • obs-studio-plugins.obs-webkitgtk
  • obs-studio-plugins.obs-websocket
  • obs-studio-plugins.waveform
  • obs-studio-plugins.wlrobs
  • obs-studio.debug
  • onionshare
  • onionshare-gui
  • onionshare-gui.dist
  • onionshare.dist
  • org-stats
  • passage
  • pika-backup
  • postgresql14JitPackages.pgvecto-rs
  • postgresql14Packages.pgvecto-rs
  • postgresql15JitPackages.pgvecto-rs
  • postgresql15Packages.pgvecto-rs
  • postgresql16JitPackages.pgvecto-rs
  • postgresql16Packages.pgvecto-rs
  • protontricks
  • protontricks.dist
  • python-cosmopolitan
  • python311Packages.caffe
  • python311Packages.caffe.bin
  • python311Packages.pypamtest
  • python311Packages.python-gammu
  • python311Packages.python-gammu.dist
  • python312Packages.pypamtest
  • python312Packages.python-gammu
  • python312Packages.python-gammu.dist
  • qgis
  • qgis-ltr
  • qvge
  • r128gain
  • r128gain.dist
  • riseup-vpn
  • rtfm
  • scanservjs
  • shotcut
  • sing-geosite
  • siyuan
  • snapshot
  • static-server
  • stumpish
  • teavpn2
  • tmuxPlugins.fingers
  • tmuxPlugins.tmux-thumbs
  • totem
  • typescript-language-server
  • vdhcoapp
  • veracrypt
  • vym
  • vym.man
  • warsow
  • warsow-engine
  • webcord-vencord
  • weechatScripts.multiline
  • weechatScripts.wee-slack
  • wingpanel-indicator-ayatana
  • xborders
  • xborders.dist
  • yabridge
  • you-get
  • you-get.dist
  • zsh-powerlevel10k

The failing packages are failing on master, too.

@github-actions github-actions bot added 10.rebuild-darwin: 11-100 This PR causes between 11 and 100 packages to rebuild on Darwin. 10.rebuild-linux: 101-500 This PR causes between 101 and 500 packages to rebuild on Linux. labels Dec 15, 2024
Copy link
Contributor

@philiptaron philiptaron left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The commit message appears to GitHub to be referencing this PR:

And when merged will likely refer to this one:

@philiptaron
Copy link
Contributor

nixpkgs-review result

Generated using nixpkgs-review.

Command: nixpkgs-review pr 365440


x86_64-linux

⏩ 4 packages marked as broken and skipped:
  • postgresql13JitPackages.pgvecto-rs
  • postgresql13Packages.pgvecto-rs
  • postgresql17JitPackages.pgvecto-rs
  • postgresql17Packages.pgvecto-rs
❌ 10 packages failed to build:
  • activitywatch
  • cosmocc
  • cosmopolitan
  • cosmopolitan.dist
  • obs-studio-plugins.obs-transition-table
  • python-cosmopolitan
  • python311Packages.heudiconv
  • python311Packages.heudiconv.dist
  • python312Packages.heudiconv
  • python312Packages.heudiconv.dist
✅ 184 packages built:
  • ansible-cmdb
  • ansible-cmdb.dist
  • apptainer
  • apptainer-overriden-nixos
  • authenticator
  • auto-cpufreq
  • auto-cpufreq.dist
  • aw-server-rust
  • bashblog
  • bcache-tools
  • bonnmotion
  • bottles
  • bottles-unwrapped
  • budgie-control-center
  • budgie-control-center.debug
  • budgie-session
  • budgie-session.debug
  • budgie-session.man
  • calyx-vpn
  • cargo-benchcmp
  • chawan
  • cheese
  • cheese.devdoc
  • cheese.man
  • cjdns
  • coz
  • curseradio
  • curseradio.dist
  • cytoscape
  • davfs2
  • dcm2niix
  • deja-dup
  • dmensamenu
  • dmensamenu.dist
  • ebpf-verifier
  • edgedb
  • emacsPackages.lsp-bridge
  • evolution-ews
  • evolutionWithPlugins
  • febio
  • febio-studio
  • flottbot
  • fprintd
  • fprintd-tod
  • fprintd-tod.devdoc
  • fprintd.devdoc
  • fractal
  • fxlinuxprintutil
  • gammu
  • glycin-loaders
  • gnome-decoder
  • gnome-music
  • govulncheck
  • grilo-plugins
  • heroic
  • home-assistant-component-tests.sms
  • hplipWithPlugin
  • hypnotix
  • ibus-engines.hangul
  • inochi-creator
  • kazam
  • kazam.dist
  • keybase
  • leocad
  • libelfin
  • libpam-wrapper
  • librum
  • libsForQt5.kdenlive
  • libvpl
  • likwid
  • loupe
  • luaformatter
  • mangohud
  • mangohud.doc
  • mangohud.man
  • mangojuice
  • mysql-workbench
  • nekoray
  • networkmanager-fortisslvpn
  • networkmanager-iodine
  • networkmanager-l2tp
  • networkmanager-openconnect
  • networkmanager-openvpn
  • networkmanager-vpnc
  • nifskope
  • nitter
  • novnc
  • obs-studio
  • obs-studio-plugins.advanced-scene-switcher
  • obs-studio-plugins.droidcam-obs
  • obs-studio-plugins.input-overlay
  • obs-studio-plugins.looking-glass-obs
  • obs-studio-plugins.obs-3d-effect
  • obs-studio-plugins.obs-backgroundremoval
  • obs-studio-plugins.obs-command-source
  • obs-studio-plugins.obs-composite-blur
  • obs-studio-plugins.obs-freeze-filter
  • obs-studio-plugins.obs-gradient-source
  • obs-studio-plugins.obs-gstreamer
  • obs-studio-plugins.obs-hyperion
  • obs-studio-plugins.obs-livesplit-one
  • obs-studio-plugins.obs-move-transition
  • obs-studio-plugins.obs-multi-rtmp
  • obs-studio-plugins.obs-mute-filter
  • obs-studio-plugins.obs-ndi
  • obs-studio-plugins.obs-nvfbc
  • obs-studio-plugins.obs-pipewire-audio-capture
  • obs-studio-plugins.obs-replay-source
  • obs-studio-plugins.obs-rgb-levels-filter
  • obs-studio-plugins.obs-scale-to-sound
  • obs-studio-plugins.obs-shaderfilter
  • obs-studio-plugins.obs-source-clone
  • obs-studio-plugins.obs-source-record
  • obs-studio-plugins.obs-source-switcher
  • obs-studio-plugins.obs-teleport
  • obs-studio-plugins.obs-text-pthread
  • obs-studio-plugins.obs-tuna
  • obs-studio-plugins.obs-vaapi
  • obs-studio-plugins.obs-vertical-canvas
  • obs-studio-plugins.obs-vintage-filter
  • obs-studio-plugins.obs-vkcapture
  • obs-studio-plugins.obs-webkitgtk
  • obs-studio-plugins.obs-websocket
  • obs-studio-plugins.waveform
  • obs-studio-plugins.wlrobs
  • obs-studio.debug
  • onionshare
  • onionshare-gui
  • onionshare-gui.dist
  • onionshare.dist
  • org-stats
  • passage
  • pika-backup
  • postgresql14JitPackages.pgvecto-rs
  • postgresql14Packages.pgvecto-rs
  • postgresql15JitPackages.pgvecto-rs
  • postgresql15Packages.pgvecto-rs
  • postgresql16JitPackages.pgvecto-rs
  • postgresql16Packages.pgvecto-rs
  • protontricks
  • protontricks.dist
  • python311Packages.caffe
  • python311Packages.caffe.bin
  • python311Packages.pypamtest
  • python311Packages.python-gammu
  • python311Packages.python-gammu.dist
  • python312Packages.pypamtest
  • python312Packages.python-gammu
  • python312Packages.python-gammu.dist
  • qgis
  • qgis-ltr
  • qvge
  • r128gain
  • r128gain.dist
  • riseup-vpn
  • rtfm
  • scanservjs
  • shotcut
  • sing-geosite
  • siyuan
  • snapshot
  • static-server
  • stumpish
  • teavpn2
  • tmuxPlugins.fingers
  • tmuxPlugins.tmux-thumbs
  • totem
  • typescript-language-server
  • vdhcoapp
  • veracrypt
  • vym
  • vym.man
  • warsow
  • warsow-engine
  • webcord-vencord
  • weechatScripts.multiline
  • weechatScripts.wee-slack
  • wingpanel-indicator-ayatana
  • xborders
  • xborders.dist
  • yabridge
  • you-get
  • you-get.dist
  • zsh-powerlevel10k

@jian-lin jian-lin removed their request for review December 16, 2024 02:53
@philiptaron philiptaron merged commit 66cdf59 into NixOS:master Dec 16, 2024
@wolfgangwalther wolfgangwalther deleted the refactor-tractor-1 branch December 16, 2024 07:41
@wolfgangwalther
Copy link
Contributor Author

wolfgangwalther commented Dec 19, 2024

@jwiegley posted a comment directly on the merged commit. For visibility, I reproduce it here:

Hi, I'm not sure why this didn't show up in the automated testing, but this change cannot work because there is no gcc variable to be replaced in setup.sh. It seems that replaceVars is more strict than substituteAll in this regard.

This relates to pkgs/misc/my-env/default.nix.

Looking at the nixpkgs-review list, this package was indeed not built. Fix in #366611.

@wolfgangwalther
Copy link
Contributor Author

I went through the builds that failed for other reasons during nixpkgs-review and those that didn't trigger a build at all, e.g. because of conditionals. Confirmed manually that all replacements are available - no other problems found.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

6.topic: emacs Text editor 6.topic: games Gaming on NixOS 6.topic: qt/kde Object-oriented framework for GUI creation 10.rebuild-darwin: 11-100 This PR causes between 11 and 100 packages to rebuild on Darwin. 10.rebuild-linux: 101-500 This PR causes between 101 and 500 packages to rebuild on Linux.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants