Skip to content

treewide: change ${pname} to string literal#336172

Merged
philiptaron merged 105 commits intoNixOS:masterfrom
pbsds:migrate-pname-1724101375
Aug 20, 2024
Merged

treewide: change ${pname} to string literal#336172
philiptaron merged 105 commits intoNixOS:masterfrom
pbsds:migrate-pname-1724101375

Conversation

@pbsds
Copy link
Member

@pbsds pbsds commented Aug 20, 2024

Description of changes

please squash merge this, i spam-commit for easier handling of merge conflicts.

done with

#!/usr/bin/env nix-shell
#!nix-shell --pure -i bash -p ripgrep sd jq git lix

export NIXPKGS_ALLOW_UNFREE=1
export NIXPKGS_ALLOW_INSECURE=1
export NIXPKGS_ALLOW_BROKEN=1

git restore .

test -s packages.json || ( set -x;
  time nix-env --extra-experimental-features no-url-literals --option system x86_64-linux -f ./. -qaP --json --meta --show-trace --no-allow-import-from-derivation > packages.json
)

jq <packages.json 'to_entries[] | select(.value.meta.position==null|not) | "\(.key)\t\(.value.meta.position)"' -r |
    sed -e "s#\t$(realpath .)/#\t#" |
    sed -e 's#:\([0-9]*\)$#\t\1#' |
    grep --fixed-strings "$( rg -F '${pname}' pkgs/ -tnix -l | sort | head -n 2500 )" |
    grep . |
    grep -iv haskell |
    grep -iv /top-level/ |
    grep -iv chicken |
    grep pkgs/by-name/ |
    grep -iv build |
    grep -E '/(package|default)\.nix' |
    while read attrpath fname col; do
        echo "$attrpath"
        data="$(nix eval --impure  --expr 'with import ./. {}; { inherit ('"$attrpath"') pname drvPath passthru meta; drvPath2='"$attrpath"'.src.drvPath; }' --json)"
        test -n "$data" || continue
        pname="$(jq <<<"$data" .pname -r)"
        test -n "$pname" || continue
        (set -x
            sd -F '${pname}' "$pname" "$fname"
            sd -F ' = pname;' " = \"$pname\";" "$fname"
        )
        data2="$(nix eval --impure  --expr 'with import ./. {}; { inherit ('"$attrpath"') pname drvPath passthru meta; drvPath2='"$attrpath"'.src.drvPath; }' --json)"
        if ! test "$data" = "$data2"; then
            (set -x; git restore "$fname")
        fi
    done

in pkgs/by-name this time around, and i selected the ones that made sense using GIT_DIFF_OPTS=-u2000 git add --patch.

I avoided various patterns like name = "${pname}-${version}" because that would hamper a different kind of treewide, and other patterns where ${pname} is being used in passthru.tests or fod storepath names, and in assertions that makes sense to copy to dervations.

should be 0 rebuilds

Things done

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
    • sandbox = relaxed
    • sandbox = true
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

Add a 👍 reaction to pull requests you find important.

pbsds added 30 commits August 20, 2024 23:38
@pbsds pbsds changed the title Migrate pname 1724101375 treewide: change ${pname} to string literal Aug 20, 2024
@ofborg ofborg bot added 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux. labels Aug 20, 2024
Copy link
Contributor

@philiptaron philiptaron left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I intend to merge once OfBorg comes back clean.


src = fetchzip {
url = "http://s-tech.elsat.net.pl/${pname}/${pname}-${version}.tar.gz";
url = "http://s-tech.elsat.net.pl/braa/braa-${version}.tar.gz";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lol

Comment on lines +62 to 63
exec = "vvvvvv";
icon = "VVVVVV";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

golly

@philiptaron philiptaron merged commit fcdecc2 into NixOS:master Aug 20, 2024
@pbsds pbsds mentioned this pull request Mar 6, 2025
13 tasks
@pbsds pbsds mentioned this pull request Mar 7, 2025
13 tasks
pbsds added a commit to pbsds/nixpkgs that referenced this pull request Mar 11, 2025
Inspired by NixOS#387725 (comment), script is based on NixOS#336172 using what i learned in NixOS#386865, part of NixOS#346453

Should be zero rebuilds.

All candidates were made using:

```shell

export NIXPKGS_ALLOW_UNFREE=1
export NIXPKGS_ALLOW_INSECURE=1
export NIXPKGS_ALLOW_BROKEN=1

git-wait restore .

test -s packages.json || ( set -x;
  time nix-env --extra-experimental-features no-url-literals --option system x86_64-linux -f ./. -qaP --json --meta --drv-path --out-path --show-trace --no-allow-import-from-derivation --arg config '{ allowAliases = false; }' > packages.json
)

list_attrpath_fname_col() {
    jq <packages.json 'to_entries[] | select(.value.meta.position==null|not) | "\(.key)\t\(.value.meta.position)"' -r |
        sed -e "s#\t$(realpath .)/#\t#" |
        sed -e 's#:\([0-9]*\)$#\t\1#' |
        grep . |
        grep -iv haskell |
        grep -iv /top-level/ |
        grep -iv chicken |
        grep pkgs/by-name/ |
        grep -iv build |
        grep -E '/(package|default)\.nix'
}

FLOCKDIR="$(mktemp -d)"
N_WORKERS=4
while read attrpath fname col; do
    grep -qE 'repo *= *("\$\{pname\}"|pname);' "$fname" || continue

    echo | (
        # mutex on fname
        flock --nonblock 200 || {
            >&2 echo "failed to aquire lock for $fname"
            exit 1
        }

        echo "$attrpath"
        data="$(nix eval --impure  --expr 'with import ./. {}; { inherit ('"$attrpath"') pname drvPath passthru meta; drvPath2='"$attrpath"'.src.drvPath; }' --json)" || exit
        test -n "$data" || exit
        pname="$(jq <<<"$data" .pname -r)"
        test -n "$pname" || exit

        (set -x
            sd -F '${pname}'  "$pname"         "$fname"
            sd -F ' = pname;' " = \"$pname\";" "$fname"
        )

        data2="$(nix eval --impure  --expr 'with import ./. {}; { inherit ('"$attrpath"') pname drvPath passthru meta; drvPath2='"$attrpath"'.src.drvPath; }' --json)"
        if [[ "$data" = "$data2" ]]; then
            (set -x; git-wait add "$fname")
        else
            (set -x; git-wait restore "$fname")
            exit
        fi

        (set -x
            sd -F ' rec {' ' {' "$fname"
        )

        data3="$(nix eval --impure  --expr 'with import ./. {}; { inherit ('"$attrpath"') pname drvPath passthru meta; drvPath2='"$attrpath"'.src.drvPath; }' --json 2>/dev/nul)"

        if [[ "$data" = "$data3" ]]; then
            (set -x; git-wait add "$fname")
        else
            (set -x; git-wait restore "$fname")
        fi

    ) 200>"$FLOCKDIR"/"$(sha256sum - <<<"$fname" | cut -d' ' -f1)".lock &

    while [[ $(jobs -p | wc -l) -ge $N_WORKERS ]]; do
        wait -n < <(jobs -p) || true
    done

done < <(list_attrpath_fname_col)

wait

git restore .

time nix-env --extra-experimental-features no-url-literals --option system x86_64-linux -f ./. -qaP --json --meta --drv-path --out-path --show-trace --no-allow-import-from-derivation --arg config '{ allowAliases = false; }' > packages2.json
```

`diff packages{,2}.json` is empty, indicating that no package nor src derivation has changed.
I checked and cherry-picked the changes using `GIT_DIFF_OPTS='-u15' git -c interactive.singleKey=true add --patch`
sandptel pushed a commit to sandptel/nixpkgs that referenced this pull request Mar 13, 2025
Inspired by NixOS#387725 (comment), script is based on NixOS#336172 using what i learned in NixOS#386865, part of NixOS#346453

Should be zero rebuilds.

All candidates were made using:

```shell

export NIXPKGS_ALLOW_UNFREE=1
export NIXPKGS_ALLOW_INSECURE=1
export NIXPKGS_ALLOW_BROKEN=1

git-wait restore .

test -s packages.json || ( set -x;
  time nix-env --extra-experimental-features no-url-literals --option system x86_64-linux -f ./. -qaP --json --meta --drv-path --out-path --show-trace --no-allow-import-from-derivation --arg config '{ allowAliases = false; }' > packages.json
)

list_attrpath_fname_col() {
    jq <packages.json 'to_entries[] | select(.value.meta.position==null|not) | "\(.key)\t\(.value.meta.position)"' -r |
        sed -e "s#\t$(realpath .)/#\t#" |
        sed -e 's#:\([0-9]*\)$#\t\1#' |
        grep . |
        grep -iv haskell |
        grep -iv /top-level/ |
        grep -iv chicken |
        grep pkgs/by-name/ |
        grep -iv build |
        grep -E '/(package|default)\.nix'
}

FLOCKDIR="$(mktemp -d)"
N_WORKERS=4
while read attrpath fname col; do
    grep -qE 'repo *= *("\$\{pname\}"|pname);' "$fname" || continue

    echo | (
        # mutex on fname
        flock --nonblock 200 || {
            >&2 echo "failed to aquire lock for $fname"
            exit 1
        }

        echo "$attrpath"
        data="$(nix eval --impure  --expr 'with import ./. {}; { inherit ('"$attrpath"') pname drvPath passthru meta; drvPath2='"$attrpath"'.src.drvPath; }' --json)" || exit
        test -n "$data" || exit
        pname="$(jq <<<"$data" .pname -r)"
        test -n "$pname" || exit

        (set -x
            sd -F '${pname}'  "$pname"         "$fname"
            sd -F ' = pname;' " = \"$pname\";" "$fname"
        )

        data2="$(nix eval --impure  --expr 'with import ./. {}; { inherit ('"$attrpath"') pname drvPath passthru meta; drvPath2='"$attrpath"'.src.drvPath; }' --json)"
        if [[ "$data" = "$data2" ]]; then
            (set -x; git-wait add "$fname")
        else
            (set -x; git-wait restore "$fname")
            exit
        fi

        (set -x
            sd -F ' rec {' ' {' "$fname"
        )

        data3="$(nix eval --impure  --expr 'with import ./. {}; { inherit ('"$attrpath"') pname drvPath passthru meta; drvPath2='"$attrpath"'.src.drvPath; }' --json 2>/dev/nul)"

        if [[ "$data" = "$data3" ]]; then
            (set -x; git-wait add "$fname")
        else
            (set -x; git-wait restore "$fname")
        fi

    ) 200>"$FLOCKDIR"/"$(sha256sum - <<<"$fname" | cut -d' ' -f1)".lock &

    while [[ $(jobs -p | wc -l) -ge $N_WORKERS ]]; do
        wait -n < <(jobs -p) || true
    done

done < <(list_attrpath_fname_col)

wait

git restore .

time nix-env --extra-experimental-features no-url-literals --option system x86_64-linux -f ./. -qaP --json --meta --drv-path --out-path --show-trace --no-allow-import-from-derivation --arg config '{ allowAliases = false; }' > packages2.json
```

`diff packages{,2}.json` is empty, indicating that no package nor src derivation has changed.
I checked and cherry-picked the changes using `GIT_DIFF_OPTS='-u15' git -c interactive.singleKey=true add --patch`
Yarny0 added a commit to Yarny0/nixpkgs that referenced this pull request Jun 22, 2025
@sternenseemann
Copy link
Member

The motivation you have given previously for this sort of change, #387725 (comment) (which also doesn't make sense in all possible cases), doesn't apply for rec set pname etc. at all. Why do this?

@pbsds
Copy link
Member Author

pbsds commented Jul 18, 2025

Most derivations are one drive-by contribution or review nitpick away from migrating to finalAttrs, as such I treat let-in and rec as if they are overrideable.

@pbsds
Copy link
Member Author

pbsds commented Jul 18, 2025

@pbsds
Copy link
Member Author

pbsds commented Jul 18, 2025

Why do this?

One of my motivations for #346453 is to show people that most of the common nitpicks are trivially fixed with scripted treewides, and should as such not cause unnecessary review cycles.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants