Skip to content

Comments

lib.fileset.fileFilter: Predicate attribute for file extension#266362

Merged
infinisil merged 1 commit intoNixOS:masterfrom
tweag:fileset.fileFilter-ext
Nov 23, 2023
Merged

lib.fileset.fileFilter: Predicate attribute for file extension#266362
infinisil merged 1 commit intoNixOS:masterfrom
tweag:fileset.fileFilter-ext

Conversation

@infinisil
Copy link
Member

@infinisil infinisil commented Nov 9, 2023

Description of changes

Makes it more convenient to use lib.fileset.fileFilter to filter by file extension:

lib.fileset.fileFilter (file:
  lib.any file.hasExt [ "nix" "hs" ]
) ./.

Whereas previously this would've needed

lib.fileset.fileFilter (file:
  lib.any (ext: lib.hasSuffix ".${ext}" file.name) [ "nix" "hs" ]
) ./.

I'd imagine this should be fairly popular, but we need to confirm first.

This work is sponsored by Antithesis

Things done

  • Tests
  • Docs

@github-actions github-actions bot added the 6.topic: lib The Nixpkgs function library label Nov 9, 2023
@infinisil infinisil changed the title lib.fileset.fileFilter: Predicate attribute for file extension lib.fileset.fileFilter: Predicate attribute for file extension Nov 9, 2023
@ofborg ofborg bot added 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux. labels Nov 9, 2023
@Kranzes
Copy link
Member

Kranzes commented Nov 11, 2023

Very looking forward for this

@infinisil infinisil force-pushed the fileset.fileFilter-ext branch from 3de1bff to c31cfee Compare November 19, 2023 00:18
@infinisil infinisil marked this pull request as ready for review November 19, 2023 00:18
@infinisil infinisil force-pushed the fileset.fileFilter-ext branch from c31cfee to e0b1528 Compare November 19, 2023 00:35
@infinisil
Copy link
Member Author

Rebased, added tests and made sure the docs are accurate. I'd say let's have this!

@infinisil infinisil force-pushed the fileset.fileFilter-ext branch from e0b1528 to 1af8f12 Compare November 19, 2023 00:39
@roberth
Copy link
Member

roberth commented Nov 20, 2023

lib.fileset.fileFilter (file:
  lib.elem file.ext [ "nix" "hs" ]
) ./.

What about tar.gz?
How about:

lib.fileset.fileFilter (file:
  file.hasExt "hs" || file.hasExt "nix"
) ./.
lib.fileset.fileFilter (file:
  ! file.hasExt "tar.gz"
) ./.

@infinisil
Copy link
Member Author

Not bad, here's another idea though:

lib.fileset.fileFilter (file:
  file.fullExt != "tar.gz"
) ./.

and

lib.fileset.fileFilter (file:
  file.lastExt != "gz"
) ./.

I do think there's some benefit to getting the extension value directly, e.g. you could do

lib.fileset.fileFilter (file:
  if file.ext == null then
    true
  else
    {
      nix = false;
      c = true;
      h = false;
    }.${file.lastExt}
    or (throw "Don't know what to do with file extension ${file.lastExt}")
) ./.

Also there wouldn't be a problem to have multiple of these. Their names make them pretty obvious.

@roberth
Copy link
Member

roberth commented Nov 20, 2023

But then you have to get into what full means. What if you have

foo-master.tar.gz
foo-1.2.tar.gz

Is that a foo-1 man page, archived and compressed?

Determining which part of the file name is extension is up to the caller, not up to fileFilter. It can not have the domain knowledge, so it must not try.

@infinisil
Copy link
Member Author

Fair enough! I guess with hasExt, you can even do

lib.fileset.fileFilter (file:
  lib.any file.hasExt [ "hs" "nix" ]
) ./.

which is pretty short

@infinisil infinisil marked this pull request as draft November 20, 2023 17:03
@ofborg ofborg bot added the 2.status: merge conflict This PR has merge conflicts with the target branch label Nov 20, 2023
@infinisil infinisil force-pushed the fileset.fileFilter-ext branch 3 times, most recently from 43eadfa to 5aaac6d Compare November 21, 2023 03:04
@infinisil infinisil marked this pull request as ready for review November 21, 2023 03:05
@ofborg ofborg bot removed the 2.status: merge conflict This PR has merge conflicts with the target branch label Nov 21, 2023
Copy link
Member

@roberth roberth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If a tree falls, but no-one's arou

I think one corner case should be addressed in the docs, and then we're good.

Copy link
Member

@roberth roberth Nov 23, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does .gitignore have a .gitignore extension, or no extension at all? In ~other words, does the empty basename exist?

Copy link
Member Author

@infinisil infinisil Nov 23, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is covered with the file.hasExt "a" test, yes, gitignore is treated as an extension! Should be added to the docs though :)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess we could ask whether that's the right behavior though, alternatively it could error 🤔

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's fine to leave it as is, I added some docs for that: (diff)

@fricklerhandwerk
Copy link
Contributor

Is it a file system tree?

@infinisil infinisil force-pushed the fileset.fileFilter-ext branch from 5aaac6d to 6816f28 Compare November 23, 2023 20:05
Copy link
Member

@roberth roberth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The right extent of extension support.👌

@roberth
Copy link
Member

roberth commented Nov 23, 2023

OfBorg is sluggish today.

Is it a file system tree?

Nice.

@infinisil
Copy link
Member Author

OfBorg is sluggish today.

Only today? 😅

@github-actions
Copy link
Contributor

Successfully created backport PR for release-23.11:

infinisil added a commit to NixOS/nix.dev that referenced this pull request Dec 1, 2023
fricklerhandwerk pushed a commit to NixOS/nix.dev that referenced this pull request Dec 1, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

6.topic: lib The Nixpkgs function library 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants