Context

With sponsoring from Antithesis ✨, following the lib.path library, I've been able to work on lib.fileset in recent months, a new library in Nixpkgs to select local files to use for sources. Originally I opened a WIP PR as a proof-of-concept and to get feedback on the general design. Later I also created a Discourse post to gather more feedback. After not getting a lot of feedback though (but also nobody complaining), we (me, @roberth and @fricklerhandwerk) were content enough with its general design and therefore agreed to accept my PRs to incrementally build up the library in Nixpkgs.

So over the last months I made a bunch of PRs, and had them reviewed and merged by @roberth and @fricklerhandwerk. Today I'm happy to say that the library is very usable, check out its reference docs! It's not perfect, but the basic feature set is there and is tested thoroughly.

Originally I used the WIP PR to summarise the on-going work, but it would be better to have a dedicated tracking issue for this instead, so I'm creating this issue for that. Furthermore I'm going to use this issue to collect feature requests for the file set library.

Finished, ongoing and proposed interface changes

• File set combinators base: lib.fileset.toSource #245623
• lib.fileset.union, lib.fileset.unions: init #255025
• lib.fileset.trace, lib.fileset.traceVal: init #256417
• lib.fileset.intersection: init #257356
• lib.fileset.difference: init #259065
• lib.fileset.fileFilter: init #260265
• lib.fileset.fromSource: init #261732
• lib.fileset: Lazier path existence check #267091
• fileset.fileFilter: Restrict second argument to paths #267384
• lib.fileset.gitTracked: init #264891
• lib.fileset.maybeMissing: init #265964
• lib.fileset.fileFilter: Predicate attribute for file extension #266362
• lib.fileset.gitTracked should support out-of-tree builds #269283
• Turn lib.fileset into a list of file paths #288592
• lib.fileset: Implicitly coerce lists to unions #267387
  Status: Unsure
• Add lib.fileset.optional :: Bool -> FileSet -> FileSet #266106
  Status: Unsure
• lib.fileset.empty feature request #266366
  Status: Unsure
• lib.fileset.trace should show whether directories include files by default #268878
  Status: Unsure
• lib.fileset.fileFilter should allow filtering by file subpath/components #269504
  Status: Unsure
• lib.fileset needs something like lib.cleanSource #269517
  Status: Unsure
• lib.fileset should have a way to filter directories #271307
  Status: Unsure

Indirectly related:

• Function for transforming store path contents #264541
  Definitely out of scope for file set library
  Status: Unsure
• Add a way to control inclusion of empty directories in builtins.path results nix#8820
  This is a proposed addition to the Nix builtins to improve file set library performance.
  Status: Unsure
• builtins.fetchGit dirty mode almost unusable in pure evaluation mode nix#9292

If you're trying to use the library but run into problems related to any of the above issues, please leave a comment there or 👍 it, especially if its status is unsure. And if you have a request for a new feature, please comment it here, ideally with an issue/PR!