fetchFromSavannah: deprecate · Pull Request #435964 · NixOS/nixpkgs

ghost · 2025-08-22T20:04:01Z

Context

fetchFromSavannah is a fetcher which interpolates its arguments into a URL to be used by fetchzip to download a CGit snapshot from GNU Savannah's cgit instance.

Most Savannah and GNU software is available through archive mirrors which are enumerated in nixpkgs via "mirror://" and is used for the majority of GNU/Savannah packages in nixpkgs and in other distributions.

I also updated the link for the savannah fetcher in #427024

The Issue

Doing a naive grep of the nixpkgs source checkout, fetchFromSavannah is only used in a total of 5 packages. With single digit usage and no discernible tests, keeping fetchFromSavannah becomes a maintenance burden when it can easily be removed.

Also, CGit offers snapshot support, but it is optional and is not a API interface. Considering that GNU expects distributions to download from their release mirrors, we can't rely on CGit in the same way one can rely on Gitea/GitLab/GitHub.

The Solution

Replace each usage of fetchFromSavannah with either a fetchurl mirror or a fetchgit call and adjust as needed.

Packages changed

Tracking PRs:

Deprecation notice

I decided not to remove fetchFromSavannah completely but to instead deprecate via lib.warn and remove its entry from fetchers documentation to account for downstream use since fetchFromSavannah was in nixpkgs for a long time, it should be removed in the subsequent release of nixpkgs though, downstreams that use the savannah fetcher can just re-implement the string interpolating function downstream.

So far I know emacs-overlay does use the savannah fetcher and nix-update has a savannah update codepath (that haphazardly works)

{
  lib,
  repoRevToNameMaybe,
  fetchzip,
}:

lib.makeOverridable (
  # cgit example, snapshot support is optional in cgit
  {
    repo,
    rev,
    name ? repoRevToNameMaybe repo rev "savannah",
    ... # For hash agility
  }@args:
  fetchzip (
    {
      inherit name;
      url =
        let
          repo' = lib.last (lib.strings.splitString "/" repo); # support repo like emacs/elpa
        in
        "https://cgit.git.savannah.gnu.org/cgit/${repo}.git/snapshot/${repo'}-${rev}.tar.gz";
      meta.homepage = "https://cgit.git.savannah.gnu.org/cgit/${repo}.git/";
      passthru.gitRepoUrl = "https://cgit.git.savannah.gnu.org/git/${repo}.git";
    }
    // removeAttrs args [
      "repo"
      "rev"
    ]
  )
  // {
    inherit rev;
  }
)

Things done

Add a 👍 reaction to pull requests you find important.

jian-lin · 2025-08-22T20:31:52Z

pkgs/applications/editors/emacs/make-emacs.nix

fetchfromsavannah currently is not widely used in Nixpkgs. However, as I mentioned before, we, Nixpkgs Emacs ecosystem maintainers, plan to use fetchfromsavannah to re-write (non-)GNU (devel-)ELPA package generation for reproducibility. fetchfromsavannah will be used by about 1500 elisp packages after the re-write. It will be used to fetch tarballs of arbitrary commit of elisp packages in https://cgit.git.savannah.gnu.org/cgit/emacs/nongnu.git/ and https://git.savannah.gnu.org/cgit/emacs/elpa.git/ .

Given this potential use case of fetchfromsavannah, do you still think removing it is a good idea? If so, what is the alternative of fetchfromsavannah?

I think the Emacs team then can have their own internal fetchFromSavannah equivalent that they use for elisp package generation as you've said here.

I'm not entirely against using Savannah's cgit snapshot feature in this way at all (outside of general savannah stability concerns). I think removing fetchFromSavannah usage tree-wide and then re-implementing for this purpose would make the Emacs team's work easier since they don't have to worry about breaking the interface for other package consumers (both in nixpkgs and out of it) anymore and can focus on Emacs' needs.

In short, I think this PR would actually aid in this endeavor since its not a nixpkgs wide fetcher anymore that has to stay backwards compatible.

I agree with @normalcea about vendoring a specific variant for emacs use.

I think the Emacs team then can have their own internal fetchFromSavannah equivalent that they use for elisp package generation

I think we all agree that fetchFromSavannah should only be used when necessary. What we do not agree is how to achieve this goal. Instead of removing fetchFromSavannah and then re-introducing it somewhere else, I propose keeping fetchFromSavannah and modifying its doc to tell users to only use it when necessary. Sharing is good and vendoring is bad.

philiptaron

I'd love to merge most of the changes in this PR in their own precursor PRs. Let's do that then focus on the deprecation/removal solely in this PR.

pkgs/by-name/gn/gnucap/modelgen-verilog.nix

pkgs/development/interpreters/ruby/default.nix

pkgs/by-name/me/mescc-tools/package.nix

philiptaron · 2025-08-22T21:48:19Z

pkgs/applications/editors/emacs/make-emacs.nix

I agree with @normalcea about vendoring a specific variant for emacs use.

pkgs/applications/editors/emacs/make-emacs.nix

pkgs/tools/misc/grub/default.nix

pkgs/top-level/all-packages.nix

jian-lin · 2025-10-01T06:18:41Z

Re @normalcea

"similar" from the doc is quite open to interpretation. As I said above, we need to improve the doc.

BTW, I do not think "similar" means other fetchers should also support fallback from fetchzip to fetchgit.

For fetchers that do not have the fetchgit fallback now, are there real use cases needing fetchgit fallback?

Re renaming fetchFromSavannah to fetchSavannahSnapshot

"snapshot" does not add more meaning, I think, because fetchFrom* fetchers almost all fetch snapshots.

In addition, if we do that rename, what should be the new implementation of fetchFromSavannah?

Re @philiptaron's first thought

I am not sure if you have read https://www.mail-archive.com/bug-gnulib%40gnu.org/msg48636.html or not. My conclusion after reading that is savannah only disables cgit snapshots when savannah is under DDoS attack or for some specific repos. Cgit snapshot is an interface and fetchFromSavannah is reliable. In fact, as a heavy user of fetchFromSavannah, emacs-overlay only had issues with fetchFromSavannah when savannah was under DDoS attack.

In addition, fetchFromSavannah is not replaceable in some cases. See #435964 (comment)

I do not quite understand @philiptaron's second thought since I am not a native English speaker. So I won't comment on that.

Conclusion: extending doc of fetchFromSavannah (maybe like this) is enough

Only use fetchFromSavannah when you really want to fetch snapshots. If possible, prefer fetchzip/fetchurl or fetchgit.

ghost · 2025-10-01T15:08:34Z

@jian-lin

For fetchers that do not have the fetchgit fallback now, are there real use cases needing fetchgit fallback?

When needing to pass args such as fetchSubmodules and deepClone and still keeping the same fetcher (which handles the URL logic that doesn't need to be repeated in consuming derivation src builders). This is done with fetchFromGitHub

"snapshot" does not add more meaning, I think, because fetchFrom* fetchers almost all fetch snapshots.

Snapshot can add meaning by setting that this is fetching a snapshot offered by Savannah rather than being the best effort general-purpose fetcher from savannah which is competing with fetchgit and mirrors.

In addition, if we do that rename, what should be the new implementation of fetchFromSavannah?

There is no new implementation, this is still a deprecation in the sense that it is a symbolic deprecation. Savannah's cgit instance and their canonical savannah urls are not related by any sort of pattern we can take advantage of so it's difficult to extend fetchFromSavannah in a confident way. But it's not a vendoring of the functionality offered by said fetcher to be internally used in nixpkgs and not shared.

Only use fetchFromSavannah when you really want to fetch snapshots. If possible, prefer fetchzip/fetchurl or fetchgit.

I don't think going down this route would be helpful in the long-term and it's also not the approach I want to take with fetcher documentation. For example "how" does one know when to prefer mirrors or fetchgit and "what" is a snapshot referring to (since it still leaves out the fact that these are served by their cgit instance).

However, I agree that these sorts of recommendations about when to use certain fetchers are also needed.

Re: @philiptaron

I also am in the same boat as @jian-lin in your second point (I'm a native english speaker so don't feel bad jian-lin for not understanding), you did say it was a collection of conflicting thoughts. I'm interested to hear your opinion specifically on the effects of renaming the savannah fetcher (allowing it to be shared with out-of-tree) while also implicitly discouraging its use as a general purpose fetcher via a rename.

At the very least I strongly agree that the documentation for the fetchers has room for improvement. I personally prefer if the fetchers were more verbose and technical so that one does not ever have the need to read the implementation to understand how to use the fetcher.

But I also don't want to go down the route of fetchFromSourcehut which while being technical also feels too conversational as documentation.

(doc/build-helpers/fetchers.chapter.md)

## `fetchFromSourcehut` {#fetchfromsourcehut}

This is used with sourcehut repositories. Similar to `fetchFromGitHub` above,
it expects `owner`, `repo`, `rev` and `hash`, but don't forget the tilde (~)
in front of the username! Expected arguments also include `vc` ("git" (default)
or "hg"), `domain` and `fetchSubmodules`.

If `fetchSubmodules` is `true`, `fetchFromSourcehut` uses `fetchgit`
or `fetchhg` with `fetchSubmodules` or `fetchSubrepos` set to `true`,
respectively. Otherwise, the fetcher uses `fetchzip`.

@jian-lin

@jian-lin's ❌ vetoing. I think we don't deprecate.

ghost · 2025-10-11T00:55:00Z

My mind has also changed, echoing what @philiptaron mentioned, all in-tree uses of fetchFromSavannah on unique (as in, not tied to any autogeneration) packages have been replaced. It doesn't seem fair to me anymore to insist on deprecation over semantics.

ghost · 2026-01-08T21:13:10Z

From #477666 it stands to reason that the Savannah admins have disabled cgit snapshotting.

Guix¹ for example, shows the "Download' section in the tags header but clicking on it leads you to the revision status rather than a download link. Grub² is the same way, Emacs³ too.

Re-opening this PR, glad I didn't delete the patches. But now the question is that a nixpkgs fetcher is in a non-working state, so this deprecation will have to be backported to 25.11 from the looks of it. This makes me think instead of outright removal, a deprecation is more warranted in the off-chance that the Savannah administrators decide to re-enable the feature or that there exists a savannah package that still has snapshots or snapshots are re-enabled for some packages but not all.

I can't find an official announcement of this change, finding a response from the Savannah admins would be helpful.

CC: @jian-lin @philiptaron

jian-lin · 2026-01-08T21:37:28Z

As I said, I agree with the deprecation or removal now since the situation has changed.

this deprecation will have to be backported to 25.11

I do not think it is a "have to". Quite the opposite, the deprecation does not seem to be a qualified backport according to https://github.com/NixOS/nixpkgs/blob/master/CONTRIBUTING.md#changes-acceptable-for-releases.

in the off-chance that the Savannah administrators decide to re-enable the feature or that there exists a savannah package that still has snapshots or snapshots are re-enabled for some packages but not all

It would be good to ask Savannah admins about the current state and future plan of cgit snapshot. Would you like to do it?

ghost · 2026-01-09T01:26:22Z

I found this thread¹ which was made relatively recently (dated December of last year), here's the relevant snippet I pulled out:

Follow-up Comment 1, sr 111357 (group administration):

Also, for another reason, please do not download tarballs from cgit snapshots,
it puts a lot of pressure on the server especially when the git repo is large.
You should modify your build recipes to clone the git repo and do a
"git-archive" to export a tarball.

when you change to the "git clone, git archive" flow, you should fix your
scripts to use our latest and fastest distributed mirror system, i.e.,
(git protocol clone url, fast) git://git.git.savannah.gnu.org
(https clone url) https://https.git.savannah.gnu.org/

It seems like the admin consensus is that downstream packagers really shouldn't be relying on snapshots. @jian-lin if you would like to email them for more details or reply to this email thread you can do it on behalf of nixpkgs but I think they made their position clear enough already.

I do not think it is a "have to". Quite the opposite, the deprecation does not seem to be a qualified backport according to https://github.com/NixOS/nixpkgs/blob/master/CONTRIBUTING.md#changes-acceptable-for-releases.

There's no word for word on what to do for fetchers, but my intention is for the deprecation to act as a "warning sign" to downstream consumers in stable since the functionality of the fetcher no longer is guaranteed. If we were removing the codepath entirely then I would agree that it wouldn't be appropriate to backport since it would crash downstream evaluations but a deprecation shouldn't do that. I'm all ears for suggestions on how else to do this properly.

https://lists.gnu.org/archive/html/savannah-hackers/2025-12/msg00014.html ↩

philiptaron

I don't think we should backport but I do think we should merge given the message above.

jian-lin

Generally LGTM

I think the commits organization can be improve. Currently the removal of manual section and the changes of redirection are in separated commits, which is not atomic. IMHO, just 1 commit is enough.

doc/release-notes/rl-2605.section.md

doc/redirects.json

ghost · 2026-01-14T22:03:32Z

Re: @jian-lin

Squashed the doc removal and redirection into one commit, they are related changes so they should be together.

outdated

jian-lin

LGTM, thanks!

One small thing is that GitHub only knows Co-authored-by: instead of Co-developed-by:.

In addition, I do not quite get why you separate the code change and its corresponding doc change into two commits. However, this does not affect my approval since @philiptaron has approved already.

doc/redirects.json

ghost · 2026-01-15T01:47:12Z

Re: @jian-lin

The choice of separation was probably when this PR was a draft and I split the changes into each commit, I squashed all the changes and fixed the "authored by" lines, also adding you as co-author if you want that.

The Savannah administrators do not want package maintainers to use cgit snapshots due to putting strain on the servers and have disabled cgit snapshots for most if not all of their repositories. See: https://lists.gnu.org/archive/html/savannah-hackers/2025-12/msg00014.html Co-authored-by: Philip Taron <[email protected]> Co-authored-by: LIN, Jian <[email protected]>

ghost marked this pull request as draft August 22, 2025 20:04

nix-owners bot requested review from AndersonTorres, adisbladis, jian-lin, matthewbauer, panchoh and ttuegel August 22, 2025 20:11

ghost force-pushed the deprecate-savannah-fetcher branch from d5a45bc to f8d6258 Compare August 22, 2025 20:18

ghost changed the base branch from master to staging August 22, 2025 20:18

nixpkgs-ci bot closed this Aug 22, 2025

nixpkgs-ci bot reopened this Aug 22, 2025

ghost changed the base branch from staging to master August 22, 2025 20:20

nixpkgs-ci bot closed this Aug 22, 2025

nixpkgs-ci bot reopened this Aug 22, 2025

jian-lin reviewed Aug 22, 2025

View reviewed changes

ghost marked this pull request as ready for review August 22, 2025 20:42

nix-owners bot requested review from GetPsyched, fricklerhandwerk and hsjobeki August 22, 2025 20:44

jian-lin self-requested a review August 22, 2025 21:06

ghost requested a review from philiptaron August 22, 2025 21:33

philiptaron requested changes Aug 22, 2025

View reviewed changes

nixpkgs-ci bot removed the 12.approvals: 1 This PR was reviewed and approved by one person. label Oct 3, 2025

ghost closed this Oct 11, 2025

jian-lin mentioned this pull request Dec 11, 2025

Fix Savannah snapshot downloads by switching from cgit to gitweb nix-community/emacs-overlay#523

Closed

ghost reopened this Jan 8, 2026

ghost force-pushed the deprecate-savannah-fetcher branch from 619431c to 4e4f235 Compare January 8, 2026 21:22

ghost changed the title ~~fetchFromSavannah: deprecate and replace usage~~ fetchFromSavannah: deprecate Jan 8, 2026

philiptaron approved these changes Jan 12, 2026

View reviewed changes

nixpkgs-ci bot added the 12.approvals: 1 This PR was reviewed and approved by one person. label Jan 12, 2026

jian-lin previously requested changes Jan 13, 2026

View reviewed changes

doc/release-notes/rl-2605.section.md Outdated Show resolved Hide resolved

doc/redirects.json Show resolved Hide resolved

ghost force-pushed the deprecate-savannah-fetcher branch from 4e4f235 to cbde561 Compare January 14, 2026 22:00

jian-lin approved these changes Jan 15, 2026

View reviewed changes

doc/redirects.json Show resolved Hide resolved

nixpkgs-ci bot added 12.approvals: 2 This PR was reviewed and approved by two persons. and removed 12.approvals: 1 This PR was reviewed and approved by one person. labels Jan 15, 2026

ghost force-pushed the deprecate-savannah-fetcher branch from cbde561 to 5d85c19 Compare January 15, 2026 01:45

ghost force-pushed the deprecate-savannah-fetcher branch from 5d85c19 to 03b2128 Compare January 15, 2026 01:48

jian-lin added this pull request to the merge queue Jan 15, 2026

Merged via the queue into NixOS:master with commit d786739 Jan 15, 2026
25 checks passed

ghost deleted the deprecate-savannah-fetcher branch January 15, 2026 02:44

Uh oh!

Conversation

ghost commented Aug 22, 2025 • edited by ghost Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Context

The Issue

The Solution

Packages changed

Deprecation notice

Things done

Uh oh!

jian-lin Aug 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ghost Aug 22, 2025 • edited by ghost Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

philiptaron Aug 22, 2025

Choose a reason for hiding this comment

Uh oh!

jian-lin Sep 17, 2025

Choose a reason for hiding this comment

Uh oh!

philiptaron left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

philiptaron Aug 22, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jian-lin commented Oct 1, 2025

Uh oh!

ghost commented Oct 1, 2025 • edited by ghost Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ghost commented Oct 11, 2025

Uh oh!

ghost commented Jan 8, 2026 • edited by ghost Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Footnotes

Uh oh!

jian-lin commented Jan 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ghost commented Jan 9, 2026 • edited by ghost Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Footnotes

Uh oh!

philiptaron left a comment

Choose a reason for hiding this comment

Uh oh!

jian-lin left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

ghost commented Jan 14, 2026

Uh oh!

jian-lin left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ghost commented Jan 15, 2026 • edited by ghost Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

ghost commented Aug 22, 2025 •

edited by ghost

Loading

jian-lin Aug 22, 2025 •

edited

Loading

ghost Aug 22, 2025 •

edited by ghost

Loading

ghost commented Oct 1, 2025 •

edited by ghost

Loading

ghost commented Jan 8, 2026 •

edited by ghost

Loading

jian-lin commented Jan 8, 2026 •

edited

Loading

ghost commented Jan 9, 2026 •

edited by ghost

Loading

jian-lin left a comment •

edited

Loading

ghost commented Jan 15, 2026 •

edited by ghost

Loading