Skip to content

Revert "Merge pull request #155048 from mweinelt/python2-must-die"#157959

Closed
Profpatsch wants to merge 1 commit intoNixOS:masterfrom
Profpatsch:revert-vte-drop
Closed

Revert "Merge pull request #155048 from mweinelt/python2-must-die"#157959
Profpatsch wants to merge 1 commit intoNixOS:masterfrom
Profpatsch:revert-vte-drop

Conversation

@Profpatsch
Copy link
Member

No. You can’t just delete software people depend on as their daily
driver based on a skewed idea of deprecation.

This reverts commit a97ae54, reversing
changes made to 37d3f75.

Motivation for this change
Things done
  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 22.05 Release Notes (or backporting 21.11 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
    • (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes
  • Fits CONTRIBUTING.md.

No. You can’t just delete software people depend on as their daily
driver based on a skewed idea of deprecation.

This reverts commit a97ae54, reversing
changes made to 37d3f75.
@Profpatsch Profpatsch requested review from aszlig and mweinelt February 3, 2022 08:56
@Profpatsch
Copy link
Member Author

For reference, I use lilyterm as my main terminal emulator application, and it didn’t just suddenly stop working because python2 hit an arbitrary milestone.


#### BINDINGS

libglademm = callPackage ./bindings/libglademm { };
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file appears to be missing.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is curious, I reverted the merge commit, maybe something went wrong.

Copy link
Member

@piegamesde piegamesde Feb 3, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have bad news for you: #155061

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happened to deprecation periods?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What deprecation periods do you mean? I never heard of any.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's possible in nixpkgs to have lib.warn as part of normal evaluations. Ofborg treats these as errors. We could potentially shove the warning into the alias scope, but I haven't seen anyone do that.

@jtojnar
Copy link
Member

jtojnar commented Feb 3, 2022

Also, I believe @samueldr was working on GTK 3-based lilyterm.

@Profpatsch
Copy link
Member Author

@AndersonTorres I am not okay with removing software I maintain without a deprecation period.

@piegamesde
Copy link
Member

No. You can’t just delete software people depend on as their daily
driver based on a skewed idea of deprecation.

Python 2 is EOL. Gtk 2 is EOL. This is not an "arbitrary milestone". It means that people stop investing their free time in keeping besaid software secure and running.

Over time, all packages need to be fixed, updated or removed. This is what other distros do as well. A lot of these packages are abandoned, unmaintained – either upstream, in nixpkgs, or both – or have a newer successor. Of course, I prefer it when packages can be fixed and don't need to be removed. I use some of these packages too.

In #157959, I tried to ping all maintainers of affected packages. I am sorry if you somehow weren't noticed in there.

I you can bring a few packages back without the old dependencies (like in #157868 for example), that would be awesome. Otherwise, we are in the fortunate situation that we can simply pin old nixpkgs and find the package in there.

@7c6f434c
Copy link
Member

7c6f434c commented Feb 3, 2022

We should stop treating the harmful situation that we tolerate software systems needing constant changes as desirable.

@Profpatsch
Copy link
Member Author

This throws a very bad light on Python, GTK and also the nix maintainers of said environments. I don’t trust them anymore to support software I use for more than a few years. I will adjust my views accordingly.

For the record, lilyterm does not depend on Python, it depends on vte, which had an optional dependency on Python, so this should have been unnecessary.

@Profpatsch Profpatsch closed this Feb 3, 2022
@piegamesde
Copy link
Member

I don’t trust them anymore to support software I use for more than a few years.

Gtk 2 has been in LTS mode for over a decade. Python 2 has prolonged their support frame for several years compared to the planned EOL. For reference, the commercial Java ecosystem wants to get paid after a few years for support.

Just because some software is not supported anymore does not make it instantly go away. The old nixpkgs versions are still there, and they won't change. But please understand that people do not want to invest their free time in keeping some obscure software with a handful of users from a past era alive.

For the record, lilyterm does not depend on Python, it depends on vte, which had an optional dependency on Python, so this should have been unnecessary.

Maybe, yes. Again, if you can bring back the package without those dependencies that would be awesome.

@AndersonTorres
Copy link
Member

For reference, I use lilyterm as my main terminal emulator application, and it didn’t just suddenly stop working because python2 hit an arbitrary milestone.

This arbitrary milestone is know as "Python developers are tired of maintaining Python 2 for more than 10 years".

@AndersonTorres I am not okay with removing software I maintain without a deprecation period.

Looks an interesting and useful policy to me. However, I never heard of a deprecation period on Nixpkgs before.
On the other hand we do not have a huge team to maintain deprecated software like a whole Python 2 ecosystem.

For the record, lilyterm does not depend on Python, it depends on vte, which had an optional dependency on Python, so this should have been unnecessary.

This is good news! If VTE is not hard dependent on Py2, it is possible to resurrect it! I will work on it.

However, it should be noted that lilyterm has no new releases for a long time.

@AndersonTorres
Copy link
Member

@Profpatsch

I have tried to build lilyterm-git. Here is it:

lilyterm> unpacking sources
lilyterm> unpacking source archive /nix/store/5mnyj36j878vc5z5fxhdzxjx2k3xbjc5-source
lilyterm> source root is source
lilyterm> patching sources
lilyterm> autoreconfPhase
lilyterm> autoreconf: export WARNINGS=
lilyterm> autoreconf: Entering directory '.'
lilyterm> autoreconf: configure.ac: not using Gettext
lilyterm> autoreconf: running: aclocal --force
lilyterm> configure.ac:16: warning: macro 'AM_PATH_GTK_2_0' not found in library
lilyterm> autoreconf: configure.ac: tracing
lilyterm> autoreconf: configure.ac: not using Libtool
lilyterm> autoreconf: running: intltoolize --copy --force
lilyterm> autoreconf: configure.ac: not using Gtkdoc
lilyterm> autoreconf: running: /nix/store/zmd15qlyb2w92hldhgwavv1682ib3f6z-autoconf-2.71/bin/autoconf --force
lilyterm> configure.ac:5: warning: AC_INIT: not a literal: "$_PACKAGE"
lilyterm> configure.ac:5: warning: AC_INIT: not a literal: "$_VERSION"
lilyterm> configure.ac:5: warning: AC_INIT: not a literal: "$_BUGREPORT"
lilyterm> configure.ac:19: warning: The macro `AC_HELP_STRING' is obsolete.
lilyterm> configure.ac:19: You should run autoupdate.
lilyterm> ./lib/autoconf/general.m4:204: AC_HELP_STRING is expanded from...
lilyterm> configure.ac:19: the top level
lilyterm> configure.ac:23: warning: The macro `AC_HELP_STRING' is obsolete.
lilyterm> configure.ac:23: You should run autoupdate.
lilyterm> ./lib/autoconf/general.m4:204: AC_HELP_STRING is expanded from...
lilyterm> configure.ac:23: the top level
lilyterm> configure.ac:52: warning: The macro `AC_PROG_INTLTOOL' is obsolete.
lilyterm> configure.ac:52: You should run autoupdate.
lilyterm> aclocal.m4:1187: AC_PROG_INTLTOOL is expanded from...
lilyterm> configure.ac:52: the top level
lilyterm> configure.ac:56: warning: The macro `GLIB_GNU_GETTEXT' is obsolete.
lilyterm> configure.ac:56: You should run autoupdate.
lilyterm> aclocal.m4:604: GLIB_GNU_GETTEXT is expanded from...
lilyterm> aclocal.m4:704: AM_GLIB_GNU_GETTEXT is expanded from...
lilyterm> configure.ac:56: the top level
lilyterm> configure.ac:56: warning: The macro `AC_TRY_LINK' is obsolete.
lilyterm> configure.ac:56: You should run autoupdate.
lilyterm> ./lib/autoconf/general.m4:2920: AC_TRY_LINK is expanded from...
lilyterm> lib/m4sugar/m4sh.m4:692: _AS_IF_ELSE is expanded from...
lilyterm> lib/m4sugar/m4sh.m4:699: AS_IF is expanded from...
lilyterm> ./lib/autoconf/general.m4:2249: AC_CACHE_VAL is expanded from...
lilyterm> ./lib/autoconf/general.m4:2270: AC_CACHE_CHECK is expanded from...
lilyterm> aclocal.m4:289: GLIB_LC_MESSAGES is expanded from...
lilyterm> aclocal.m4:604: GLIB_GNU_GETTEXT is expanded from...
lilyterm> aclocal.m4:704: AM_GLIB_GNU_GETTEXT is expanded from...
lilyterm> configure.ac:56: the top level
lilyterm> configure.ac:56: warning: The macro `AC_TRY_LINK' is obsolete.
lilyterm> configure.ac:56: You should run autoupdate.
lilyterm> ./lib/autoconf/general.m4:2920: AC_TRY_LINK is expanded from...
lilyterm> lib/m4sugar/m4sh.m4:692: _AS_IF_ELSE is expanded from...
lilyterm> lib/m4sugar/m4sh.m4:699: AS_IF is expanded from...
lilyterm> ./lib/autoconf/general.m4:2249: AC_CACHE_VAL is expanded from...
lilyterm> ./lib/autoconf/general.m4:2270: AC_CACHE_CHECK is expanded from...
lilyterm> lib/m4sugar/m4sh.m4:699: AS_IF is expanded from...
lilyterm> ./lib/autoconf/headers.m4:89: _AC_CHECK_HEADER_COMPILE is expanded from...
lilyterm> ./lib/autoconf/headers.m4:56: AC_CHECK_HEADER is expanded from...
lilyterm> aclocal.m4:388: GLIB_WITH_NLS is expanded from...
lilyterm> aclocal.m4:604: GLIB_GNU_GETTEXT is expanded from...
lilyterm> aclocal.m4:704: AM_GLIB_GNU_GETTEXT is expanded from...
lilyterm> configure.ac:56: the top level
lilyterm> configure.ac:56: warning: The macro `AC_TRY_LINK' is obsolete.
lilyterm> configure.ac:56: You should run autoupdate.
lilyterm> ./lib/autoconf/general.m4:2920: AC_TRY_LINK is expanded from...
lilyterm> lib/m4sugar/m4sh.m4:699: AS_IF is expanded from...
lilyterm> ./lib/autoconf/headers.m4:89: _AC_CHECK_HEADER_COMPILE is expanded from...
lilyterm> ./lib/autoconf/headers.m4:56: AC_CHECK_HEADER is expanded from...
lilyterm> aclocal.m4:388: GLIB_WITH_NLS is expanded from...
lilyterm> aclocal.m4:604: GLIB_GNU_GETTEXT is expanded from...
lilyterm> aclocal.m4:704: AM_GLIB_GNU_GETTEXT is expanded from...
lilyterm> configure.ac:56: the top level
lilyterm> configure.ac:56: warning: The macro `AC_OUTPUT_COMMANDS' is obsolete.
lilyterm> configure.ac:56: You should run autoupdate.
lilyterm> ./lib/autoconf/status.m4:1025: AC_OUTPUT_COMMANDS is expanded from...
lilyterm> aclocal.m4:388: GLIB_WITH_NLS is expanded from...
lilyterm> aclocal.m4:604: GLIB_GNU_GETTEXT is expanded from...
lilyterm> aclocal.m4:704: AM_GLIB_GNU_GETTEXT is expanded from...
lilyterm> configure.ac:56: the top level
lilyterm> configure.ac:60: warning: The macro `AC_HEADER_STDC' is obsolete.
lilyterm> configure.ac:60: You should run autoupdate.
lilyterm> ./lib/autoconf/headers.m4:704: AC_HEADER_STDC is expanded from...
lilyterm> configure.ac:60: the top level
lilyterm> configure.ac:16: error: possibly undefined macro: AM_PATH_GTK_2_0
lilyterm>       If this token and others are legitimate, please use m4_pattern_allow.
lilyterm>       See the Autoconf documentation.
lilyterm> autoreconf: error: /nix/store/zmd15qlyb2w92hldhgwavv1682ib3f6z-autoconf-2.71/bin/autoconf failed with exit status: 1
error: builder for '/nix/store/hprzv3jwq32ddqfp94hka7q14j3lnjmj-lilyterm-0.9.9.4+date=2019-07-25.drv' failed with exit code 1;
       last 10 log lines:
       > aclocal.m4:704: AM_GLIB_GNU_GETTEXT is expanded from...
       > configure.ac:56: the top level
       > configure.ac:60: warning: The macro `AC_HEADER_STDC' is obsolete.
       > configure.ac:60: You should run autoupdate.
       > ./lib/autoconf/headers.m4:704: AC_HEADER_STDC is expanded from...
       > configure.ac:60: the top level
       > configure.ac:16: error: possibly undefined macro: AM_PATH_GTK_2_0
       >       If this token and others are legitimate, please use m4_pattern_allow.
       >       See the Autoconf documentation.
       > autoreconf: error: /nix/store/zmd15qlyb2w92hldhgwavv1682ib3f6z-autoconf-2.71/bin/autoconf failed with exit status: 1
       For full logs, run 'nix log /nix/store/hprzv3jwq32ddqfp94hka7q14j3lnjmj-lilyterm-0.9.9.4+date=2019-07-25.drv'.

It looks like a hard dependency on GTK2.

@samueldr
Copy link
Member

samueldr commented Feb 5, 2022

Just saying, since I was pinged, I won't be using the Nixpkgs-provided lilyterm since anyway I was heavily patching it already to remove features.

But the tip-most git revision did work with GTK3, albeit it fails at runtime due to other issues. I removed some misfeatures (to me) causing issues. You probably don't want to use my fork, but just in case ¯\_(ツ)_/¯

@samueldr
Copy link
Member

samueldr commented Feb 5, 2022

And I have to agree that removing things hapazardly like that is not a good look. No warning, no grace period (even just one full release cycle).

How can I rely on Nixpkgs if things just vanish? If ~6 months ago a warning was explicitly placed, I would have had time to look and fix what I was using.

Instead, I had to scramble in two evenings to fixup what is might be the most important piece of software in my software stack.

@AndersonTorres
Copy link
Member

I will open an issue in Discourse and in matrix about this.

@nixos-discourse
Copy link

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/should-we-consider-a-deprecation-removal-policy-for-nixpkgs/17516/1

@jonringer
Copy link
Contributor

For reference, I use lilyterm as my main terminal emulator application, and it didn’t just suddenly stop working because python2 hit an arbitrary milestone.

Python2 was originally announced to be EOL in 2008, and was meant to be EOL in 2016, then extended to 2020. 14 years is a more than generous timelines for ecosystems to adjust and write python2+3 compatible code.

I think it's fairly safe to say that software still dependent on python2 in 2022 has been abandoned.

@flokli
Copy link
Member

flokli commented Feb 6, 2022

Isn't nixpkgs unstable the place where we should be able to deprecate/remove things?

People who prefer to have some grace period can always stick to the stable channels, and read release notes before upgrading to a new release.

@samueldr
Copy link
Member

samueldr commented Feb 6, 2022

Isn't nixpkgs unstable the place where we should be able to deprecate/remove things?

People who prefer to have some grace period can always stick to the stable channels, and read release notes before upgrading to a new release.

Announcements prior to removal should be made, in a way that gives unstable users time to act. The lack of procedure for correctly handling deprecation is the root cause of the issue. This is not like moving an attribute somewhere else, this is removing a still working package, which users may depend on.

It isn't fair to say that in unstable everything goes. Or else we might as well randomize attribute names every few weeks to keep users on their toes.

@samueldr
Copy link
Member

samueldr commented Feb 6, 2022

I think it's fairly safe to say that software still dependent on python2 in 2022 has been abandoned.

Without a warning, the maintainer of the package was unable to know they had to look at the packaging of their packages for reliance on python2. If there had been a warning/grace period for a full release, maintainers could have taken a look, and in this particular instance, fix the packaging.

Dropping python2 and python2 actually dependent software, I don't have an issue with. What I have an issue with is the method used.

@AndersonTorres
Copy link
Member

This is not like moving an attribute somewhere else, this is removing a still working package, which users may depend on.

I have tried to compile lilyterm without python2 dependency. Nonetheless, It is not working (because GTK2 dependency).
"Still working" does not apply here.

It isn't fair to say that in unstable everything goes. Or else we might as well randomize attribute names every few weeks to keep users on their toes.

As far as I remember it does not apply because, well, there is a policy to make them have the same name.

If there had been a warning/grace period for a full release, maintainers could have taken a look, and in this particular instance, fix the packaging.

Technically the maintainers were called; however by any reason Profpatsch was not called or did not notice.

Indeed I removed evilvte by myself, because it was old and unmaintained and with security breaches.

@samueldr
Copy link
Member

samueldr commented Feb 6, 2022

A proper deprecation scheme would warn users too. I misspoke when I said maintainers.

We know that in Nixpkgs, a lot of packages are co-maintained by non-maintainer users.

With a warning period that is done through the evaluation, users like me, and I suppose @Profpatsch, would have had a heads-up to look at the issue.

I have tried to compile lilyterm without python2 dependency. Nonetheless, It is not working (because GTK2 dependency).
"Still working" does not apply here.

The “still working” here means that before the removal, the existing package was working just fine. Removing a package that is obviously broken already is (imo) different than removing one that is currently still working.

@jonringer
Copy link
Contributor

It would be nice to have a deprecation mechanism which is compatible with ofborg and hydra.

Perhaps we could have something that is "the opposite" of how aliases work today, in which users have to explicitly opt out of. And for nixpkgs machinery we can just do something like config.disregardDeprecations = true; to avoid emitting stuff to stderr for ofborg and hydra.

Then again, the average user will probably do something like, "Why am I getting a python is deprecation error, when I'm not using it?"

@samueldr
Copy link
Member

samueldr commented Feb 7, 2022

Then again, the average user will probably do something like, "Why am I getting a python is deprecation error, when I'm not using it?"

That's thinking at the trunk, while we're considering leaf packages here.

I agree that if the deprecation starts at a level low enough that is un-actionable, it wouldn't be useful.

In this particular scenario, the leaf packages that still build would have to be mentioned in the notice, somehow. For packages that currently are broken the value would be much much less.

The target of removal was pygtk. A good message would state something like %s depends on pygtk, which is slated for removal by 20yy-mm-dd.. I'm not sure if it's possible to do in some form of automatic manner.

This way "base" users would have a message explaining which package is at fault. While it would be generally unactionable for "base" users, they would be given some time to look for alternatives. More involved users would have a warning (ideally for a full release cycle) about the impending change, and some of them, hopefully, would be able to schedule some time to look at the issue.

@jonringer
Copy link
Contributor

jonringer commented Feb 7, 2022

The target of removal was pygtk. A good message would state something like %s depends on pygtk, which is slated for removal by 20yy-mm-dd.. I'm not sure if it's possible to do in some form of automatic manner.

I don't think this could be done generically. From the nix perspective, it's just evaluating thunks. We would need something like builtins.unsafeGetTopLevelDrv in order for nix to go back up the stack and inspect which entrypoint eventually included the direct or transitive dependency.

The other thing is that we would need some way to differentiate what should be an origin point. Building a nixos toplevel configuration is just a derivation, but it's not really meaningful to have "nixos-configuration-.... depends on python2". However,
if we were able to do the entire stack, then that would be meaningful:

  Warning attempting to build nixos-configuration-21.11.87923048 because:
     nixos-configuration-21.11.87923048  requires lilyterm
     lilyterm requires vte
     vte requires python2
    python2 is slated for removal, please attempt to remedy this.
# deprecated-packages.nix
let
  deprecationStatement = drv: lib.warn "${drv} depends on ${builtins.unsafeGetTopLevelDrv.name}, which is deprecated";
  deprecatedPackages = [
    ....
   ];
 in
 prev: final: mapAttrs deprecationStatement deprecatedPackages

I could also be complete wrong. cc @edolstra

@AndersonTorres
Copy link
Member

Then again, the average user will probably do something like, "Why am I getting a python is deprecation error, when I'm not using it?"

I get the Package cl is deprecated every single day I launch my Emacs :)

Maybe a message like "python 2 is deprecated" gets hit by an average user, and it goes reported to a superior instance, like the local sysadmin that will investigate this.

About this, it will be useful some sort of backtrace of this warning - IIRC nix has something like why-depends subcommand.

@AndersonTorres AndersonTorres mentioned this pull request May 31, 2022
13 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

9 participants