Skip to content

Python unstable 2022-01-13#154969

Merged
mweinelt merged 432 commits intostagingfrom
python-unstable
Jan 23, 2022
Merged

Python unstable 2022-01-13#154969
mweinelt merged 432 commits intostagingfrom
python-unstable

Conversation

@jonringer
Copy link
Contributor

@jonringer jonringer commented Jan 14, 2022

Motivation for this change

It's that time again, not going to immediately trigger eval since the recent CVE master merges caused a lot of queue builds.

https://hydra.nixos.org/jobset/nixpkgs/python-unstable

Previous python-unstable: #149301

Things done
  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 22.05 Release Notes (or backporting 21.11 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
    • (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes
  • Fits CONTRIBUTING.md.

@github-actions github-actions bot added the 6.topic: python Python is a high-level, general-purpose programming language. label Jan 14, 2022
@ofborg ofborg bot added 2.status: merge conflict This PR has merge conflicts with the target branch 10.rebuild-darwin: 501+ This PR causes many rebuilds on Darwin and should normally target the staging branches. 10.rebuild-darwin: 5001+ This PR causes many rebuilds on Darwin and must target the staging branches. 10.rebuild-linux: 501+ This PR causes many rebuilds on Linux and should normally target the staging branches. 10.rebuild-linux: 5001+ This PR causes many rebuilds on Linux and must target the staging branches. and removed 2.status: merge conflict This PR has merge conflicts with the target branch labels Jan 14, 2022
@dotlambda dotlambda changed the title Python unstable 2022-01-013 Python unstable 2022-01-13 Jan 14, 2022
@ofborg ofborg bot added the 8.has: package (new) This PR adds a new package label Jan 14, 2022
@ofborg ofborg bot added 2.status: merge conflict This PR has merge conflicts with the target branch and removed 2.status: merge conflict This PR has merge conflicts with the target branch labels Jan 14, 2022
@ofborg ofborg bot added 2.status: merge conflict This PR has merge conflicts with the target branch and removed 2.status: merge conflict This PR has merge conflicts with the target branch labels Jan 16, 2022
@ofborg ofborg bot removed the 2.status: merge conflict This PR has merge conflicts with the target branch label Jan 17, 2022
@ofborg ofborg bot added the 2.status: merge conflict This PR has merge conflicts with the target branch label Jan 17, 2022
@FRidh
Copy link
Member

FRidh commented Jan 17, 2022

Just did a force push after rebasing on staging.

@ofborg ofborg bot removed the 2.status: merge conflict This PR has merge conflicts with the target branch label Jan 17, 2022
@cpcloud cpcloud mentioned this pull request Jan 17, 2022
13 tasks
@ofborg ofborg bot removed the 2.status: merge conflict This PR has merge conflicts with the target branch label Jan 23, 2022
@mweinelt mweinelt merged commit 645ddf9 into staging Jan 23, 2022
@jonringer
Copy link
Contributor Author

Thanks @mweinelt @dotlambda @fabaff for the help :)

@SuperSandro2000
Copy link
Member

@jonringer can you protect the branch so that it does not get deleted by accident?

@trofi
Copy link
Contributor

trofi commented Jan 23, 2022

Bisec says c84b744 python3Packages.pyparsing: 2.4.7 -> 3.0.6 broke spice build:

$ nix build -f. spice -L
...
spice> spice-common| Message: Checking for python module six
spice> spice-common| Message: Checking for python module pyparsing
spice> subprojects/spice-common/meson.build:137:8: ERROR: Problem encountered: Python module pyparsing not found
spice> A full log can be found at /build/spice-0.15.0/build/meson-logs/meson-log.txt

@mweinelt
Copy link
Member

Fixed in #156305

@trofi
Copy link
Contributor

trofi commented Jan 23, 2022

Bisect says c4f55d1 python3Packages.fonttools: 4.26.2 -> 4.28.5 broke python3Packages.afdko tests in staging:

$ nix build -f . python3Packages.afdko
...
python3.9-afdko> tests/ttxn_test.py FFFFFFFFFFFFF                                         [ 73%]
...
python3.9-afdko> =================================== FAILURES ===================================
python3.9-afdko> ___________ test_dump[args0-SourceSansPro-Black_subset.otf-OTF.ttx] ____________
python3.9-afdko> args = [], font_filename = 'SourceSansPro-Black_subset.otf'
python3.9-afdko> exp_filename = 'OTF.ttx'
python3.9-afdko>     @pytest.mark.parametrize('args, font_filename, exp_filename', [
python3.9-afdko>         ([], OTF_FONT, 'OTF.ttx'),
python3.9-afdko>         ([], TTF_FONT, 'TTF.ttx'),
python3.9-afdko>         (['nh'], OTF_FONT, 'OTF_no_hints.ttx'),
python3.9-afdko>         (['tGPOS'], OTF_FONT, 'OTF_GPOS_only.ttx'),
python3.9-afdko>         (['tGSUB'], OTF_FONT, 'OTF_GSUB_only.ttx'),
python3.9-afdko>         (['tGPOS'], OTF2_FONT, 'OTF2_GPOS_only.ttx'),
python3.9-afdko>         (['tGSUB'], OTF2_FONT, 'OTF2_GSUB_only.ttx'),
python3.9-afdko>         (['tGPOS'], OTF3_FONT, 'OTF3_GPOS_only.ttx'),
python3.9-afdko>         (['tGSUB'], OTF3_FONT, 'OTF3_GSUB_only.ttx'),
python3.9-afdko>         (['tGPOS'], TTF2_FONT, 'TTF2_GPOS_only.ttx'),
python3.9-afdko>         (['tGSUB'], TTF2_FONT, 'TTF2_GSUB_only.ttx'),
python3.9-afdko>         (['tGPOS', 'se'], TTF3_FONT, 'TTF3_GPOS_only.ttx'),
python3.9-afdko>         (['tGSUB', 'se'], TTF3_FONT, 'TTF3_GSUB_only.ttx'),
python3.9-afdko>     ])
python3.9-afdko>     def test_dump(args, font_filename, exp_filename):
python3.9-afdko>         save_path = tempfile.mkstemp()[1]
python3.9-afdko> >       runner(['-t', TOOL, '-f', font_filename, '-o',
python3.9-afdko>                 f'o{save_path}'] + args)
python3.9-afdko> tests/ttxn_test.py:39:
python3.9-afdko> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
python3.9-afdko> tests/runner.py:259: in main
python3.9-afdko>     return run_tool(opts)
python3.9-afdko> tests/runner.py:107: in run_tool
python3.9-afdko>     return subprocess.check_call(args, timeout=TIMEOUT)
python3.9-afdko> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
python3.9-afdko> popenargs = (['ttxn', '-o/build/tmp1ws_e4mg', '/build/afdko-3.7.1/tests/ttxn_data/input/SourceSansPro-Black_subset.otf'],)
python3.9-afdko> kwargs = {'timeout': 240}, retcode = 1
python3.9-afdko> cmd = ['ttxn', '-o/build/tmp1ws_e4mg', '/build/afdko-3.7.1/tests/ttxn_data/input/SourceSansPro-Black_subset.otf']
python3.9-afdko>     def check_call(*popenargs, **kwargs):
python3.9-afdko>         """Run command with arguments.  Wait for command to complete.  If
python3.9-afdko>         the exit code was zero then return, otherwise raise
python3.9-afdko>         CalledProcessError.  The CalledProcessError object will have the
python3.9-afdko>         return code in the returncode attribute.
python3.9-afdko>         The arguments are the same as for the call function.  Example:
python3.9-afdko>         check_call(["ls", "-l"])
python3.9-afdko>         """
python3.9-afdko>         retcode = call(*popenargs, **kwargs)
python3.9-afdko>         if retcode:
python3.9-afdko>             cmd = kwargs.get("args")
python3.9-afdko>             if cmd is None:
python3.9-afdko>                 cmd = popenargs[0]
python3.9-afdko> >           raise CalledProcessError(retcode, cmd)
python3.9-afdko> E           subprocess.CalledProcessError: Command '['ttxn', '-o/build/tmp1ws_e4mg', '/build/afdko-3.7.1/tests/ttxn_data/input/SourceSansPro-Black_subset.otf']' returned non-zero exit status 1.
python3.9-afdko> /nix/store/hxgsg4nn1w47lrbzb20l0b2kzhgi6fmy-python3-3.9.10/lib/python3.9/subprocess.py:373: CalledProcessError
python3.9-afdko> ----------------------------- Captured stderr call -----------------------------
python3.9-afdko> Dumping "/build/afdko-3.7.1/tests/ttxn_data/input/SourceSansPro-Black_subset.otf" to "/build/tmp1ws_e4mg"...
python3.9-afdko> Traceback (most recent call last):
python3.9-afdko>   File "/nix/store/gis1kmb2lk16plkfxsz6jfdhlvl25pxq-python3.9-afdko-3.7.1/bin/.ttxn-wrapped", line 9, in <module>
python3.9-afdko>     sys.exit(main())
python3.9-afdko>   File "/nix/store/gis1kmb2lk16plkfxsz6jfdhlvl25pxq-python3.9-afdko-3.7.1/lib/python3.9/site-packages/afdko/ttxn.py", line 2324, in main
python3.9-afdko>     ttnDump(input_file, output, options, showExtensionFlag,
python3.9-afdko>   File "/nix/store/7688i22zni4pkznasabcc0n7hcsixscv-python3.9-fonttools-4.28.5/lib/python3.9/site-packages/fontTools/misc/loggingTools.py", line 372, in wrapper
python3.9-afdko>     return func(*args, **kwds)
python3.9-afdko>   File "/nix/store/gis1kmb2lk16plkfxsz6jfdhlvl25pxq-python3.9-afdko-3.7.1/lib/python3.9/site-packages/afdko/ttxn.py", line 2120, in ttnDump
python3.9-afdko>     ttf = TTXNTTFont(input_file, 0, allowVID=options.allowVID,
python3.9-afdko> AttributeError: 'Options' object has no attribute 'allowVID'
python3.9-afdko> ERROR: Command '['ttxn', '-o/build/tmp1ws_e4mg', '/build/afdko-3.7.1/tests/ttxn_data/input/SourceSansPro-Black_subset.otf']' returned non-zero exit status 1.
python3.9-afdko> ------------------------------ Captured log call -------------------------------
python3.9-afdko> ERROR    runner:runner.py:112 Command '['ttxn', '-o/build/tmp1ws_e4mg', '/build/afdko-3.7.1/tests/ttxn_data/input/SourceSansPro-Black_subset.otf']' returned non-zero exit status 1.
...

@jonringer
Copy link
Contributor Author

@jonringer can you protect the branch so that it does not get deleted by accident?

@SuperSandro2000 I cannot. I think you have to be admin.

cc @domenkozar

@c0bw3b c0bw3b mentioned this pull request Jan 24, 2022
13 tasks
@mweinelt mweinelt deleted the python-unstable branch January 25, 2022 11:10
@mweinelt
Copy link
Member

mweinelt commented Jan 25, 2022

Poked on matrix regarding branch protection. Was asked to rename the branch to python-updates so we are aligned with other langs (cmp. haskell-updates). I didn't mind, I hope noone else does either.

@SuperSandro2000
Copy link
Member

Does hydra mind?

@mweinelt
Copy link
Member

mweinelt commented Jan 25, 2022

Was already updated with that change in mind. The jobset is also now visible on the project page again.

https://hydra.nixos.org/jobset/nixpkgs/python-unstable#tabs-configuration

@smunix
Copy link
Contributor

smunix commented Feb 11, 2022

This update has broken kitty #155717

❯ sudo nixos-rebuild switch                                                                                                                                                                                                                                                  [99/370]
building the system configuration...                                                                                                                                                                                                                                                 
warning: Git tree '/etc/nixos' is dirty                        
error: builder for '/nix/store/dhah3nazhf161y7rb77770khvmsxs8xh-kitty-0.24.2.drv' failed with exit code 2;                                                                                                                                                                           
       last 10 log lines:                                    
       >     for result in self.emit(name, *args, allowed_exceptions=allowed_exceptions):                                                 
       >   File "/nix/store/13z9v2893iapkj12gk47q04w2197gysq-python3.9-sphinx-4.4.0/lib/python3.9/site-packages/sphinx/events.py", line 109, in emit
       >     raise ExtensionError(__("Handler %r for event %r threw an exception") %                                                      
       > sphinx.errors.ExtensionError: Handler <function _html_page_context at 0x7ffff3218c10> for event 'html-page-context' threw an exception (exception: [Errno 2] No such file or directory: '/nix/store/pppq4jc77qw9qv4rqghqa32rjdm314hq-python3.9-furo-2022.1.2/lib/python3.9/s
ite-packages/furo/theme/furo/static/styles/furo.css')                                                                                     
       >                                         
       > Extension error (furo):                    
       > Handler <function _html_page_context at 0x7ffff3218c10> for event 'html-page-context' threw an exception (exception: [Errno 2] No such file or directory: '/nix/store/pppq4jc77qw9qv4rqghqa32rjdm314hq-python3.9-furo-2022.1.2/lib/python3.9/site-packages/furo/theme/furo/s
tatic/styles/furo.css')                    
       > make[1]: *** [Makefile:24: html] Error 2                                                                                                                                                                                                                                    
       > make[1]: Leaving directory '/build/source/docs'                                                                                                                                                                                                                             

@mweinelt
Copy link
Member

"This" is pretty vague for a package set update. The build of kitty might need to pin an older version of python3Packages.furo. Maybe there is a fix for this update upstream already.

@smunix
Copy link
Contributor

smunix commented Feb 11, 2022

Agreed, the fix needs to be made in kitty indeed. I'll discuss this with Kitty's maintainers.

@Luflosi
Copy link
Contributor

Luflosi commented Feb 11, 2022

This is not a kitty problem. furo is broken in nixpkgs because it depends on webpack to build some critical files, which is itself broken.

@Luflosi
Copy link
Contributor

Luflosi commented Feb 11, 2022

I created #159255, which fixes the problem.

@06kellyjac
Copy link
Member

the tomlkit update broke remarshal which is used by pkgs.formats (fixed in #159074)

https://github.com/NixOS/nixpkgs/blob/master/pkgs/pkgs-lib/formats.nix

Were there any tests to catch this? since pkgs.formats is used in quite a lot of modules and is a pain to fix without waiting for fixes to hit unstable
If not maybe we need more tests around "pkgs-lib"?

@mweinelt
Copy link
Member

Python-unstable does not run NixOS tests, but maybe we should given that the test runner is also written in python.

@06kellyjac
Copy link
Member

Yeah, at the very least we should probably run some of the super core tests e.g pkgs-lib and those meta tests for the nixos testsuite itself (which I think are still in a PR but I cant find them right now)

@jonringer
Copy link
Contributor Author

jonringer commented Feb 11, 2022

Solved by #159322

@smunix
Copy link
Contributor

smunix commented Feb 12, 2022

I created #159255, which fixes the problem.

thanks @Luflosi

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

6.topic: python Python is a high-level, general-purpose programming language. 8.has: clean-up This PR removes packages or removes other cruft 8.has: package (new) This PR adds a new package 10.rebuild-darwin: 501+ This PR causes many rebuilds on Darwin and should normally target the staging branches. 10.rebuild-darwin: 5001+ This PR causes many rebuilds on Darwin and must target the staging branches. 10.rebuild-linux: 501+ This PR causes many rebuilds on Linux and should normally target the staging branches. 10.rebuild-linux: 5001+ This PR causes many rebuilds on Linux and must target the staging branches.

Projects

None yet

Development

Successfully merging this pull request may close these issues.