Skip to content

staging-next 2025-05-30#412425

Merged
vcunat merged 1295 commits intomasterfrom
staging-next
Jun 18, 2025
Merged

staging-next 2025-05-30#412425
vcunat merged 1295 commits intomasterfrom
staging-next

Conversation

@vcunat
Copy link
Member

@vcunat vcunat commented May 30, 2025

@vcunat vcunat added the 1.severity: security Issues which raise a security issue, or PRs that fix one label May 30, 2025
@github-actions github-actions bot added the 4.workflow: staging A staging-next or staging-next-XX.YY branch label May 30, 2025
@ofborg ofborg bot added 2.status: merge conflict This PR has merge conflicts with the target branch and removed 2.status: merge conflict This PR has merge conflicts with the target branch labels May 30, 2025
@github-actions github-actions bot added 10.rebuild-darwin: 5001+ This PR causes many rebuilds on Darwin and must target the staging branches. 10.rebuild-darwin: 501+ This PR causes many rebuilds on Darwin and should normally target the staging branches. 10.rebuild-linux: 5001+ This PR causes many rebuilds on Linux and must target the staging branches. 10.rebuild-linux: 501+ This PR causes many rebuilds on Linux and should normally target the staging branches. labels May 30, 2025
@mweinelt
Copy link
Member

mweinelt commented May 30, 2025

@mdaniels5757
Copy link
Member

mdaniels5757 commented May 31, 2025

@ofborg ofborg bot added the 2.status: merge conflict This PR has merge conflicts with the target branch label May 31, 2025
@ofborg ofborg bot removed the 2.status: merge conflict This PR has merge conflicts with the target branch label May 31, 2025
@sternenseemann
Copy link
Member

sternenseemann commented May 31, 2025

Looks like libxml2/lxml changes affected some test cases.

@risicle
Copy link
Contributor

risicle commented May 31, 2025

FWIW the instructions on the manual check (https://github.com/NixOS/nixpkgs/actions/runs/15364814782/job/43236341422?pr=412425#step:6:234) seem to be incorrect and should actually direct people to use nix-shell doc/shell.nix

@risicle
Copy link
Contributor

risicle commented May 31, 2025

Oh great I've made it worse.

@sternenseemann
Copy link
Member

sternenseemann commented May 31, 2025

  • python3.pkgs.aiohttp
log
=================================== FAILURES ===================================
____________________ test_keepalive_expires_on_time[pyloop] ____________________
[gw2] linux -- Python 3.13.3 /nix/store/386v0rwajd9s5553kjkji2dv6dkck2b8-python3-3.13.3/bin/python3.13

aiohttp_client = <function aiohttp_client.<locals>.go at 0x7fffed9fd1c0>

    async def test_keepalive_expires_on_time(aiohttp_client: AiohttpClient) -> None:
        """Test that the keepalive handle expires on time."""
    
        async def handler(request: web.Request) -> web.Response:
            body = await request.read()
            assert b"" == body
            return web.Response(body=b"OK")
    
        app = web.Application()
        app.router.add_route("GET", "/", handler)
    
        connector = aiohttp.TCPConnector(limit=1)
        client = await aiohttp_client(app, connector=connector)
    
        loop = asyncio.get_running_loop()
        now = loop.time()
    
        # Patch loop time so we can control when the keepalive timeout is processed
        with mock.patch.object(loop, "time") as loop_time_mock:
            loop_time_mock.return_value = now
            resp1 = await client.get("/")
            await resp1.read()
            request_handler = client.server.handler.connections[0]
    
            # Ensure the keep alive handle is set
            assert request_handler._keepalive_handle is not None
    
            # Set the loop time to exactly the keepalive timeout
            loop_time_mock.return_value = request_handler._next_keepalive_close_time
    
            # sleep twice to ensure the keep alive timeout is processed
            await asyncio.sleep(0)
            await asyncio.sleep(0)
    
            # Ensure the keep alive handle expires
>           assert request_handler._keepalive_handle is None
E           assert <TimerHandle when=17057419.62006039 RequestHandler._process_keepalive()> is None
E            +  where <TimerHandle when=17057419.62006039 RequestHandler._process_keepalive()> = <RequestHandler connected>._keepalive_handle

aiohttp_client = <function aiohttp_client.<locals>.go at 0x7fffed9fd1c0>
app        = <Application 0x7fffec5517c0>
client     = <aiohttp.test_utils.TestClient object at 0x7fffec532e50>
connector  = <aiohttp.connector.TCPConnector object at 0x7fffedbfa2c0>
handler    = <function test_keepalive_expires_on_time.<locals>.handler at 0x7fffed9fcc20>
loop       = <_UnixSelectorEventLoop running=False closed=False debug=False>
loop_time_mock = <MagicMock name='time' id='140737314286160'>
now        = 17053789.62006039
request_handler = <RequestHandler connected>
resp1      = <ClientResponse(http://127.0.0.1:36891/) [200 OK]>
<CIMultiDictProxy('Content-Length': '2', 'Content-Type': 'application/octet-stream', 'Date': 'Sat, 31 May 2025 21:37:15 GMT', 'Server': 'Python/3.13 aiohttp/3.11.18')>


tests/test_web_functional.py:2364: AssertionError
============================= slowest 10 durations =============================
9.51s call     tests/test_run_app.py::TestShutdown::test_shutdown_new_conn_rejected
4.01s call     tests/test_run_app.py::TestShutdown::test_shutdown_pending_handler_responds
2.50s call     tests/test_run_app.py::TestShutdown::test_shutdown_wait_for_handler
2.41s call     tests/test_run_app.py::TestShutdown::test_shutdown_handler_cancellation_suppressed
2.00s call     tests/test_client_functional.py::test_read_timeout_between_chunks[pyloop]
2.00s call     tests/test_client_functional.py::test_set_cookies_max_age[pyloop]
1.61s call     tests/test_run_app.py::TestShutdown::test_shutdown_timeout_handler
1.51s call     tests/test_run_app.py::TestShutdown::test_shutdown_timeout_not_reached
1.01s call     tests/test_run_app.py::TestShutdown::test_shutdown_close_websockets
1.01s call     tests/test_run_app.py::TestShutdown::test_shutdown_close_idle_keepalive
=========================== short test summary info ============================
SKIPPED [1] tests/test_client_session.py:968: The check is applied in DEBUG mode only
SKIPPED [1] tests/test_connector.py:3103: Proactor Event loop present only in Windows
SKIPPED [1] tests/test_connector.py:3111: Proactor Event loop present only in Windows
SKIPPED [1] tests/test_connector.py:3122: Proactor Event loop present only in Windows
SKIPPED [1] tests/test_connector.py:3320: Proactor Event loop present only in Windows
SKIPPED [1] tests/test_multipart_helpers.py:99: need more smart parser which respects quoted text
SKIPPED [1] tests/test_multipart_helpers.py:446: should raise decoding error: %82 is invalid for latin1
SKIPPED [1] tests/test_multipart_helpers.py:455: should raise decoding error: %E4 is invalid for utf-8
SKIPPED [1] tests/test_resolver.py:404: aiodns <3.2.0 required
SKIPPED [1] tests/test_multipart_helpers.py:510: urllib.parse.unquote is tolerate to standalone % chars
SKIPPED [1] tests/test_multipart_helpers.py:519: urllib.parse.unquote is tolerate to standalone % chars
SKIPPED [1] tests/test_proxy_functional.py:130: asyncio on this python supports TLS in TLS
SKIPPED [1] tests/test_proxy_functional.py:416: we need to reconsider how we test this
SKIPPED [1] tests/test_proxy_functional.py:437: we need to reconsider how we test this
SKIPPED [1] tests/test_web_app.py:379: The check is applied in DEBUG mode only
SKIPPED [1] tests/test_urldispatch.py:1021: aiohttp folder is not placed in user's HOME
SKIPPED [1] tests/test_web_runner.py:121: Proactor Event loop present only in Windows
SKIPPED [1] tests/test_web_runner.py:133: Proactor Event loop present only in Windows
XFAIL tests/autobahn/test_autobahn.py::test_client
XFAIL tests/autobahn/test_autobahn.py::test_server
XFAIL tests/test_connector.py::test_del_with_scheduled_cleanup[pyloop]
XFAIL tests/test_client_functional.py::test_broken_connection[pyloop]
XFAIL tests/test_http_parser.py::test_http_request_parser_utf8_request_line[c-parser-pyloop] - Regression test for Py parser. May match C behaviour later.
XFAIL tests/test_http_parser.py::test_parse_unusual_request_line[c-parser-pyloop] - Regression test for Py parser. May match C behaviour later.
XFAIL tests/test_http_parser.py::test_parse_uri_utf8[c-parser-pyloop] - reason: Not valid HTTP. Maybe update py-parser to reject later.
XFAIL tests/test_websocket_parser.py::test_parse_frame_header_new_data_err[pyloop]
FAILED tests/test_web_functional.py::test_keepalive_expires_on_time[pyloop] - assert <TimerHandle when=17057419.62006039 RequestHandler._process_keepaliv...
======= 1 failed, 3461 passed, 18 skipped, 8 xfailed in 62.44s (0:01:02) =======

@mweinelt
Copy link
Member

That test failure on aiohttp looks transient to me.

@vcunat
Copy link
Member Author

vcunat commented Jun 1, 2025

@sternenseemann
Copy link
Member

@mweinelt I can consistently reproduce the failure (albeit only on a single machine). Everything timing relevant looks to be mocked in that test, so it should not fail.

@vcunat
Copy link
Member Author

vcunat commented Jun 1, 2025

I managed to build it locally on the first try. x86_64-linux

/nix/store/596b6bg1nxyqpnaw5sy1ahglkn7c34q8-python3.13-aiohttp-3.11.18

@mweinelt
Copy link
Member

mweinelt commented Jun 1, 2025

Tried it on various machines and architectures, but I fail to reproduce. Anything special about the machine where it fails for you?

@sternenseemann
Copy link
Member

Tried it on various machines and architectures, but I fail to reproduce. Anything special about the machine where it fails for you?

I also haven't been able to reproduce it elsewhere. No ideas what could be the issue.

vcunat referenced this pull request Jun 2, 2025
https://hydra.nixos.org/build/298833701/nixlog/3/tail

What I tried:
 - updating to 0.20.8 doesn't solve the issue
   (we could still do that easily, e.g. on staging;
    I just didn't feel like trashing successful binaries for aarch64-darwin)
 - upgrading to 0.22.1 brings others (2 patches not applying and more problems...)
@mdaniels5757 mdaniels5757 mentioned this pull request Jun 2, 2025
3 tasks
@fabianhjr
Copy link
Member

The following PR should unbreak libosinfo in staging-next: #413412

Currently affects gnome/nautilus on staging-next

error: builder for '/nix/store/348q9hzm3vsgbpgn28whfyhazrbamag7-libosinfo-1.11.0.drv' failed with exit code 1;
       last 25 log lines:
       > ../osinfo/osinfo_loader.c: At top level:
       > ../osinfo/osinfo_loader.c:1905:42: error: expected ';', ',' or ')' before 'ATTRIBUTE_UNUSED'
       >  1905 | catchXMLError(void *ctx, const char *msg ATTRIBUTE_UNUSED, ...)
       >       |                                          ^~~~~~~~~~~~~~~~
       > ../osinfo/osinfo_loader.c: In function 'osinfo_loader_process_xml':
       > ../osinfo/osinfo_loader.c:1948:25: error: 'catchXMLError' undeclared (first use in this function)
       >  1948 |     pctxt->sax->error = catchXMLError;
       >       |                         ^~~~~~~~~~~~~
       > ../osinfo/osinfo_loader.c:1948:25: note: each undeclared identifier is reported only once for each function it appears in
       > [63/111] Compiling C object osinfo/libosinfo-1.0.so.0.1011.0.p/osinfo_image.c.o
       > [64/111] Compiling C object osinfo/libosinfo-1.0.so.0.1011.0.p/osinfo_tree.c.o
       > [65/111] Compiling C object osinfo/libosinfo-1.0.so.0.1011.0.p/osinfo_imagelist.c.o
       > [66/111] Compiling C object tests/test-device.p/test-device.c.o
       > [67/111] Compiling C object tools/osinfo-detect.p/osinfo-detect.c.o
       > [68/111] Compiling C object tools/osinfo-query.p/osinfo-query.c.o
       > [69/111] Compiling C object tools/osinfo-install-script.p/osinfo-install-script.c.o
       > [70/111] Compiling C object osinfo/libosinfo-1.0.so.0.1011.0.p/osinfo_os.c.o
       > [71/111] Compiling C object osinfo/libosinfo-1.0.so.0.1011.0.p/osinfo_media.c.o
       > [72/111] Compiling C object tests/test-devicelinklist.p/test-devicelinklist.c.o
       > [73/111] Compiling C object tests/test-firmware.p/test-firmware.c.o
       > [74/111] Compiling C object tests/test-entity.p/test-entity.c.o
       > [75/111] Compiling C object osinfo/libosinfo-1.0.so.0.1011.0.p/osinfo_db.c.o
       > [76/111] Compiling C object tests/test-filter.p/test-filter.c.o
       > [77/111] Compiling C object tests/test-db.p/test-db.c.o
       > ninja: build stopped: subcommand failed.
       For full logs, run:
         nix log /nix/store/348q9hzm3vsgbpgn28whfyhazrbamag7-libosinfo-1.11.0.drv
error: 1 dependencies of derivation '/nix/store/5qby3b1n74nqmhr9vvqwgi78kqz8376d-localsearch-3.9.0.drv' failed to build
error: 1 dependencies of derivation '/nix/store/dxkr73qa13lfla321sm4g3znl6gdar1n-gnome-control-center-48.1.drv' failed to build
error: 1 dependencies of derivation '/nix/store/a8nm31q6i504xxc7vjlw5jmsnb1i8gvr-localsearch-3.9.0_fish-completions.drv' failed to build
error: 1 dependencies of derivation '/nix/store/2ndqrcmzaswm3lny1wssa9ygvi6idxs5-nautilus-48.1.drv' failed to build
error: 1 dependencies of derivation '/nix/store/lr776jazhkdaawqv801xhidg4acyx0zw-system-generators.drv' failed to build
error: 1 dependencies of derivation '/nix/store/m27pcsgzwnshb5fya41g0lpgxg2akmpg-system-shutdown.drv' failed to build
error: 1 dependencies of derivation '/nix/store/kpw7k587jh0w8l8rjjdjs2jajws1wzyv-user-generators.drv' failed to build
error: 1 dependencies of derivation '/nix/store/j87rnz05cn0a81rd6cca3ys7kznxpa9r-file-roller-44.5.drv' failed to build
error: 1 dependencies of derivation '/nix/store/1mpaylc67pzh0bkcabaqkzqv31d3xn5r-gnome-control-center-48.1_fish-completions.drv' failed to build
error: 1 dependencies of derivation '/nix/store/hy329x7jm0vmnkhrvzgghx0xwql0wf11-nautilus-48.1_fish-completions.drv' failed to build
error: 1 dependencies of derivation '/nix/store/kw3wrax211z4xr38x881iwskfyn0dwqp-unit-display-manager.service.drv' failed to build
error: 1 dependencies of derivation '/nix/store/qri9bzf61bhi8di75kai74sbfis5yyr0-file-roller-44.5_fish-completions.drv' failed to build
error: 4 dependencies of derivation '/nix/store/8via3i853z33y81rj3rrl83j90qaa7w5-man-paths.drv' failed to build
error: 4 dependencies of derivation '/nix/store/gvfimby57m677mzrhkbbl6kb43xhchdq-system-path.drv' failed to build
error: 1 dependencies of derivation '/nix/store/y3xhfqg2prvj6rcrswpsfkrmbgfxfi7r-X-Restart-Triggers-polkit.drv' failed to build
error: 3 dependencies of derivation '/nix/store/19q9lsc1f940hdh4y3f17pwgbk957x6c-dbus-1.drv' failed to build
error: 1 dependencies of derivation '/nix/store/6px7l8if1c1ldba60vk17yvkwwah6l35-etc-pam-environment.drv' failed to build
error: 1 dependencies of derivation '/nix/store/fybnqd8x1fprb8ksqinzrlhq06giw03x-man-cache.drv' failed to build
error: 1 dependencies of derivation '/nix/store/9nkc44693rlgclnfv26h82k6zyf87gxb-set-environment.drv' failed to build
error: 4 dependencies of derivation '/nix/store/hrwbi0hbcs74y0cxz0a0gnnv36bv627p-system_fish-completions.drv' failed to build
error: 1 dependencies of derivation '/nix/store/gn2wwd1gmza0jhjbsmbvhdjdwm517qmz-unit-accounts-daemon.service.drv' failed to build
error: 1 dependencies of derivation '/nix/store/g8xbmrs2jg3j3wsjvysblhqv8jmg1imw-X-Restart-Triggers-dbus.drv' failed to build
error: 1 dependencies of derivation '/nix/store/rs53shdkrczal4davyz7f4jyqxil7635-etc-fish-nixos-env-preinit.fish.drv' failed to build
error: 1 dependencies of derivation '/nix/store/vdz9vclfcgmlbq69f6qja8ylw893kmm0-etc-man_db.conf.drv' failed to build
error: 1 dependencies of derivation '/nix/store/23hj6a6i4xgpn2wrx7jf77kjs0h1j88g-etc-profile.drv' failed to build
error: 1 dependencies of derivation '/nix/store/rizzdzrw8qczlcq34h081kqxlkf9x71h-unit-polkit.service.drv' failed to build
error: 1 dependencies of derivation '/nix/store/sj7cv7hhrc3smnb0mp0ah4vvg1d0mmcm-abstractions-bash.drv' failed to build
error: 1 dependencies of derivation '/nix/store/5cgvqh6ahv3mp5lcmd1glcspjhpswp6i-unit-dbus.service.drv' failed to build
error: 1 dependencies of derivation '/nix/store/jrkwszr2ahw0d9x3nnk8vls05lja86dr-unit-dbus.service.drv' failed to build
error: 1 dependencies of derivation '/nix/store/2mybvynwdrhjv94didl14153yvrjrih0-apparmor.d.drv' failed to build
error: 2 dependencies of derivation '/nix/store/y8s0f63mgc2ix3y5pha1b03p684c3wnv-user-units.drv' failed to build
error: 1 dependencies of derivation '/nix/store/4y7n9ddvlv119074zsdqb7vk4qg7qrkv-X-Restart-Triggers-apparmor.drv' failed to build
error: 1 dependencies of derivation '/nix/store/pnrpy5rd11vahlb1r07wnzzfrkr6gs9w-unit-apparmor.service.drv' failed to build
error: 6 dependencies of derivation '/nix/store/sf9wflwbdib585fclxsydplbgqgw90iy-system-units.drv' failed to build
error: 14 dependencies of derivation '/nix/store/n2l1s7digfacbjn0kak0v6qx65g3ah80-etc.drv' failed to build
error: 2 dependencies of derivation '/nix/store/ccr6gx9aa2wivhfdfmjispybir8q02al-nixos-system-fabian-desktop-25.11.20250602.f5096bb.drv' failed to build

@dtomvan
Copy link
Contributor

dtomvan commented Jun 17, 2025

Following patch seems to fix pkgs-lib-tests (https://hydra.nixos.org/build/300150957/nixlog/2):

Details
diff --git a/pkgs/pkgs-lib/formats.nix b/pkgs/pkgs-lib/formats.nix
index 0836519c20..fd80205617 100644
--- a/pkgs/pkgs-lib/formats.nix
+++ b/pkgs/pkgs-lib/formats.nix
@@ -504,10 +504,10 @@
       generate =
         name: value:
         pkgs.callPackage (
-          { runCommand, remarshal }:
+          { runCommand, remarshal_0_17 }:
           runCommand name
             {
-              nativeBuildInputs = [ remarshal ];
+              nativeBuildInputs = [ remarshal_0_17 ];
               value = builtins.toJSON value;
               passAsFile = [ "value" ];
               preferLocalBuild = true;

Tested with nom-build pkgs/top-level/release.nix -A pkgs-lib-tests.

Edit: another failure https://hydra.nixos.org/build/300239075/nixlog/3

@mweinelt
Copy link
Member

Then likely a regression through f80b06e

@mweinelt
Copy link
Member

I think we should just adapt the test output and move on. A newline here doesn't matter for us. cc @Stunkymonkey @h7x4

@vcunat
Copy link
Member Author

vcunat commented Jun 17, 2025

pkgs-lib-tests fixed in 26650e0

@vcunat vcunat merged commit a03f275 into master Jun 18, 2025
21 checks passed
@ofalvai ofalvai mentioned this pull request Jun 18, 2025
13 tasks
@qbisi
Copy link
Contributor

qbisi commented Jun 18, 2025

legacyPackages.aarch64-darwin.python312Packages.adios2 failed after this pr been merged to master.
The pythonImports check failed.

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "<string>", line 1, in <lambda>
  File "/nix/store/q54d9qqlb27g2nf9ja4zsa89gl7b3yvm-python3-3.12.11/lib/python3.12/importlib/__init__.py", line 90, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 999, in exec_module
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
  File "/nix/store/ji7b3kxqivzqkgk5vl6m16nplyvv4ry2-adios2-2.10.2/lib/python3.12/site-packages/adios2/__init__.py", line 7, in <module>
    import adios2.bindings
  File "/nix/store/ji7b3kxqivzqkgk5vl6m16nplyvv4ry2-adios2-2.10.2/lib/python3.12/site-packages/adios2/bindings/__init__.py", line 1, in <module>
    from .adios2_bindings import *
ImportError: dlopen(/nix/store/ji7b3kxqivzqkgk5vl6m16nplyvv4ry2-adios2-2.10.2/lib/python3.12/site-packages/adios2/bindings/adios2_bindings.cpython-312-darwin.so, 0x0002): symbol not found in flat namespace '_PyDict_GetItemStringRef'

@PerchunPak
Copy link
Member

@qbisi please open an issue, your message will be lost in this dead thread

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

1.severity: security Issues which raise a security issue, or PRs that fix one 4.workflow: staging A staging-next or staging-next-XX.YY branch 10.rebuild-darwin: 501+ This PR causes many rebuilds on Darwin and should normally target the staging branches. 10.rebuild-darwin: 5001+ This PR causes many rebuilds on Darwin and must target the staging branches. 10.rebuild-linux: 501+ This PR causes many rebuilds on Linux and should normally target the staging branches. 10.rebuild-linux: 5001+ This PR causes many rebuilds on Linux and must target the staging branches.

Projects

None yet

Development

Successfully merging this pull request may close these issues.