postgresql: Remove dev dependencies from closure by jtojnar · Pull Request #179962 · NixOS/nixpkgs

jtojnar · 2022-07-03T00:46:22Z

Description of changes

Reduces size of out: 327.3M → 243.6M

Things done

jtojnar · 2022-07-03T01:20:28Z

pkgs/servers/sql/postgresql/patches/prevent-output-cycle.patch

We could shave 16M more by removing man and doc output references.

pkgs/servers/sql/postgresql/default.nix

pkgs/servers/sql/postgresql/patches/prevent-output-cycle.patch

pkgs/servers/sql/postgresql/default.nix

SuperSandro2000

or are the files copied into an output?

pkgs/servers/sql/postgresql/patches/hardcode-pg_config-paths.patch

jtojnar · 2022-07-12T16:15:20Z

@ofborg build postgresql_10 postgresql_11 postgresql_12 postgresql_13 postgresql_14

jtojnar · 2022-07-12T20:20:02Z

@ofborg build postgresql_10 postgresql_11 postgresql_12 postgresql_13 postgresql_14

pkgs/data/sgml+xml/schemas/sgml-dtd/docbook/4.2.nix

pkgs/data/sgml+xml/schemas/sgml-dtd/docbook/common.nix

pkgs/servers/sql/postgresql/default.nix

pkgs/data/sgml+xml/schemas/sgml-dtd/docbook/common.nix

pkgs/data/sgml+xml/schemas/sgml-dtd/docbook/4.2.nix

`pg_config` stores configure flags into the library/program for debugging or something. Since `PKG_CONFIG_PATH` somehow ends up in there, `dev` outputs of dependencies get pulled into the runtime closure. Let’s clear the paths in the variable to reduce the closure size. Reduces size of out: 287.8M → 236.5M

No closure reduction this time since there are still more references.

jtojnar · 2023-12-08T18:58:57Z

Looks like the current reduction of postgresql.out (18%: 287.8M → 236.5M → 235.5M) is not as pronounced as the last one (26%: 327.3M → 292.6M → 243.4M).

So unless there were some regressions, the commit splitting the dev output may not be worth the added complexity now.

Ma27 · 2023-12-09T16:04:45Z

pkgs/servers/sql/postgresql/patches/hardcode-pg_config-paths.patch

What's the rationale in here to make this /dev/null first and then replace that with @dev@?

IIRC pg_config.c is installed to dev output but config_info.c ends up as a part of out.

Ma27 · 2023-12-09T16:06:40Z

pkgs/servers/sql/postgresql/patches/hardcode-pg_config-paths.patch

Also why is this passed by reference here?

It is passed by value, is not it? We could pass a pointer to it if we want to avoid copying the whole struct but I do not think this is worth it – the functions will probably only be called at extension build time and compiler might inline the function anyway.

meh, I mixed them up, I meant pass-by-value and wanted to ask why 🙃

Ma27 · 2023-12-09T16:14:27Z

pkgs/servers/sql/postgresql/patches/prevent-output-cycle.patch

Doesn't this break with e.g. https://github.com/postgres/postgres/blob/aa210e0c121eb8f58c86d4fcc833a5a6fbb6f5a9/src/interfaces/ecpg/preproc/ecpg.c#L256 since get_include_path uses INCLUDEDIR generated by this makefile?

It will return the wrong value but the assumption is that nothing uses the function for anything other than printing it in some build information pages.

Ma27 · 2023-12-09T16:15:43Z

pkgs/servers/sql/postgresql/default.nix

I'm probably missing something, but how are .so files moved to the lib output now?

We previously passed setOutputFlags = false; so we had to set it explicitly. Now that I removed it, multiple-outputs hook passes this for us.

Ma27 · 2023-12-09T16:16:47Z

pkgs/development/python-modules/psycopg/default.nix

Can you elaborate why this is now needed?

This is just a proper place for this as a library dependency. Ideally, we would remove it from nativeBuildInputs but that will probably not work until psycopg/psycopg2#1001. cc @SuperSandro2000 who made this change.

Yes, IIRC they use pg_config

Added a comment.

pkgs/servers/sql/postgresql/default.nix

Ma27 · 2023-12-09T16:22:03Z

pkgs/servers/sql/postgresql/patches/remove-refs-from-configure-flags.patch

What's happening to e.g. tzdata which is also part of the configure flags and a store path? This means that every single reference in configureFlags silently gets discarded, correct?

Yes. But these should just be for informational reports.

SuperSandro2000 · 2023-12-09T23:25:12Z

So unless there were some regressions, the commit splitting the dev output may not be worth the added complexity now.

Now that we are done with it, we should keep it, to keep out dev things from the final system.

jtojnar · 2023-12-10T00:13:57Z

Weirdly, python310Packages.psycopg tests fail non-deterministically (first I thought the dev commit was responsible but I have since managed to build it without errors so I am not sure):

_______________________ test_leak[asyncio-dict_row-iter] _______________________

aconn_cls = <class 'psycopg.AsyncConnection'>
dsn = 'host=/build/run/postgresql user=psycopg'
faker = <tests.fix_faker.Faker object at 0x7fffb096a680>, fetch = 'iter'
row_factory = <function dict_row at 0x7fffb34784c0>

    @pytest.mark.slow
    @pytest.mark.parametrize("fetch", ["one", "many", "all", "iter"])
    @pytest.mark.parametrize("row_factory", ["tuple_row", "dict_row", "namedtuple_row"])
    async def test_leak(aconn_cls, dsn, faker, fetch, row_factory):
        faker.choose_schema(ncols=5)
        faker.make_records(10)
        row_factory = getattr(rows, row_factory)
    
        async def work():
            async with await aconn_cls.connect(dsn) as conn, conn.transaction(
                force_rollback=True
            ):
                async with psycopg.AsyncClientCursor(conn, row_factory=row_factory) as cur:
                    await cur.execute(faker.drop_stmt)
                    await cur.execute(faker.create_stmt)
                    async with faker.find_insert_problem_async(conn):
                        await cur.executemany(faker.insert_stmt, faker.records)
                    await cur.execute(faker.select_stmt)
    
                    if fetch == "one":
                        while True:
                            tmp = await cur.fetchone()
                            if tmp is None:
                                break
                    elif fetch == "many":
                        while True:
                            tmp = await cur.fetchmany(3)
                            if not tmp:
                                break
                    elif fetch == "all":
                        await cur.fetchall()
                    elif fetch == "iter":
                        async for rec in cur:
                            pass
    
        n = []
        gc_collect()
        for i in range(3):
            await work()
            gc_collect()
            n.append(gc_count())
    
>       assert n[0] == n[1] == n[2], f"objects leaked: {n[1] - n[0]}, {n[2] - n[1]}"
E       AssertionError: objects leaked: 1, -1
E       assert 216420 == 216421

tests/test_client_cursor_async.py:680: AssertionError
=========================== short test summary info ============================
FAILED tests/test_copy_async.py::test_copy_to_leaks[asyncio-rows-Format.BINARY-True] - AssertionError: objects leaked: 1, -1
FAILED tests/test_copy_async.py::test_copy_to_leaks[asyncio-rows-Format.TEXT-False] - AssertionError: objects leaked: -1, 0
FAILED tests/test_copy_async.py::test_copy_from_leaks[asyncio-Format.TEXT-False] - AssertionError: objects leaked: -1, 0
FAILED tests/test_copy_async.py::test_copy_to_leaks[asyncio-iter-Format.BINARY-True] - AssertionError: objects leaked: -1, 0
FAILED tests/test_copy_async.py::test_copy_to_leaks[asyncio-read-Format.TEXT-False] - AssertionError: objects leaked: 1, -1
FAILED tests/test_copy_async.py::test_copy_to_leaks[asyncio-read-Format.BINARY-True] - AssertionError: objects leaked: 0, 1
FAILED tests/test_copy_async.py::test_copy_to_leaks[asyncio-rows-Format.TEXT-True] - AssertionError: objects leaked: 0, 1
FAILED tests/test_copy_async.py::test_copy_to_leaks[asyncio-row-Format.TEXT-False] - AssertionError: objects leaked: 1, -1
FAILED tests/test_copy_async.py::test_copy_to_leaks[asyncio-read-Format.TEXT-True] - AssertionError: objects leaked: -1, 0
FAILED tests/test_copy_async.py::test_copy_to_leaks[asyncio-iter-Format.TEXT-False] - AssertionError: objects leaked: 0, 1
FAILED tests/test_copy_async.py::test_copy_to_leaks[asyncio-row-Format.BINARY-True] - AssertionError: objects leaked: 0, 1
FAILED tests/test_copy_async.py::test_copy_to_leaks[asyncio-row-Format.TEXT-True] - AssertionError: objects leaked: 0, 1
FAILED tests/test_cursor_async.py::test_leak[asyncio-tuple_row-iter-Format.BINARY-b] - AssertionError: objects leaked: -1, 0
FAILED tests/test_cursor_async.py::test_leak[asyncio-dict_row-iter-Format.BINARY-b] - AssertionError: objects leaked: 0, 1
FAILED tests/test_cursor_async.py::test_leak[asyncio-dict_row-iter-Format.TEXT-b] - AssertionError: objects leaked: 1, -1
FAILED tests/test_cursor_async.py::test_leak[asyncio-tuple_row-all-Format.TEXT-t] - AssertionError: objects leaked: 0, 1
FAILED tests/test_cursor_async.py::test_leak[asyncio-dict_row-iter-Format.BINARY-s] - AssertionError: objects leaked: 0, 1
FAILED tests/test_cursor_async.py::test_leak[asyncio-namedtuple_row-many-Format.TEXT-b] - AssertionError: objects leaked: 1, -1
FAILED tests/test_cursor_async.py::test_leak[asyncio-tuple_row-many-Format.BINARY-s] - AssertionError: objects leaked: 0, 1
FAILED tests/test_cursor_async.py::test_leak[asyncio-tuple_row-one-Format.TEXT-b] - AssertionError: objects leaked: 1, -1
FAILED tests/test_cursor_async.py::test_leak[asyncio-namedtuple_row-many-Format.BINARY-b] - AssertionError: objects leaked: 0, 1
FAILED tests/test_cursor_async.py::test_leak[asyncio-tuple_row-iter-Format.BINARY-s] - AssertionError: objects leaked: 0, 1
FAILED tests/test_cursor_async.py::test_leak[asyncio-tuple_row-iter-Format.TEXT-b] - AssertionError: objects leaked: -1, 0
FAILED tests/test_cursor_async.py::test_leak[asyncio-namedtuple_row-all-Format.TEXT-t] - AssertionError: objects leaked: 1, -1
FAILED tests/test_cursor_async.py::test_leak[asyncio-dict_row-iter-Format.TEXT-s] - AssertionError: objects leaked: 1, -1
FAILED tests/test_cursor_async.py::test_leak[asyncio-dict_row-one-Format.TEXT-s] - AssertionError: objects leaked: 0, 1
FAILED tests/test_cursor_async.py::test_leak[asyncio-tuple_row-all-Format.TEXT-s] - AssertionError: objects leaked: -1, 0
FAILED tests/test_cursor_async.py::test_leak[asyncio-tuple_row-all-Format.TEXT-b] - AssertionError: objects leaked: 1, -1
FAILED tests/test_cursor_async.py::test_leak[asyncio-tuple_row-iter-Format.TEXT-s] - AssertionError: objects leaked: 0, 1
FAILED tests/test_cursor_async.py::test_leak[asyncio-namedtuple_row-many-Format.TEXT-s] - AssertionError: objects leaked: -1, 0
FAILED tests/test_cursor_async.py::test_leak[asyncio-tuple_row-all-Format.BINARY-s] - AssertionError: objects leaked: 0, 1
FAILED tests/test_cursor_async.py::test_leak[asyncio-namedtuple_row-one-Format.TEXT-s] - AssertionError: objects leaked: 0, 1
FAILED tests/test_cursor_async.py::test_leak[asyncio-dict_row-iter-Format.BINARY-t] - AssertionError: objects leaked: -1, 0
FAILED tests/test_cursor_async.py::test_leak[asyncio-tuple_row-iter-Format.BINARY-t] - AssertionError: objects leaked: -1, 0
FAILED tests/test_cursor_async.py::test_leak[asyncio-tuple_row-all-Format.BINARY-b] - AssertionError: objects leaked: -1, 0
FAILED tests/test_cursor_async.py::test_leak[asyncio-namedtuple_row-all-Format.TEXT-s] - AssertionError: objects leaked: 1, -1
FAILED tests/test_cursor_async.py::test_leak[asyncio-dict_row-many-Format.TEXT-s] - AssertionError: objects leaked: 1, -1
FAILED tests/test_cursor_async.py::test_leak[asyncio-tuple_row-all-Format.BINARY-t] - AssertionError: objects leaked: 1, -1
FAILED tests/test_cursor_async.py::test_leak[asyncio-tuple_row-one-Format.BINARY-b] - AssertionError: objects leaked: 1, -1
FAILED tests/test_cursor_async.py::test_leak[asyncio-dict_row-iter-Format.TEXT-t] - AssertionError: objects leaked: 0, 1
FAILED tests/test_cursor_async.py::test_leak[asyncio-dict_row-all-Format.TEXT-t] - AssertionError: objects leaked: -1, 0
FAILED tests/test_cursor_async.py::test_leak[asyncio-tuple_row-many-Format.TEXT-s] - AssertionError: objects leaked: 1, -1
FAILED tests/test_cursor_async.py::test_leak[asyncio-dict_row-all-Format.TEXT-b] - AssertionError: objects leaked: 1, -1
FAILED tests/test_cursor_async.py::test_leak[asyncio-dict_row-all-Format.BINARY-s] - AssertionError: objects leaked: 1, -1
FAILED tests/test_cursor_async.py::test_leak[asyncio-dict_row-all-Format.BINARY-b] - AssertionError: objects leaked: -1, 0
FAILED tests/test_cursor_async.py::test_leak[asyncio-namedtuple_row-one-Format.BINARY-b] - AssertionError: objects leaked: 0, 1
FAILED tests/test_cursor_async.py::test_leak[asyncio-tuple_row-many-Format.TEXT-t] - AssertionError: objects leaked: -1, 0
FAILED tests/test_cursor_async.py::test_leak[asyncio-tuple_row-iter-Format.TEXT-t] - AssertionError: objects leaked: 1, -1
FAILED tests/test_cursor_async.py::test_leak[asyncio-dict_row-all-Format.BINARY-t] - AssertionError: objects leaked: -1, 0
FAILED tests/test_cursor_async.py::test_leak[asyncio-dict_row-one-Format.TEXT-t] - AssertionError: objects leaked: 1, -1
FAILED tests/test_cursor_async.py::test_leak[asyncio-tuple_row-many-Format.BINARY-b] - AssertionError: objects leaked: -1, 0
FAILED tests/test_cursor_async.py::test_leak[asyncio-namedtuple_row-many-Format.BINARY-t] - AssertionError: objects leaked: -1, 0
FAILED tests/test_cursor_async.py::test_leak[asyncio-dict_row-one-Format.BINARY-b] - AssertionError: objects leaked: 1, -1
FAILED tests/test_cursor_async.py::test_leak[asyncio-namedtuple_row-one-Format.TEXT-b] - AssertionError: objects leaked: -1, 0
FAILED tests/test_cursor_async.py::test_leak[asyncio-namedtuple_row-iter-Format.BINARY-s] - AssertionError: objects leaked: 1, -1
FAILED tests/test_cursor_async.py::test_leak[asyncio-namedtuple_row-iter-Format.TEXT-b] - AssertionError: objects leaked: 1, -1
FAILED tests/test_cursor_async.py::test_leak[asyncio-namedtuple_row-iter-Format.BINARY-t] - AssertionError: objects leaked: 0, 1
FAILED tests/test_cursor_async.py::test_leak[asyncio-tuple_row-one-Format.BINARY-t] - AssertionError: objects leaked: 1, -1
FAILED tests/test_cursor_async.py::test_leak[asyncio-namedtuple_row-one-Format.TEXT-t] - AssertionError: objects leaked: 0, 1
FAILED tests/test_cursor_async.py::test_leak[asyncio-namedtuple_row-all-Format.BINARY-s] - AssertionError: objects leaked: -1, 0
FAILED tests/test_cursor_async.py::test_leak[asyncio-namedtuple_row-one-Format.BINARY-s] - AssertionError: objects leaked: 1, -1
FAILED tests/test_cursor_async.py::test_leak[asyncio-dict_row-many-Format.TEXT-b] - AssertionError: objects leaked: 1, -1
FAILED tests/test_cursor_async.py::test_leak[asyncio-namedtuple_row-many-Format.BINARY-s] - AssertionError: objects leaked: 0, 1
FAILED tests/test_cursor_async.py::test_leak[asyncio-namedtuple_row-iter-Format.TEXT-t] - AssertionError: objects leaked: -1, 0
FAILED tests/test_cursor_async.py::test_leak[asyncio-dict_row-many-Format.BINARY-t] - AssertionError: objects leaked: -1, 0
FAILED tests/test_cursor_async.py::test_leak[asyncio-dict_row-one-Format.BINARY-t] - AssertionError: objects leaked: 1, -1
FAILED tests/test_client_cursor_async.py::test_leak[asyncio-dict_row-many] - AssertionError: objects leaked: -1, 0
FAILED tests/test_client_cursor_async.py::test_leak[asyncio-namedtuple_row-many] - AssertionError: objects leaked: -1, 1
FAILED tests/test_client_cursor_async.py::test_leak[asyncio-namedtuple_row-one] - AssertionError: objects leaked: -1, 0
FAILED tests/test_client_cursor_async.py::test_leak[asyncio-tuple_row-one] - AssertionError: objects leaked: -1, 0
FAILED tests/test_client_cursor_async.py::test_leak[asyncio-namedtuple_row-all] - AssertionError: objects leaked: 0, 1
FAILED tests/test_client_cursor_async.py::test_leak[asyncio-namedtuple_row-iter] - AssertionError: objects leaked: 0, -1
FAILED tests/test_client_cursor_async.py::test_leak[asyncio-tuple_row-all] - AssertionError: objects leaked: 0, 1
FAILED tests/test_client_cursor_async.py::test_leak[asyncio-tuple_row-iter] - AssertionError: objects leaked: -1, 0
FAILED tests/test_client_cursor_async.py::test_leak[asyncio-tuple_row-many] - AssertionError: objects leaked: -1, 0
FAILED tests/test_client_cursor_async.py::test_leak[asyncio-dict_row-iter] - AssertionError: objects leaked: 1, -1
= 76 failed, 4393 passed, 148 skipped, 78 deselected, 29 xfailed in 589.11s (0:09:49) =

SuperSandro2000 · 2023-12-10T02:04:19Z

I think we can just ignore those. There are many other tests that pass and from the description they read like they can't be that stable.

The package contains some files only necessary for linking other software against PgSQL’s libraries. Those files include compiler flags that reference `dev` outputs of other libraries, unnecessarily increasing the runtime closure. Let’s move those files to a `dev` output, reducing the runtime closure of `out`. We also need to clear out some paths hardcoded into the libs to avoid a dependency cycle between the `dev` and `out` outputs. This includes the path to PGXS files for the `libpgcommon` and `postgres` server (used e.g. by `SELECT pg_config()` query); we restore the path explicitly for `pg_config` program. The `out` output will be the root of the graph. This further reduces the closure size of out from 236.5M to 235.5M.

wolfgangwalther · 2024-08-24T12:08:58Z

Superseded by #294504

This was supposed to happen in NixOS#294504, but the commit was accidentally left out when splitting off some libpq-related changes. Originated in NixOS#179962, by Sandro. Co-authored-by: Sandro Jäckel <[email protected]> Co-authored-by: Wolfgang Walther <[email protected]>

ofborg bot requested review from danbst, globin, ivan, marsam and thoughtpolice July 3, 2022 00:54

jtojnar changed the base branch from master to staging July 3, 2022 01:07

jtojnar marked this pull request as ready for review July 3, 2022 01:08

jtojnar commented Jul 3, 2022

View reviewed changes

pkgs/servers/sql/postgresql/default.nix Outdated Show resolved Hide resolved

jtojnar commented Jul 3, 2022

View reviewed changes

pkgs/servers/sql/postgresql/patches/prevent-output-cycle.patch Outdated Show resolved Hide resolved

jtojnar marked this pull request as draft July 3, 2022 01:22

jtojnar force-pushed the postgres-closure-size branch from 79cf3ff to 516faaf Compare July 3, 2022 14:34

jtojnar marked this pull request as ready for review July 3, 2022 14:36

ajs124 reviewed Jul 12, 2022

View reviewed changes

pkgs/servers/sql/postgresql/default.nix Outdated Show resolved Hide resolved

SuperSandro2000 reviewed Jul 12, 2022

View reviewed changes

jtojnar force-pushed the postgres-closure-size branch from 516faaf to 4461a4d Compare July 12, 2022 16:12

jtojnar force-pushed the postgres-closure-size branch from 4461a4d to 67b91ed Compare July 12, 2022 19:55

jtojnar force-pushed the postgres-closure-size branch from 67b91ed to cbca884 Compare July 12, 2022 22:04

ofborg bot added the 8.has: package (new) This PR adds a new package label Jul 12, 2022

SuperSandro2000 reviewed Jul 13, 2022

View reviewed changes

jtojnar force-pushed the postgres-closure-size branch 2 times, most recently from 2894ece to beaa49a Compare August 18, 2022 22:52

SuperSandro2000 reviewed Aug 19, 2022

View reviewed changes

pkgs/data/sgml+xml/schemas/sgml-dtd/docbook/common.nix Outdated Show resolved Hide resolved

pkgs/servers/sql/postgresql/default.nix Outdated Show resolved Hide resolved

pkgs/data/sgml+xml/schemas/sgml-dtd/docbook/common.nix Outdated Show resolved Hide resolved

SuperSandro2000 approved these changes Aug 19, 2022

View reviewed changes

pkgs/data/sgml+xml/schemas/sgml-dtd/docbook/common.nix Outdated Show resolved Hide resolved

pkgs/data/sgml+xml/schemas/sgml-dtd/docbook/common.nix Outdated Show resolved Hide resolved

pkgs/data/sgml+xml/schemas/sgml-dtd/docbook/4.2.nix Outdated Show resolved Hide resolved

jtojnar added 2 commits December 8, 2023 19:57

postgresql: Remove more dev output references from closure

cc2469c

No closure reduction this time since there are still more references.

jtojnar force-pushed the postgres-closure-size branch from d07037b to 7ab57c1 Compare December 8, 2023 18:59

Ma27 reviewed Dec 9, 2023

View reviewed changes

jtojnar marked this pull request as draft December 9, 2023 21:47

github-actions bot added 8.has: documentation This PR adds or changes documentation 6.topic: ruby A dynamic, open source programming language with a focus on simplicity and productivity. labels Dec 9, 2023

jtojnar marked this pull request as ready for review December 10, 2023 10:22

jtojnar requested a review from mweinelt as a code owner December 10, 2023 10:22

jtojnar and others added 6 commits December 10, 2023 13:37

treewide: prepare for pg_config moving to dev

c271f39

python310Packages.psycopg: prepare for pg_config moving to dev

3333c69

python310Packages.psycopg2cffi: prepare for pg_config moving to dev

716f239

php82Packages.pdo_psql: prepare for pg_config moving to dev

2cd0977

php82Packages.psql: prepare for pg_config moving to dev

dff2bb7

jtojnar force-pushed the postgres-closure-size branch from 57c494f to 8f21582 Compare December 10, 2023 12:58

wolfgangwalther mentioned this pull request Mar 9, 2024

postgresql: split -lib and -dev outputs cleanly #294504

Merged

13 tasks

wegank added the 2.status: merge conflict This PR has merge conflicts with the target branch label Mar 20, 2024

wegank added the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Jul 4, 2024

wolfgangwalther mentioned this pull request Aug 24, 2024

treewide: fix pg_config / postgresql headers moved to dev output #336994

Merged

15 tasks

wolfgangwalther closed this Aug 24, 2024

Uh oh!

Conversation

jtojnar commented Jul 3, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description of changes

Things done

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

SuperSandro2000 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jtojnar commented Jul 12, 2022

Uh oh!

jtojnar commented Jul 12, 2022

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jtojnar commented Dec 8, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

SuperSandro2000 commented Dec 9, 2023

Uh oh!

jtojnar commented Dec 10, 2023

Uh oh!

SuperSandro2000 commented Dec 10, 2023

Uh oh!

wolfgangwalther commented Aug 24, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

jtojnar commented Jul 3, 2022 •

edited

Loading

jtojnar commented Dec 8, 2023 •

edited

Loading