postgresql: fix regress tests after tzdata update by wolfgangwalther · Pull Request #342026 · NixOS/nixpkgs

wolfgangwalther · 2024-09-15T09:41:42Z

Description of changes

postgresql is currently broken on staging, after the tzdata update (#339913) due to failing regression tests.

This was discussed upstream here:
https://postgr.es/m/0a997455-5aba-4cf2-a354-d26d8bcbfae6%40technowledgy.de

This will be in the next minor releases, until then apply the patch. Applying the patch from github via fetchurl is not possible, because the upstream patch includes some doc changes, which cause our build to fail. We could fix that by adding new dependencies to rebuild the docs... but it's easier to just keep the patch in nixpkgs for 1-2 months.

TLDR: The patches have been slightly modified from upstream, removing the docs changes.

@Ma27

Things done

Add a 👍 reaction to pull requests you find important.

Ma27 · 2024-09-15T10:02:43Z

Thanks!
Will try to review/merge both of your PRs today :)

alyssais · 2024-09-15T14:01:48Z

Applying the patch from github via fetchurl is not possible, because the upstream patch includes some doc changes, which cause our build to fail.

We can't use fetchpatch's exclude argument?

This was discussed upstream here: https://postgr.es/m/0a997455-5aba-4cf2-a354-d26d8bcbfae6%40technowledgy.de This will be in the next minor releases, until then apply the patch.

wolfgangwalther · 2024-09-15T15:10:53Z

We can't use fetchpatch's exclude argument?

TIL. Much better, thanks.

This was discussed upstream here: https://postgr.es/m/0a997455-5aba-4cf2-a354-d26d8bcbfae6%40technowledgy.de This will be in the next minor releases, until then apply the patch. (cherry picked from commit 52bb840) (from PR #342026)

This was discussed upstream here: https://postgr.es/m/0a997455-5aba-4cf2-a354-d26d8bcbfae6%40technowledgy.de Also applied by nixpkgs here: NixOS/nixpkgs#342026

* chore: fix missing required `teams` meta attribute * chore: update old (unsupported) version of nix used in script * fix: overlay and use our `buildPgrxExtension` function These overlay packages will be removed once the extensions are updated to use the new `buildPgrxExtension` function. * chore: fix formatting * chore: replace (deprecated) `substituteAll` by `replaceVars` * chore: update (and refactor) flake inputs * chore: bump `pgrouting` version to 3.8.0 The previously used version was failing to build: /build/source/src/common/get_check_data.c: In function 'pgr_SPI_getText': /build/source/src/common/get_check_data.c:307:28: error: passing argument 1 of 'DatumGetCString' makes integer from pointer without a cast [-Wint-conversion] 307 | return DatumGetCString(SPI_getvalue(*tuple, *tupdesc, info.colNumber)); | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | | | char * In file included from /build/source/include/c_common/postgres_connection.h:29, from /build/source/src/common/get_check_data.c:27: /nix/store/nn6vpjqlsxxmynwcrsib9agb3hpr5cqj-postgresql-17.4/include/server/postgres.h:335:23: note: expected 'Datum' {aka 'long unsigned int'} but argument is of type 'char *' 335 | DatumGetCString(Datum X) | ~~~~~~^ make[2]: *** [src/common/CMakeFiles/common.dir/build.make:149: src/common/CMakeFiles/common.dir/get_check_data.c.o] Error 1 * fix: backport tzdata fixes for postgresql 15 This was discussed upstream here: https://postgr.es/m/0a997455-5aba-4cf2-a354-d26d8bcbfae6%40technowledgy.de Also applied by nixpkgs here: NixOS/nixpkgs#342026 * fix: plv8 3.1 requires older v8 We cannot use nodejs.libv8 as it is too new for plv8 3.1. We will be able to use it with plv8 3.2. * chore: update fmt * chore: update cargo-pgrx 1.14.3 cargo hash * chore(postgresql): add isOrioleDB condition * fix: pgregress tests * fix: update schema for dbmate schema_migrations table dbmate 2.27 is now using a varchar for the version column in the schema_migrations table. See amacneil/dbmate#641 * Structured attributes are now required when building with separateDebugInfo See NixOS/nixpkgs#394674 * chore: version bump * fix: rebase on develop and fix code api changes from nixpkgs * tests: try to get passing tests * feat: pin icu75 for postgres build to maintain compat with existing deployments * chore: remove commented out code not needed in latest changes to pg * fix: apparently incorrectly rebased changes, and % nix fmt * chore: remove nixpkgs-pgbackrest input, and % nix flake update * fix: replace deprecated v8 package, and wrong packages arguments * chore: fix formatting * fix(darwin): remove reference to apple-sdk * fix(ext): missing required attributes in some extension tests * chore: % nix flake update && nix fmt Updated nixpkgs, rust-overlay, and nix-fast-build to latest versions. * fix(orioledb): fixed installedExtensions reference to use proper self.packages * fix(postgresql): remove redundant patch substitutions Removed duplicate substituteInPlace calls that were redundantly applied during previous merge operations. Cleaned up the patch application to avoid duplicate substitutions. * fix(pgrx): remove deprecated darwin.apple_sdk.frameworks.Security * fix(postgis): remove custom sfcgal package dependency Updated postgis.nix to use nixpkgs sfcgal instead of our custom package, eliminating the need to maintain a separate sfcgal build definition. * fix(pg_net): pin curl to older nixpkgs version pg_net requires an older version of curl (pinned from nixpkgs commit a76c4553d7e741e17f289224eda135423de0491d) to resolve build compatibility issues. * fix(pgrx): error: multiple input filenames provided Updated cargo-pgrx buildPgrxExtension with rustc wrapper to handle empty postmaster_stub.rs files that cause duplicate symbol errors. (first two filenames are `` and `/build/tmp.*/postmaster_stubs/nix/store/*/bin/postmaster_stub.rs`) pg_jsonschema cargo checks fails on specific versions (0.2.0, 0.3.1, 0.3.3). * fix(tests): updated test expectations Remove deprecated functions that were removed in newer pgrouting versions (_pgr_betweennesscentrality and _pgr_contractionhierarchies). * feat: (re)add check_pg_regress method to PostgresExtensionTest * refactor: replace fetchTarball with flake input for nixpkgs-oldstable Improves reproducibility and avoids evaluation-time network access by using a proper flake input instead of fetchTarball for the older nixpkgs needed by pg_net and plv8. * fix(pg_net): disable dangling-assignment warning for v0.19.5 on aarch64-darwin Version 0.19.5 has a dangling pointer issue in src/core.c:177 that causes compilation to fail on aarch64-darwin with newer clang versions. This adds -Wno-error=dangling-assignment to allow compilation to proceed. The flag is restricted to aarch64-darwin only since GCC on Linux doesn't recognize this warning option. * fix(plv8): add headerpad_max_install_names for macOS builds Add -headerpad_max_install_names linker flag to prevent 'install_name_tool: changing install names or rpaths can't be redone' errors on macOS. This allocates sufficient header space for install_name_tool to modify the shared library during post-install processing. * fix(tests): correct library naming and symlinks for background worker extensions Test assertions were inconsistent: expecting {version}.so and {extension_name}-{version}.so patterns while actual libraries use {lib_name}-{version}.so (e.g. pg_partman_bgw-5.3.1.so, timescaledb-loader-2.16.1.so). Updated PostgresExtensionTest to accept Optional[str] lib_name parameter and use consistent {lib_name}-{version}.so assertions throughout. Added missing timescaledb-loader.so symlink creation and updated the switch script to maintain both symlinks when switching versions. Fixes 'Expected timescaledb version 2.16.1, but found timescaledb-loader-2.16.1.so' and 'Expected pg_partman version 5.3.1, but found pg_partman_bgw-5.3.1.so' test failures. * refactor(exts): replace nixpkgs-oldstable with specific overlay packages Use overlay instead of passing full nixpkgs-oldstable to avoid exposing entire old nixpkgs attribute set. Now provides specific packages (curl_8_4, v8_oldstable) through overlay. * fix: restore revision support for orioledb builds (introduced it #1880) Keep revision support that was accidentally simplified during nixpkgs update. Restores conditional logic to support building orioledb from specific git revision hashes instead of only tags. * fix(exts): remove old files that were moved to subdirectories These files were moved to subdirectories and the old .nix files should be removed since they were moved to new directory structure with default.nix files. * fix(flake): remove nixpkgs-go124 and nix-fast-build dependencies Remove nixpkgs-go124 dependency since current nixpkgs has Go 1.25.4 which is sufficient for packer. Remove nix-fast-build input as unused and add follows directives to resolve duplicate dependencies. * fix(flake): update flake.lock and cargo-pgrx hashes Run flake-linter to eliminate duplicate dependencies, and remove unused cargo-pgrx / rustc versions hashes. * fix(exts): remove remaining duplicated files Move pgrouting.nix to pgrouting/default.nix and groonga package to pgroonga/groonga.nix. Update imports and fix relative paths for versions.json and patch files. * refactor(pgrx): conditionally apply rustc wrapper when < 0.12 Replace inline bash script generation with writeShellScriptBin for rustc wrapper that filters empty postmaster_stub.rs arguments. Apply wrapper only for pgrx < 0.12 since issue was fixed upstream in pgrx#1435 and pgrx#1441. * refactor(nixpkgs): use let binding for oldstable instead of overlay Move nixpkgs-oldstable import to a let binding to avoid importing it three times... * refactor(groonga): move groonga package to pgroonga extension Remove supabase-groonga from global flake packages and import it locally in pgroonga extension instead. Other components access groonga via pgroonga.passthru.groonga since groonga is only used by pgroonga. * fix(ansible): remove supabase-groonga from package install list The supabase-groonga package was moved to pgroonga extension in commit 1f0ed1f, but Ansible was still trying to install it separately causing build failures. * Revert "fix(ansible): remove supabase-groonga from package install list" This reverts commit 55474686405394826ad07a032cbcf0e36a924554. * Revert "refactor(groonga): move groonga package to pgroonga extension" This reverts commit cfa9bcb92219f391469ae743f95c8856b0a99561. * fix(darwin): remove deprecated apple_sdk frameworks from supascan Remove references to darwin.apple_sdk.frameworks.{IOKit,CoreFoundation} which have been deprecated in nixpkgs. Disable CGO to avoid Darwin framework dependencies entirely. * chore: bump version suffix to test * chore: bump to retest * chore: suffix to test * chore: bump to test * feat: update nixpkgs and fix plpgsql-check build with gcc 15 With this new nixpkgs update come gcc 15, which introduced stricter C standard compliance. plpgsql-check fails to build due to this. To fix the build, we explicitly set the C standard to gnu17. * fix: nix version update nix 2.29 doesn't exist in the updated nixpkgs anymore * fix: replace all deprecated system attribute usages Replace deprecated system attribute accesses throughout the codebase with stdenv.hostPlatform.system to resolve Nix deprecation warnings. Changes: - Replace inherit (stdenv) system with system = stdenv.hostPlatform.system - Replace pkgs.system with pkgs.stdenv.hostPlatform.system (55 occurrences) - Replace system function parameters with stdenv to avoid auto-injection - Replace final.system in overlay with final.stdenv.hostPlatform.system Affected files: - nix/postgresql/generic.nix - nix/packages/default.nix - nix/ext/tests/*.nix (16 files) - nix/packages/lib.nix - nix/packages/dbmate-tool.nix - nix/packages/show-commands.nix - nix/cargo-pgrx/mkPgrxExtension.nix - nix/overlays/default.nix * chore: bump versions for testing * chore: bump version to release --------- Co-authored-by: Yvan Sraka <[email protected]> Co-authored-by: Sam Rose <[email protected]> Co-authored-by: samrose <[email protected]>

* chore: fix missing required `teams` meta attribute * chore: update old (unsupported) version of nix used in script * fix: overlay and use our `buildPgrxExtension` function These overlay packages will be removed once the extensions are updated to use the new `buildPgrxExtension` function. * chore: fix formatting * chore: replace (deprecated) `substituteAll` by `replaceVars` * chore: update (and refactor) flake inputs * chore: bump `pgrouting` version to 3.8.0 The previously used version was failing to build: /build/source/src/common/get_check_data.c: In function 'pgr_SPI_getText': /build/source/src/common/get_check_data.c:307:28: error: passing argument 1 of 'DatumGetCString' makes integer from pointer without a cast [-Wint-conversion] 307 | return DatumGetCString(SPI_getvalue(*tuple, *tupdesc, info.colNumber)); | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | | | char * In file included from /build/source/include/c_common/postgres_connection.h:29, from /build/source/src/common/get_check_data.c:27: /nix/store/nn6vpjqlsxxmynwcrsib9agb3hpr5cqj-postgresql-17.4/include/server/postgres.h:335:23: note: expected 'Datum' {aka 'long unsigned int'} but argument is of type 'char *' 335 | DatumGetCString(Datum X) | ~~~~~~^ make[2]: *** [src/common/CMakeFiles/common.dir/build.make:149: src/common/CMakeFiles/common.dir/get_check_data.c.o] Error 1 * fix: backport tzdata fixes for postgresql 15 This was discussed upstream here: https://postgr.es/m/0a997455-5aba-4cf2-a354-d26d8bcbfae6%40technowledgy.de Also applied by nixpkgs here: NixOS/nixpkgs#342026 * fix: plv8 3.1 requires older v8 We cannot use nodejs.libv8 as it is too new for plv8 3.1. We will be able to use it with plv8 3.2. * chore: update fmt * chore: update cargo-pgrx 1.14.3 cargo hash * chore(postgresql): add isOrioleDB condition * fix: pgregress tests * fix: update schema for dbmate schema_migrations table dbmate 2.27 is now using a varchar for the version column in the schema_migrations table. See amacneil/dbmate#641 * Structured attributes are now required when building with separateDebugInfo See NixOS/nixpkgs#394674 * chore: version bump * fix: rebase on develop and fix code api changes from nixpkgs * tests: try to get passing tests * feat: pin icu75 for postgres build to maintain compat with existing deployments * chore: remove commented out code not needed in latest changes to pg * fix: apparently incorrectly rebased changes, and % nix fmt * chore: remove nixpkgs-pgbackrest input, and % nix flake update * fix: replace deprecated v8 package, and wrong packages arguments * chore: fix formatting * fix(darwin): remove reference to apple-sdk * fix(ext): missing required attributes in some extension tests * chore: % nix flake update && nix fmt Updated nixpkgs, rust-overlay, and nix-fast-build to latest versions. * fix(orioledb): fixed installedExtensions reference to use proper self.packages * fix(postgresql): remove redundant patch substitutions Removed duplicate substituteInPlace calls that were redundantly applied during previous merge operations. Cleaned up the patch application to avoid duplicate substitutions. * fix(pgrx): remove deprecated darwin.apple_sdk.frameworks.Security * fix(postgis): remove custom sfcgal package dependency Updated postgis.nix to use nixpkgs sfcgal instead of our custom package, eliminating the need to maintain a separate sfcgal build definition. * fix(pg_net): pin curl to older nixpkgs version pg_net requires an older version of curl (pinned from nixpkgs commit a76c4553d7e741e17f289224eda135423de0491d) to resolve build compatibility issues. * fix(pgrx): error: multiple input filenames provided Updated cargo-pgrx buildPgrxExtension with rustc wrapper to handle empty postmaster_stub.rs files that cause duplicate symbol errors. (first two filenames are `` and `/build/tmp.*/postmaster_stubs/nix/store/*/bin/postmaster_stub.rs`) pg_jsonschema cargo checks fails on specific versions (0.2.0, 0.3.1, 0.3.3). * fix(tests): updated test expectations Remove deprecated functions that were removed in newer pgrouting versions (_pgr_betweennesscentrality and _pgr_contractionhierarchies). * feat: (re)add check_pg_regress method to PostgresExtensionTest * refactor: replace fetchTarball with flake input for nixpkgs-oldstable Improves reproducibility and avoids evaluation-time network access by using a proper flake input instead of fetchTarball for the older nixpkgs needed by pg_net and plv8. * fix(pg_net): disable dangling-assignment warning for v0.19.5 on aarch64-darwin Version 0.19.5 has a dangling pointer issue in src/core.c:177 that causes compilation to fail on aarch64-darwin with newer clang versions. This adds -Wno-error=dangling-assignment to allow compilation to proceed. The flag is restricted to aarch64-darwin only since GCC on Linux doesn't recognize this warning option. * fix(plv8): add headerpad_max_install_names for macOS builds Add -headerpad_max_install_names linker flag to prevent 'install_name_tool: changing install names or rpaths can't be redone' errors on macOS. This allocates sufficient header space for install_name_tool to modify the shared library during post-install processing. * fix(tests): correct library naming and symlinks for background worker extensions Test assertions were inconsistent: expecting {version}.so and {extension_name}-{version}.so patterns while actual libraries use {lib_name}-{version}.so (e.g. pg_partman_bgw-5.3.1.so, timescaledb-loader-2.16.1.so). Updated PostgresExtensionTest to accept Optional[str] lib_name parameter and use consistent {lib_name}-{version}.so assertions throughout. Added missing timescaledb-loader.so symlink creation and updated the switch script to maintain both symlinks when switching versions. Fixes 'Expected timescaledb version 2.16.1, but found timescaledb-loader-2.16.1.so' and 'Expected pg_partman version 5.3.1, but found pg_partman_bgw-5.3.1.so' test failures. * refactor(exts): replace nixpkgs-oldstable with specific overlay packages Use overlay instead of passing full nixpkgs-oldstable to avoid exposing entire old nixpkgs attribute set. Now provides specific packages (curl_8_4, v8_oldstable) through overlay. * fix: restore revision support for orioledb builds (introduced it #1880) Keep revision support that was accidentally simplified during nixpkgs update. Restores conditional logic to support building orioledb from specific git revision hashes instead of only tags. * fix(exts): remove old files that were moved to subdirectories These files were moved to subdirectories and the old .nix files should be removed since they were moved to new directory structure with default.nix files. * fix(flake): remove nixpkgs-go124 and nix-fast-build dependencies Remove nixpkgs-go124 dependency since current nixpkgs has Go 1.25.4 which is sufficient for packer. Remove nix-fast-build input as unused and add follows directives to resolve duplicate dependencies. * fix(flake): update flake.lock and cargo-pgrx hashes Run flake-linter to eliminate duplicate dependencies, and remove unused cargo-pgrx / rustc versions hashes. * fix(exts): remove remaining duplicated files Move pgrouting.nix to pgrouting/default.nix and groonga package to pgroonga/groonga.nix. Update imports and fix relative paths for versions.json and patch files. * refactor(pgrx): conditionally apply rustc wrapper when < 0.12 Replace inline bash script generation with writeShellScriptBin for rustc wrapper that filters empty postmaster_stub.rs arguments. Apply wrapper only for pgrx < 0.12 since issue was fixed upstream in pgrx#1435 and pgrx#1441. * refactor(nixpkgs): use let binding for oldstable instead of overlay Move nixpkgs-oldstable import to a let binding to avoid importing it three times... * refactor(groonga): move groonga package to pgroonga extension Remove supabase-groonga from global flake packages and import it locally in pgroonga extension instead. Other components access groonga via pgroonga.passthru.groonga since groonga is only used by pgroonga. * fix(ansible): remove supabase-groonga from package install list The supabase-groonga package was moved to pgroonga extension in commit 1f0ed1f, but Ansible was still trying to install it separately causing build failures. * Revert "fix(ansible): remove supabase-groonga from package install list" This reverts commit 55474686405394826ad07a032cbcf0e36a924554. * Revert "refactor(groonga): move groonga package to pgroonga extension" This reverts commit cfa9bcb92219f391469ae743f95c8856b0a99561. * fix(darwin): remove deprecated apple_sdk frameworks from supascan Remove references to darwin.apple_sdk.frameworks.{IOKit,CoreFoundation} which have been deprecated in nixpkgs. Disable CGO to avoid Darwin framework dependencies entirely. * chore: bump version suffix to test * feat(pg_repack): use default nixos extension test Lower the maintenance by reusing the default extension test. * fix: guard upgrade path checks when support_upgrade is false Address CodeRabbit review feedback: when support_upgrade is false (e.g., for pg_repack), the test was still calling check_upgrade_path unconditionally, which is semantically incorrect. Fixed by wrapping upgrade path test calls with conditional guards: - Only call test.check_upgrade_path() when support_upgrade is true - Applies to both PostgreSQL 15 and 17 test sections - Extensions like pg_repack (support_upgrade=false) now skip upgrade tests This ensures test logic matches the extension's actual upgrade capabilities. --------- Co-authored-by: Yvan Sraka <[email protected]> Co-authored-by: Sam Rose <[email protected]> Co-authored-by: samrose <[email protected]>

wolfgangwalther requested a review from thoughtpolice as a code owner September 15, 2024 09:41

ofborg bot requested review from Ma27, danbst, globin and ivan September 15, 2024 11:14

wolfgangwalther force-pushed the postgresql-tzdata branch from 1ef6f48 to 52bb840 Compare September 15, 2024 15:10

alyssais approved these changes Sep 15, 2024

View reviewed changes

Scrumplex approved these changes Sep 15, 2024

View reviewed changes

Ma27 approved these changes Sep 15, 2024

View reviewed changes

Ma27 merged commit 9387bb8 into NixOS:staging Sep 15, 2024

wolfgangwalther deleted the postgresql-tzdata branch September 15, 2024 21:13

This was referenced Jan 20, 2025

tzdata: 2024b -> 2025a #374725

Merged

postgresql: fix build #375623

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

postgresql: fix regress tests after tzdata update#342026

postgresql: fix regress tests after tzdata update#342026
Ma27 merged 1 commit intoNixOS:stagingfrom
wolfgangwalther:postgresql-tzdata

wolfgangwalther commented Sep 15, 2024

Uh oh!

Ma27 commented Sep 15, 2024

Uh oh!

alyssais commented Sep 15, 2024

Uh oh!

wolfgangwalther commented Sep 15, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

wolfgangwalther commented Sep 15, 2024

Description of changes

Things done

Uh oh!

Ma27 commented Sep 15, 2024

Uh oh!

alyssais commented Sep 15, 2024

Uh oh!

wolfgangwalther commented Sep 15, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants