Skip to content

SDL_compat: use SDL3 through sdl2-compat#386495

Merged
K900 merged 2 commits intoNixOS:masterfrom
marcin-serwin:push-nvnlzzuqztvk
Mar 15, 2025
Merged

SDL_compat: use SDL3 through sdl2-compat#386495
K900 merged 2 commits intoNixOS:masterfrom
marcin-serwin:push-nvnlzzuqztvk

Conversation

@marcin-serwin
Copy link
Contributor

I have manually tested all the packages depending on SDL_compat:

  • rott (shareware version): menu doesn't render with the currently packaged sdl2-compat version (2.30.52), but upgrading to 2.32.52 fixes the issue
  • dosbox: runs fine
  • openxcom (with "X-COM: UFO Defense" Steam files): runs fine
  • katawa-shoujo: boots to black screen with the currently packaged sdl2-compat version (2.30.52), but upgrading to 2.32.52 fixes the issue
  • zandronum (with Doom 1 shareware): runs fine

Things done

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
    • sandbox = relaxed
    • sandbox = true
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 25.05 Release Notes (or backporting 24.11 and 25.05 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

Add a 👍 reaction to pull requests you find important.

@github-actions github-actions bot added 10.rebuild-darwin: 1-10 This PR causes between 1 and 10 packages to rebuild on Darwin. 10.rebuild-linux: 11-100 This PR causes between 11 and 100 packages to rebuild on Linux. labels Mar 2, 2025
@nix-owners nix-owners bot requested a review from peterhoeg March 2, 2025 22:37
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we actually need this option?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i believe not; .override { sdl2-compat = SDL2; } is simple enough if needed occasionally, and could always be an alias in all-packages.nix if it ever becomes a recurring pattern.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same vibe. Lets just drop it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@LordGrimmauld
Copy link
Contributor

LordGrimmauld commented Mar 12, 2025

This was previously blocked because it depended on #386759, but that has been merged four days ago. What is the status on this? Any blockers?

I am also a little concerned about the seemingly flaky tests reported in #386759 (comment). If this becomes a channel blocker over at hydra, things will be ugly.

@grimmauld-bot
Copy link

nixpkgs-review result

Generated using nixpkgs-review.

Command: nixpkgs-review pr 386495


x86_64-linux

✅ 14 packages built:
  • SDL_compat
  • bottles
  • dosbox
  • katawa-shoujo
  • keen4
  • open-watcom-v2
  • open-watcom-v2-unwrapped
  • openxcom
  • rott
  • rott-shareware
  • zandronum
  • zandronum-alpha
  • zandronum-alpha-server
  • zandronum-server

@LordGrimmauld
Copy link
Contributor

I now set my main system to use SDL_compat through sdl12-compat, and everything still builds and works perfectly. Including bottles, which sometimes can be fragile to dosbox changes. From a functionality perspective, i call this ready.

@marcin-serwin
Copy link
Contributor Author

I am also a little concerned about the seemingly flaky tests reported in #386759 (comment). If this becomes a channel blocker over at hydra, things will be ugly.

Good point. There is an environment variable that skips timer tests. Perhaps we should set it preemptively.

@LordGrimmauld
Copy link
Contributor

I am also a little concerned about the seemingly flaky tests reported in #386759 (comment). If this becomes a channel blocker over at hydra, things will be ugly.

Good point. There is an environment variable that skips timer tests. Perhaps we should set it preemptively.

if this is as easy as an export in preCheck then i am in favor of that change. And that can be a separate commit with its own explanation too, will make the git blame more helpful.

Copy link
Contributor

@LordGrimmauld LordGrimmauld left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

preCheck = ''
  # skip timing-based tests as those are flaky
  export SDL_TESTS_QUICK=true
'';

untested, but should do the trick.

@marcin-serwin
Copy link
Contributor Author

if this is as easy as an export in preCheck then i am in favor of that change. And that can be a separate commit with its own explanation too, will make the git blame more helpful.

Done

Copy link
Contributor

@LordGrimmauld LordGrimmauld left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alright. This looks good to me. Running one last nixpkgs-review, but env vars in check phase really do not break stuff. No blockers from me anymore.
The ninja or no ninja is a consideration i am unqualified to make.

@nix-owners nix-owners bot requested a review from nadiaholmquist March 12, 2025 21:03
@LordGrimmauld
Copy link
Contributor

LordGrimmauld commented Mar 12, 2025

Oh uh... Crap, you are setting yourself up for merge conflicts. This commit should probably go into #388079 now that i think about it? Or you rebase one of the two PRs when the other gets the merge. but currently this will conflict. I am an idiot for telling you to do it in here, sorry!

Eh, you are smart, you'll figure it out.

Copy link
Contributor

@nadiaholmquist nadiaholmquist left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@wegank wegank added 12.approved-by: package-maintainer This PR was reviewed and approved by a maintainer listed in any of the changed packages. 12.approvals: 2 This PR was reviewed and approved by two persons. labels Mar 12, 2025
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Assuming there is no harm in setting the environment variable at the build stage, we should just do

env.SDL_TESTS_QUICK = 1;

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i just tested, either works perfectly. Your approach is cleaner.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done, but I moved it to #388079 to avoid merge conflicts.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where does libx11 come from here? Was it propagated by normal SDL2 or something?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it was, see

dlopenPropagatedBuildInputs =
[ ]
# Propagated for #include <GLES/gl.h> in SDL_opengles.h.
++ lib.optional (openglSupport && !stdenv.hostPlatform.isDarwin) libGL
# Propagated for #include <X11/Xlib.h> and <X11/Xatom.h> in SDL_syswm.h.
++ lib.optionals x11Support [ libX11 ];

It might however make sense to do the following:

# SDL_compat
propagatedBuildInputs = sdl2-compat.propagatedBuildInputs ++ [ ... ];
# sdl2-compat
propagatedBuildInputs = sdl3.propagatedBuildInputs ++ [ ... ];

This adds extra complexity, but would allow us to not maintain multiple lists of the same thing.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(not quite familiar with how propagatedBuildInputs works, there might be better ways.)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This shouldn't be necessary though, that's what propagation does already - if sdl2-compat is in sdl-compat's propagatedBuildInputs, sdl2's propagatedBuildInputs will also be propagated.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, thats the issue!!
See

buildInputs = [
sdl3
];

sdl2-compat is not listing sdl3 as propagated build input. THAT is the issue and should be fixed. Then explicit libX11 can be dropped here.

Copy link
Contributor

@K900 K900 Mar 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It should, however, propagate libX11, because include/SDL2/SDL_syswm.h refers to xlib includes.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sdl2-compat also does a dlopen on sdl3 at runtime, see #388079. The fact that is currently missing is more a mistake/bug than anything.
(these two PRs really should have been one, huh?)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i need to do some testing. But either way, libX11 being in SDL_compat explicitly seems to not be the correct solution.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sdl2-compat also does a dlopen on sdl3 at runtime, see #388079. The fact that is currently missing is more a mistake/bug than anything.

But that is handled by rpath and doesn't really have to do with propagated build inputs, or am I missing something?

But either way, libX11 being in SDL_compat explicitly seems to not be the correct solution.

I'm still a bit fuzzy on how exactly propagated build inputs work but I don't understand why that wouldn't be the correct approach. It seems like sdl2-compat doesn't need to propagate libX11, since I've managed to build some games without it, but SDL1 does so that's why it appears here. It's a bit weird that SDL2 propagated libX11 but maybe that was the issue in the first place and now it got accidentally corrected in sdl2-compat?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SDL2 propagated libX11. If sdl2-compat is supposed to be a drop-in replacement, it should do the same imo.

@wegank wegank added 12.approvals: 1 This PR was reviewed and approved by one person. and removed 12.approvals: 2 This PR was reviewed and approved by two persons. labels Mar 13, 2025
@K900
Copy link
Contributor

K900 commented Mar 14, 2025

#388079 addresses the libx11 thing, correct?

@LordGrimmauld
Copy link
Contributor

#388079 addresses the libx11 thing, correct?

i just tried, and no; commenting out libX11 here, SDL_compat already fails to build.

@marcin-serwin
Copy link
Contributor Author

After some experimenting and reading about dependency propagation my current
understanding is that the proper fix is to remove propagatedBuildInputs from
SDL{1,2,3} and SDL_compat.

My reasoning: Package B should add package A to propagatedBuildInputs if any
package C depending on package B also depends (directly) on package A, for
example, because compiling package C requires passing -lB -lA to the compiler.
This is not the case for any of the SDL libs; it is possible to compile a
package that only depends on the SDL itself (excluding standard dependencies
like libc). If you look at the outputs of *-config tools you can see that on
NixOS they only link to the SDL library.

`*-config` outputs
$ sdl-config --libs
-L/nix/store/ryxfmihhya90il7fij24g5l5l23vql9s-SDL-1.2.15/lib -Wl,-rpath,/nix/store/ryxfmihhya90il7fij24g5l5l23vql9s-SDL-1.2.15/lib -lSDL -lpthread -L/nix/store/nhrql1l31nyz7hg8n16mw3g4vza0r8zy-SDL-1.2.15-dev/lib -L/nix/store/nhrql1l31nyz7hg8n16mw3g4vza0r8zy-SDL-1.2.15-dev/lib -L/nix/store/nhrql1l31nyz7hg8n16mw3g4vza0r8zy-SDL-1.2.15-dev/lib

$ sdl2-config --libs
-L/nix/store/0k3g1sgznf7mpz1ycg0sdnlfb3r5405f-SDL2-2.32.0/lib -Wl,-rpath,/nix/store/0k3g1sgznf7mpz1ycg0sdnlfb3r5405f-SDL2-2.32.0/lib -Wl,--enable-new-dtags -lSDL2

$ pkg-config --libs sdl3
-L/nix/store/1lc2wbfgf3ll0qa5capjkp1b0dhi81m3-sdl3-3.2.6-lib/lib -Wl,-rpath,/nix/store/1lc2wbfgf3ll0qa5capjkp1b0dhi81m3-sdl3-3.2.6-lib/lib -Wl,--enable-new-dtags -lSDL3

It makes sense that this is the case: If an SDL app wants to create a window
then it calls SDL_CreateWindow which transfers the control to the shared
library libSDL.so which then either dlopens or links directly to the
appropriate backend, like X11, and calls the appropriate functions there. Notice
that the app is oblivious to whether X11 is present in the system, it may as
well be talking to Wayland. The actual backend is a direct dependency of the
libSDL.so so it's present on the system due to being in buildInputs of SDL
package.

Of course, some packages may break the abstraction and choose to talk to
X11/Pipewire/Vulkan/whatever directly, bypassing the SDL, but then they should
declare it via their buildInputs and not rely on SDL to propagate it. This is
the case here, with SDL_compat depending on X11.

I have managed to build some simple applications after removing propagated build
inputs from the SDL packages and they are running fine so experimentally it
seems to be working too.

Is this correct or is there some other reason requiring the SDL packages to
propagate deps? @getchoo, perhaps you know since you've recently packaged SDL3.

@LordGrimmauld LordGrimmauld mentioned this pull request Mar 18, 2025
13 tasks
LordGrimmauld added a commit to LordGrimmauld/nixpkgs that referenced this pull request Mar 18, 2025
commit 7aeac03 (PR NixOS#386495) dropped libX11 and libGLU from propagatedBuildInputs in SDL_compat, and NixOS#389106 pointed SDL to SDL_compat, which broke this build

Hydra fail: https://hydra.nixos.org/build/292729609
@LordGrimmauld LordGrimmauld mentioned this pull request Mar 18, 2025
13 tasks
LordGrimmauld added a commit to LordGrimmauld/nixpkgs that referenced this pull request Mar 18, 2025
commit 7aeac03 (PR NixOS#386495) dropped libX11 and libGLU from propagatedBuildInputs in SDL_compat, and NixOS#389106 pointed SDL to SDL_compat, which broke this build

Hydra fail: https://hydra.nixos.org/build/292729441
@LordGrimmauld LordGrimmauld mentioned this pull request Mar 18, 2025
13 tasks
@LordGrimmauld
Copy link
Contributor

Some context about these: https://github.com/libsdl-org/sdl12-compat/blob/35ef746cf1a4fde8c442dc92fe9b18c3f90d7855/include/SDL/SDL_opengl.h#L51
Turns out SDL_compat does explicitly use GL headers. Surprisingly many packages use that include and just expect GL to be there. I am happy to fix these, but i am no longer certain killing the propagatedBuildInputs was the correct decision. At least this is out of the mass-rebuild path, so these breaks all only have minor impact.

LordGrimmauld added a commit to LordGrimmauld/nixpkgs that referenced this pull request Mar 18, 2025
commit 7aeac03 (PR NixOS#386495) dropped libX11 and libGLU from propagatedBuildInputs in SDL_compat, and NixOS#389106 pointed SDL to SDL_compat, which broke this build

Hydra fail: https://hydra.nixos.org/build/292730126
@LordGrimmauld LordGrimmauld mentioned this pull request Mar 18, 2025
13 tasks
@marcin-serwin
Copy link
Contributor Author

marcin-serwin commented Mar 18, 2025

Some context about these: https://github.com/libsdl-org/sdl12-compat/blob/35ef746cf1a4fde8c442dc92fe9b18c3f90d7855/include/SDL/SDL_opengl.h#L51 Turns out SDL_compat does explicitly use GL headers.

This is an optional include file and if a package uses it then it should declare OpenGL in its dependencies. This is also the position of the SDL contributors: libsdl-org/sdl2-compat#405 (comment).

Surprisingly many packages use that include and just expect GL to be there.

If a package uses it and does not depend on libGL then it's a problem with the derivation. Failures like these were expected.

@K900
Copy link
Contributor

K900 commented Mar 18, 2025

Yeah, I don't think it makes sense to propagate optional dependencies that are not used by a lot of things.

LordGrimmauld added a commit to LordGrimmauld/nixpkgs that referenced this pull request Mar 18, 2025
commit 7aeac03 (PR NixOS#386495) dropped libX11 and libGLU from propagatedBuildInputs in SDL_compat, and NixOS#389106 pointed SDL to SDL_compat, which broke this build

Hydra fail: https://hydra.nixos.org/build/292734958
@LordGrimmauld LordGrimmauld mentioned this pull request Mar 18, 2025
13 tasks
LordGrimmauld added a commit to LordGrimmauld/nixpkgs that referenced this pull request Mar 18, 2025
commit 7aeac03 (PR NixOS#386495) dropped libX11 and libGLU from propagatedBuildInputs in SDL_compat, and NixOS#389106 pointed SDL to SDL_compat, which broke this build

Hydra fail: https://hydra.nixos.org/build/292743187
@LordGrimmauld LordGrimmauld mentioned this pull request Mar 18, 2025
13 tasks
LordGrimmauld added a commit to LordGrimmauld/nixpkgs that referenced this pull request Mar 18, 2025
commit 7aeac03 (PR NixOS#386495) dropped libX11 and libGLU from propagatedBuildInputs in SDL_compat, and NixOS#389106 pointed SDL to SDL_compat, which broke this build

Hydra fail: https://hydra.nixos.org/build/292743363
@LordGrimmauld LordGrimmauld mentioned this pull request Mar 18, 2025
13 tasks
LordGrimmauld added a commit to LordGrimmauld/nixpkgs that referenced this pull request Mar 18, 2025
commit 7aeac03 (PR NixOS#386495) dropped libX11 and libGLU from propagatedBuildInputs in SDL_compat, and NixOS#389106 pointed SDL to SDL_compat, which broke this build

Hydra fail: https://hydra.nixos.org/build/292810698
@LordGrimmauld LordGrimmauld mentioned this pull request Mar 18, 2025
13 tasks
LordGrimmauld added a commit to LordGrimmauld/nixpkgs that referenced this pull request Mar 18, 2025
commit 7aeac03 (PR NixOS#386495) dropped libX11 and libGLU from propagatedBuildInputs in SDL_compat, and NixOS#389106 pointed SDL to SDL_compat, which broke this build

Hydra fail: https://hydra.nixos.org/build/292771916
@LordGrimmauld LordGrimmauld mentioned this pull request Mar 18, 2025
13 tasks
LordGrimmauld added a commit to LordGrimmauld/nixpkgs that referenced this pull request Mar 19, 2025
commit 7aeac03 (PR NixOS#386495) dropped libX11 and libGLU from propagatedBuildInputs in SDL_compat, and NixOS#389106 pointed SDL to SDL_compat, which broke this build
LordGrimmauld added a commit to LordGrimmauld/nixpkgs that referenced this pull request Mar 19, 2025
commit 7aeac03 (PR NixOS#386495) dropped libX11 and libGLU from propagatedBuildInputs in SDL_compat, and NixOS#389106 pointed SDL to SDL_compat, which broke this build

Hydra fail: https://hydra.nixos.org/build/292732003
@LordGrimmauld LordGrimmauld mentioned this pull request Mar 19, 2025
13 tasks
LordGrimmauld added a commit to LordGrimmauld/nixpkgs that referenced this pull request Mar 19, 2025
commit 7aeac03 (PR NixOS#386495) dropped libX11 and libGLU from propagatedBuildInputs in SDL_compat, and NixOS#389106 pointed SDL to SDL_compat, which broke this build

Hydra fail: https://hydra.nixos.org/build/292728946
@LordGrimmauld LordGrimmauld mentioned this pull request Mar 19, 2025
13 tasks
antonmosich added a commit to antonmosich/nixpkgs that referenced this pull request Apr 18, 2025
commit 7aeac03 (PR NixOS#386495) dropped libX11 and libGLU from
propagatedBuildInputs in SDL_compat, and NixOS#389106 pointed SDL to
SDL_compat, which broke this build
@antonmosich antonmosich mentioned this pull request Apr 18, 2025
13 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

10.rebuild-darwin: 1-10 This PR causes between 1 and 10 packages to rebuild on Darwin. 10.rebuild-linux: 11-100 This PR causes between 11 and 100 packages to rebuild on Linux. 12.approvals: 1 This PR was reviewed and approved by one person. 12.approved-by: package-maintainer This PR was reviewed and approved by a maintainer listed in any of the changed packages.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants