Skip to content

gst_all_1.gst-plugins-rs: check that system libwebp was linked#254915

Merged
yu-re-ka merged 1 commit intoNixOS:masterfrom
yu-re-ka:libwebp-gst-plugins-rs
Sep 14, 2023
Merged

gst_all_1.gst-plugins-rs: check that system libwebp was linked#254915
yu-re-ka merged 1 commit intoNixOS:masterfrom
yu-re-ka:libwebp-gst-plugins-rs

Conversation

@yu-re-ka
Copy link
Contributor

Description of changes

#254798

Things done

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 23.11 Release Notes (or backporting 23.05 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

@ofborg ofborg bot requested a review from lilyinstarlight September 13, 2023 09:48
@ofborg ofborg bot added 10.rebuild-darwin: 11-100 This PR causes between 11 and 100 packages to rebuild on Darwin. 10.rebuild-linux: 11-100 This PR causes between 11 and 100 packages to rebuild on Linux. labels Sep 13, 2023
@lilyinstarlight
Copy link
Member

lilyinstarlight commented Sep 13, 2023

The idea behind this looks fine to me, but I'll give it a more thorough review late today. Thank you!

It kinda sucks libwebp-sys2 doesn't even bother trying pkg-config when on cross before deciding to just statically build their vendored libwebp... (since it would theoretically just work fine for us)

Edit: Wait wouldn't this fail for cross build/host pairs where stdenv.buildPlatform.canExecute stdenv.hostPlatform (e.g. x86_64->i686)?

@yu-re-ka
Copy link
Contributor Author

Edit: Wait wouldn't this fail for cross build/host pairs where [...]

Indeed it would, but intentionally because it would have the insecure libwebp version. currently there is no easy way to build a secure version on those systems.
One intermediate solution would be to add a withSystemLibwebp which is !cross && !musl, but then also set knownVulnerabilities if the the flag is false. So cross-compile would then be allowed, but only with NIXPKGS_ALLOW_INSECURE=1

@lilyinstarlight
Copy link
Member

Yeah I'd rather set meta.knownVulnerabilities than fail the build. I think we can just set both meta.knownVulnerabilities and disable the installCheck based on the same stdenv.hostPlatform.isStatic || stdenv.buildPlatform != stdenv.hostPlatform conditional

@lilyinstarlight
Copy link
Member

lilyinstarlight commented Sep 14, 2023

So, curiously enough, I cross-compiled gst-plugins-rs (x86_64-linux -> aarch64-linux) and it does still link against system libwebpdemux.so, despite what the upstream libwebp-sys2 crate seems like it should be doing based on the build.rs 🤔

[lily@build01:~/nixpkgs]$ readelf -a /nix/store/23prbi00d4mqy2328mnn2jnmrgnxf0nb-gst-plugins-rs-aarch64-unknown-linux-gnu-0.11.0+fixup/lib/gstreamer-1.0/libgstrswebp.so | grep -i libwebpdemux.so
 0x0000000000000001 (NEEDED)             Shared library: [libwebpdemux.so.2]

And indeed the symbols are dynamically loaded (i.e. it's not just unnecessarily marked "needed" for some reason in the ELF header)

[lily@build01:~/nixpkgs]$ objdump -T /nix/store/23prbi00d4mqy2328mnn2jnmrgnxf0nb-gst-plugins-rs-aarch64-unknown-linux-gnu-0.11.0+fixup/lib/gstreamer-1.0/libgstrswebp.so | grep -i webp
/nix/store/23prbi00d4mqy2328mnn2jnmrgnxf0nb-gst-plugins-rs-aarch64-unknown-linux-gnu-0.11.0+fixup/lib/gstreamer-1.0/libgstrswebp.so:     file format elf64-little
0000000000000000      DF *UND*  0000000000000000  Base        WebPAnimDecoderDelete
0000000000000000      DF *UND*  0000000000000000  Base        WebPAnimDecoderNewInternal
0000000000000000      DF *UND*  0000000000000000  Base        WebPAnimDecoderGetInfo
0000000000000000      DF *UND*  0000000000000000  Base        WebPAnimDecoderOptionsInitInternal
0000000000000000      DF *UND*  0000000000000000  Base        WebPAnimDecoderHasMoreFrames
0000000000000000      DF *UND*  0000000000000000  Base        WebPAnimDecoderGetNext
00000000000117a0 g    DF .text  000000000000000c  Base        gst_plugin_rswebp_get_desc
0000000000011748 g    DF .text  0000000000000058  Base        gst_plugin_rswebp_register

@lilyinstarlight
Copy link
Member

lilyinstarlight commented Sep 14, 2023

Ah it actually checks pkg-config first in all but the case where the Cargo static feature is enabled on the crate, so I misread the build.rs initially. We do not I think enable the static crate feature for stdenv.hostPlatform.isStatic anyway for Rust, just add -C target-feature=+crt-static to RUSTFLAGS. And gst-plugins-rs does not add the static feature either afaict (also this is probably never gonna successfully statically build in nixpkgs anyway, without a lot of effort, because too much gobject stuff to untangle...)

So I think it should never be using the vendored libwebp in gst-plugins-rs in nixpkgs and we can just unconditionally run this installCheck (since it also does not depend on native code execution, although cross forces it off anyway in stdenv.mkDerivation for legacy reasons...)

@yu-re-ka
Copy link
Contributor Author

Ah it actually checks pkg-config first in all but the case where the Cargo static feature is enabled on the crate

Great, even better!

Copy link
Member

@lilyinstarlight lilyinstarlight left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So yeah I'm fine with this as-is then. Are you considering this a permanent change?

If so, libwebp might not be the only one we do this for if other serious vulns come out, so the installCheck would need to be updated to accomodate. But that can be done if/when that occurs, so I don't think it's a blocker here

@yu-re-ka yu-re-ka merged commit d6578bc into NixOS:master Sep 14, 2023
@yu-re-ka yu-re-ka deleted the libwebp-gst-plugins-rs branch September 14, 2023 01:51
@yu-re-ka
Copy link
Contributor Author

I would like to keep this permanently. One thing less to worry about when the next libwebp vulnerability comes around.
Other dependencies can be added to the check as needed.

@lilyinstarlight
Copy link
Member

lilyinstarlight commented Sep 14, 2023

Other dependencies can be added to the check as needed.

I meant the conditional would need to be adjusted to not bother with the webp plugin check and instead the tests in the installCheckPhase would need to be dynamically added based on selected plugins

But again that can be done if/when others are added, since I really don't care enough

@lilyinstarlight
Copy link
Member

Just realized, I forgot to say thank you for checking on this package!

The libwebp thing seems a bit of an unfortunate mess of random vendoring and I appreciate your (and everyone's) work on it!

@jtojnar
Copy link
Member

jtojnar commented Sep 14, 2023

Could we rm -r the vendored libwebp instead to have if fail early?

@lilyinstarlight
Copy link
Member

lilyinstarlight commented Sep 14, 2023

Could we rm -r the vendored libwebp instead to have if fail early?

I'm not sure how we'd do that for a vendored libwebp copy in a dependent crate. If you have something more specific, I suppose we could add it though (although it shouldn't be using the vendored libwebp in our gst-plugins-rs anyway on major platforms or for cross to major platforms afaict)

@jtojnar
Copy link
Member

jtojnar commented Sep 14, 2023

Ah, looks like importCargoLock does not have mechanism for that. Nevermind then.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

10.rebuild-darwin: 11-100 This PR causes between 11 and 100 packages to rebuild on Darwin. 10.rebuild-linux: 11-100 This PR causes between 11 and 100 packages to rebuild on Linux.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants