Skip to content

Comments

gstreamer-1.16.0#54398

Merged
nh2 merged 4 commits intoNixOS:masterfrom
nh2:gstreamer-1.15.1
Jun 20, 2019
Merged

gstreamer-1.16.0#54398
nh2 merged 4 commits intoNixOS:masterfrom
nh2:gstreamer-1.15.1

Conversation

@nh2
Copy link
Contributor

@nh2 nh2 commented Jan 21, 2019 •
edited
Loading

Motivation for this change

The latest gstreamer version The current gstreamer development version, leading up to the next to-be-released 1.16 stable version. Already doing some work here so we have less work to do when 1.16 comes out.

Edit: This is now, and the PR has been updated.

Copying an important part of the commit message:

During the 1.14 -> 1.15 upgrade, lots of stuff stopped working because gstreamer changed what features are enabled by default and which ones are automatically turned on/off via pkgconfig dependency detection.

This resulted in the gstreamer ("core" attribute in nixpkgs) package to have only 15 of its previous 163 build targets enabled, and downstream packages breaking correspondingly.

To ease maintainability and to ensure users will find the expected features available (and when not, will see in the nix file why not), we now pass the -Dauto_features=enabled Meson build flag to all gstreamer builds, which sets all auto dependencies to enabled, and we explicitly disable those that we can't build .

This means in particular that gst-plugins-bad now has vastly more integrations (namely all for which nixpkgs has libraries available).

@nh2's notes while working on this (for posterity)

  • various meson patches: Removed: Seem no longer necessary, but I haven't checked all the details and whether the patches were fully merged or only partially.
    See e.g. https://lists.freedesktop.org/archives/gstreamer-bugs/2018-August/223857.html where they say

    Committed without the gl part which was installed into libdir on purpose. You will have to take account of that in your packaging.

  • libice: gstreamer-check-1.0.pc not found (and indeed it's not there)
  • core configure output: 
    -Found pkg-config: /nix/store/4iij6zf2z7gx68rx3kg28xi5rqvnfja6-pkg-config-0.29.2/bin/pkg-config (0.29.2)
-Dependency libunwind found: YES 1.3.1
-Found CMake: NO
-Dependency libdw found: NO (tried pkgconfig and cmake)
-Message: Support for backtraces is partial only.
+Dependency libunwind skipped: feature libunwind disabled
+Dependency libdw skipped: feature libdw disabled
+Checking for function "backtrace" : YES
    • likely the same as before (partial support), but I improved it by adding an elfutils dep which provides libdw
  • core configure output:
    -Build targets in project: 163
    +Build targets in project: 15
  • gst-validate build complains that GstPbutils-1.0.gir couldn't be found / GEN'd even though it says checking for GST_PBUTILS... yes before
    • probably due to core configure output: Program g-ir-scanner skipped: feature introspection disabled

Things done

  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Assured whether relevant documentation is up to date
  • Fits CONTRIBUTING.md.

Tested via:

NIX_PATH=nixpkgs=. nix-shell -p gst_all_1.gstreamer -p gst_all_1.gst-plugins-good -p gst_all_1.gst-plugins-base -p gst_all_1.gst-plugins-bad -p gst_all_1.gst-plugins-ugly -p gst_all_1.gst-libav -p gst_all_1.gst-rtsp-server -p gst_all_1.gst-validate -p gst_all_1.gst-vaapi

CC @matthewbauer

TODO

@nh2 nh2 changed the title The latest gstreamer version. gstreamer-1.15.1 Jan 21, 2019
@nh2 nh2 requested a review from matthewbauer January 21, 2019 02:03
@GrahamcOfBorg GrahamcOfBorg added 8.has: clean-up This PR removes packages or removes other cruft 8.has: package (new) This PR adds a new package 10.rebuild-darwin: 101-500 This PR causes between 101 and 500 packages to rebuild on Darwin. 10.rebuild-linux: 501+ This PR causes many rebuilds on Linux and should normally target the staging branches. labels Jan 21, 2019
@nh2 nh2 mentioned this pull request Jan 21, 2019
Merged
10 tasks
@jtojnar
Copy link
Member

jtojnar commented Jan 21, 2019

I did not enable gtksink by default due to its closure size (150 MB vs 300 MB).

Also please note that 1.15 is an unstable development branch for 1.16 stable release.

@nh2
Copy link
Contributor Author

nh2 commented Jan 21, 2019

I did not enable gtksink by default due to its closure size (150 MB vs 300 MB).

@jtojnar That makes sense; the PR doesn't change this, it is still optional.

Also please note that 1.15 is an unstable development branch for 1.16 stable release.

Oops OK. Let's wait until 1.16 before continuing with this then.

@nh2 nh2 changed the title gstreamer-1.15.1 WIP: gstreamer-1.15.1 (to eventually become the gstreamer-1.16 PR) Jan 21, 2019
@jtojnar
Copy link
Member

jtojnar commented Jan 21, 2019

The removed patches come from https://bugzilla.gnome.org/show_bug.cgi?id=794856 and have been committed.

@jtojnar jtojnar added 2.status: work-in-progress 2.status: wait-for-upstream Waiting for upstream fix (or their other action). labels Jan 21, 2019
@nh2
Copy link
Contributor Author

nh2 commented Jan 21, 2019

The removed patches come from https://bugzilla.gnome.org/show_bug.cgi?id=794856 and have been committed.

@jtojnar Thanks! I ticked off the bullet point in the issue description based on this feedback.

closure size

We will also want to do a closure size check on the added -bad dependencies eventually.

I think it makes sense that it'll increase and will likely be worth it, as for end users nothing is more annoying than a gstreamer plugin but it not working on your machine because the distro hasn't enabled it, and -bad is supposed to contain as many integrations as possible, but we should still know the number.

Also, I wouldn't be opposed to introducing feature flags as arguments to the callPackage for each of the external dependencies so that users can easily build a smaller -bad package for their use cases, but I think the default should be ? true for most features so that in the default case multimedia "just works".

@nh2 nh2 force-pushed the gstreamer-1.15.1 branch from 39eb078 to 802bc29 Compare January 22, 2019 04:28
@jtojnar
Copy link
Member

jtojnar commented Mar 6, 2019

115.2 was released, 1.16 in few weeks: https://gstreamer.freedesktop.org/news/#2019-02-27T10:00:00Z

@nh2 nh2 force-pushed the gstreamer-1.15.1 branch from 512cee1 to 4876f0b Compare May 1, 2019 14:17
@nh2
Copy link
Contributor Author

nh2 commented May 1, 2019

1.16 was released.

I've updated the PR with another commit to update to 1.16 and fix a few build errors.

I've also made an upstream PR to fix a script not being executable, which broke the build:

https://gitlab.freedesktop.org/gstreamer/gstreamer/merge_requests/141

@ofborg ofborg bot added the 10.rebuild-linux: 501-1000 This PR causes many rebuilds on Linux and should normally target the staging branches. label May 1, 2019
@nh2 nh2 changed the title WIP: gstreamer-1.15.1 (to eventually become the gstreamer-1.16 PR) gstreamer-1.16.0 May 1, 2019
@nh2 nh2 removed 2.status: wait-for-upstream Waiting for upstream fix (or their other action). 2.status: work-in-progress 8.has: package (new) This PR adds a new package labels May 1, 2019
@nh2
Copy link
Contributor Author

nh2 commented May 1, 2019

Let's see if this works:

@GrahamcOfBorg build gst_all_1

@adisbladis
Copy link
Member

adisbladis commented May 1, 2019

@GrahamcOfBorg build gst_all_1

jtojnar
jtojnar reviewed May 1, 2019
View reviewed changes
jtojnar
jtojnar reviewed May 1, 2019
View reviewed changes
@nh2 nh2 force-pushed the gstreamer-1.15.1 branch 2 times, most recently from 7750c09 to d563757 Compare May 3, 2019 00:22
@nh2
Copy link
Contributor Author

nh2 commented May 3, 2019

@GrahamcOfBorg build gst_all_1

@nh2
Copy link
Contributor Author

nh2 commented May 3, 2019

I've backed out the evaluation fixes Darwin out of this PR, as that isn't really related to the upgrade, and will handle those with a separate branch.

If ofborg goes green, this is good to merge from my side.

@nh2 nh2 mentioned this pull request May 3, 2019
Closed
12 tasks
@jtojnar
Copy link
Member

jtojnar commented May 3, 2019

Dependency changes described in release notes do not affect us.

What is up with ffmpeg? And do we care about the rust bindings?

@nh2
Copy link
Contributor Author

nh2 commented May 4, 2019

Dependency changes described in release notes do not affect us.

Thanks for checking!

What is up with ffmpeg?

I've updated gst-libav (which is the ffmpeg binding, right?) in this PR as one of the many gst packages, and it builds successfully.

And do we care about the rust bindings?

To my knowledge they aren't packaged in nixpkgs yet.

I'll leave that to somebody else, as I have no experience with packaging Rust for nixpkgs, and care mainly about the update to 1.16 and the improved meson build system I've added in this PR.

@nh2 nh2 mentioned this pull request May 4, 2019
Closed
11 tasks
jtojnar
jtojnar requested changes May 4, 2019
View reviewed changes
@nh2 nh2 force-pushed the gstreamer-1.15.1 branch from d563757 to b84a8ad Compare May 4, 2019 17:40
@nh2
Copy link
Contributor Author

nh2 commented May 5, 2019

@jtojnar Good to go?

@nixos-discourse
Copy link

nixos-discourse commented May 14, 2019

This pull request has been mentioned on Nix community. There might be relevant details there:

https://discourse.nixos.org/t/what-are-your-goals-for-19-09/2875/24

@jtojnar jtojnar mentioned this pull request Jun 19, 2019
Merged
nh2 added 4 commits June 19, 2019 12:44
@nh2
gstreamer: 1.14.4 -> 1.15.1, and extended gst-plugins-bad.
6fa0062 
During the 1.14 -> 1.15 upgrade, lots of stuff stopped working because
gstreamer changed what features are enabled by default and which ones are
automatically turned on/off via pkgconfig dependency detection.

This resulted in the `gstreamer` ("core" attribute in nixpkgs) package
to have only 15 of its previous 163 build targets enabled, and downstream
packages breaking correspondingly.

To ease maintainability and to ensure users will find the expected features
available (and when not, will see in the nix file why not), we now pass
the `-Dauto_features=enabled` Meson build flag to all gstreamer builds,
which sets all `auto` dependencies to `enabled`, and we explicitly disable
those that we can't build.

This means in particular that `gst-plugins-bad` now has vastly more integrations
(namely all for which nixpkgs has libraries available).
@nh2
gstreamer: 1.15.1 -> 1.16.0
8d495e0 
Changes needed:

* core: dbghelp disabled as it's not needed
* core: Fixed script not being executable and skipped by patchShebangs
* base: gtk_doc disabled because of wayland symbol error
* ges: Patch removed that was merged upstream:
  https://bugzilla.gnome.org/show_bug.cgi?id=794856#c16
@nh2
gst_all_1.gst-plugins-bad: Fix gsm dependency
7174e88
@nh2
gst_all_1.gst-plugins-bad: Fix rtmp dependency
3be4c04
@nh2 nh2 force-pushed the gstreamer-1.15.1 branch from b84a8ad to 3be4c04 Compare June 19, 2019 10:44
@nh2
Copy link
Contributor Author

nh2 commented Jun 19, 2019

@GrahamcOfBorg build gst_all_1

@nh2 nh2 merged commit a4c5d37 into NixOS:master Jun 20, 2019
jtojnar added a commit to jtojnar/nixpkgs that referenced this pull request Jun 25, 2019
@jtojnar
meson: enable auto_features by default
e529ec3 
We disabled them by default when they were introduced in

NixOS#42994

but enabling them probably makes more sense:

NixOS#54398
NixOS#63491
@nixos-discourse
Copy link

nixos-discourse commented Jul 3, 2019

This pull request has been mentioned on Nix community. There might be relevant details there:

https://discourse.nixos.org/t/what-are-your-goals-for-19-09/2875/29

@nixos-discourse
Copy link

nixos-discourse commented Jul 31, 2019

This pull request has been mentioned on Nix community. There might be relevant details there:

https://discourse.nixos.org/t/nixos-weekly-11-nixery-nixfmt-and-cachix-releases-nixcon-2019-tickets-a-job-and-first-impressions-post/3591/1

@nh2 nh2 mentioned this pull request Oct 26, 2019
Closed
@jtojnar jtojnar mentioned this pull request Apr 9, 2020
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@jtojnar jtojnar jtojnar approved these changes

+1 more reviewer

@matthewbauer matthewbauer matthewbauer approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

8.has: clean-up This PR removes packages or removes other cruft 10.rebuild-darwin: 101-500 This PR causes between 101 and 500 packages to rebuild on Darwin. 10.rebuild-linux: 501-1000 This PR causes many rebuilds on Linux and should normally target the staging branches. 10.rebuild-linux: 501+ This PR causes many rebuilds on Linux and should normally target the staging branches.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@nh2 @jtojnar @adisbladis @nixos-discourse @matthewbauer @GrahamcOfBorg