Native video support for AV1 by emilk · Pull Request #7557 · rerun-io/rerun

emilk · 2024-10-01T13:37:22Z

What

Part of More native Video support #7298

What

Supports native decoding of AV1 videos.

Downsides: it is extremely slow in debug builds

In release builds it is ok, but there is still A LOT of performance on the table.

TODO before merging

Update MSRV to Rust 1.79 #7563
Re-add nasm to pixi.toml, because it is needed to compile rav1d
Make the asm-features of rav1d opt-in so users don't need nasm to compile rerun
Put it behind a feature flag, in case compiling rav1d is difficult on some platforms
Fix error handling so we don't crash on bad videos
~~Fix performance of debug builds, if possible~~
- dav1d is always fast, but rav1d is super-slow in debug builds
Or show error message in debug builds
Review moved mp4 demux code to see nothing was lost

Proof

native-video.mp4

Checklist

I have read and agree to Contributor Guide and the Code of Conduct
I've included a screenshot or gif (if applicable)
I have tested the web demo (if applicable):
- Using examples from latest main build: rerun.io/viewer
- Using full set of examples from nightly build: rerun.io/viewer
The PR title and labels are set such as to maximize their usefulness for the next release's CHANGELOG
If applicable, add a new check to the release checklist!
If have noted any breaking changes to the log API in CHANGELOG.md and the migration guide

To run all checks from main, comment on the PR with @rerun-bot full-check.

github-actions · 2024-10-01T13:37:56Z

Deployed docs

Commit	Link
`4ec11fa`	https://landing-9oxg78mpl-rerun.vercel.app/docs

Wumpf

preliminary round or reviewing. excited about this!

Cargo.toml

crates/store/re_video/examples/frames.rs

crates/store/re_video/src/decode/av1.rs

Wumpf · 2024-10-04T09:53:18Z

crates/store/re_video/src/demux/mod.rs

@@ -0,0 +1,268 @@
+//! Video demultiplexing.


todo note to self: review this later again, checking what has moved / is new etc.. This stuff changed quite a bit so there's a risk of regressing progress, should keep an eye on this

I can make a separate PR where I just move the code on main, and then we can rebase and compare

crates/viewer/re_renderer/Cargo.toml

crates/viewer/re_renderer/src/video/decoder/native_av1.rs

Wumpf

much simpler decoder now, nice!

Looking good, but I stick with my usual line of complaining about seqcst: Seqcst is a bad default and hard to deal with imho (Mara has a really nice section about that in her book if you want to learn more check https://marabos.nl/atomics/memory-ordering.html#common-misconceptions -> Myth: Sequentially consistent memory ordering is a great default and is always correct.)

crates/store/re_video/src/decode/av1.rs

# Conflicts: # crates/store/re_video/src/lib.rs # crates/store/re_video/src/mp4.rs # crates/viewer/re_viewer_context/src/cache/video_cache.rs

jprochazk and others added 5 commits September 3, 2024 11:56

temp

4176571

temp

2e648eb

temp

a6a1f6f

Use rav1d from our GitHub

a4ecab1

Merge branch 'main' into emilk/native-av1-decode

c100a62

emilk added 5 commits October 1, 2024 16:13

Work on getting it to compile

308df1c

Almost got it to compile!

0b2c27c

Get it compiling

1bd8b56

Add profiling scopes

c5d40f0

IT WORKS!

ae6656a

emilk mentioned this pull request Oct 2, 2024

More native Video support #7298

Closed

emilk added 📺 re_viewer affects re_viewer itself feat-video anything video decoding, player, querying, data modelling of videos etc. labels Oct 2, 2024

emilk added 4 commits October 2, 2024 07:18

Add a bunch of log statements and profile scopes

ab17e38

Always optimize wpgu and re_video

0b86e60

Range-check input data

d5dee3c

Add nasm to pixi

b197cc3

emilk added the include in changelog label Oct 3, 2024

emilk added 3 commits October 3, 2024 15:57

Merge branch 'main' into emilk/native-av1-decode

9f18e9c

Merge branch 'main' into emilk/native-av1-decode

aeaf137

Remove unused enum Codec

d98b785

emilk force-pushed the emilk/native-av1-decode branch from a03b115 to d98b785 Compare October 3, 2024 14:02

emilk added 7 commits October 3, 2024 16:26

Add the video_av1 opt-in feature flag

72e6f21

Fix web build

c5f93ef

Fix examples (I think)

b53b6ef

Fix re_video example

fb5a7df

Switch to https://github.com/rerun-io/rav1d

ec5384a

Update rav1d and use it under its own name

9b19f3c

refactor some code

c5c9c16

toml fmt

bfaa7f4

emilk changed the title ~~Native AV1 decoding~~ Native video support for AV1 Oct 4, 2024

Wumpf self-requested a review October 4, 2024 08:51

update to latest rav1d with windows fix

e8c03f0

Wumpf reviewed Oct 4, 2024

View reviewed changes

emilk added 4 commits October 4, 2024 13:54

Remove zeroed_texture

aa26f98

merge imports

4a58dc0

Remove TODO

585a8ad

Remove unused import

5558071

emilk mentioned this pull request Oct 4, 2024

Copy dav1d as the Rust API memorysafety/rav1d#1362

Closed

emilk added 8 commits October 4, 2024 14:31

Fix example

3c187e8

Document the frame example

3163a70

Remove confused docstring

b7a6e34

Make decoder use &mut self to avoid certain data races

baca5a8

Don't break after receiving a Flush when waiting for Reset

8f3529d

Simplify the AV1 decoder thread comms, removing a data race

833a11d

toml-fmt

29938fc

Clarify handling of Error::Again

ecd1a45

emilk mentioned this pull request Oct 7, 2024

Native AV1 decoding via rav1d/dav1d #7605

Closed

7 tasks

emilk marked this pull request as ready for review October 7, 2024 06:58

Wumpf approved these changes Oct 7, 2024

View reviewed changes

Wumpf reviewed Oct 7, 2024

View reviewed changes

crates/store/re_video/src/decode/av1.rs Outdated Show resolved Hide resolved

emilk added 6 commits October 7, 2024 11:58

Disable video playback in debug builds

c376098

Change atomic Ordering, and add comments

e5a416d

Update demux code from main

9ad20e8

Merge branch 'main' into emilk/native-av1-decode

b3ff254

# Conflicts: # crates/store/re_video/src/lib.rs # crates/store/re_video/src/mp4.rs # crates/viewer/re_viewer_context/src/cache/video_cache.rs

Comment on dav1d settings

16f8d95

Fix web build

4ec11fa

emilk merged commit 2ce1c82 into main Oct 7, 2024

emilk deleted the emilk/native-av1-decode branch October 7, 2024 10:38

Conversation

emilk commented Oct 1, 2024 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What

What

TODO before merging

Proof

Checklist

Uh oh!

github-actions bot commented Oct 1, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Deployed docs

Uh oh!

Wumpf left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Wumpf Oct 4, 2024

Choose a reason for hiding this comment

Uh oh!

emilk Oct 6, 2024

Choose a reason for hiding this comment

Uh oh!

emilk Oct 6, 2024

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Wumpf left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

emilk commented Oct 1, 2024 •

edited by github-actions bot

Loading

github-actions bot commented Oct 1, 2024 •

edited

Loading

Wumpf left a comment •

edited

Loading