Skip to content

Comments

Add real-time video processing use case#249

Merged
anssiko merged 1 commit intomainfrom
video-processing-uc
Jan 27, 2022
Merged

Add real-time video processing use case#249
anssiko merged 1 commit intomainfrom
video-processing-uc

Conversation

@anssiko
Copy link
Member

@anssiko anssiko commented Jan 20, 2022

In https://www.w3.org/2022/01/13-webmachinelearning-minutes.html#t06 it was suggested we add a real-time video processing use case. Here's a proposal for that.

A few questions:

  • I put this in the "Framework Use Cases" section. Not a perfect fit but I wanted to differentiate from the "Application Use Cases" that in the context of this spec refer to various ML models. Maybe we can find a better name for the "Framework Use Cases" section?
  • In this proposal, I did not explicitly define what makes up "the media pipeline" given the details of that are still WIP. Should we instead refer informatively the specs that we currently think will make up this pipeline and maintain this use case as our understanding evolves?

@anssiko
Copy link
Member Author

anssiko commented Jan 20, 2022

(It seems pr-preview did not run even if the PR passes the checks, any idea why @dontcallmedom?)

@dontcallmedom
Copy link
Contributor

dontcallmedom commented Jan 20, 2022

I don't know re pr-preview, and don't think I have an easy way to debug it either unfortunately

huningxin
huningxin reviewed Jan 21, 2022
View reviewed changes
index.bs
### Integration with real-time video processing ### {#usecase-real-time-video-processing}

The user experience of WebRTC-based video conferencing is enhanced using real-time video processing. For example, background blur implemented using a [[#usecase-segmentation]] model blurs the background in the user's live camera feed. To satisfy the performance requirements of this use case, the WebNN API integrates with primitives from other Web APIs that make up the media pipeline to allow WebNN API-based transformation of real-time video streams.

Copy link
Contributor

@huningxin huningxin Jan 21, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should the full-GPU video processing pipeline be a requirement? For other Web APIs, should we mention WebGPU and WebGL? Because it should require the pre/post-processing are also on the GPU.

Copy link
Member Author

@anssiko anssiko Jan 25, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the proposal, that was the key question on my mind also. Let's discuss this on our call to capture the whole group's perspective.

@anssiko
Copy link
Member Author

anssiko commented Jan 27, 2022

Reviewed on WebML WG Teleconference – 27 January 2022: https://www.w3.org/2022/01/27-webmachinelearning-minutes.html#t15

@anssiko anssiko merged commit 2994243 into main Jan 27, 2022
@anssiko anssiko deleted the video-processing-uc branch January 27, 2022 16:06
github-actions bot added a commit that referenced this pull request Jan 27, 2022
@anssiko @github-actions
Merge pull request #249 from webmachinelearning/video-processing-uc
8110df6 
SHA: 2994243
Reason: push, by @anssiko

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
@huningxin huningxin mentioned this pull request Apr 27, 2022
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@huningxin huningxin huningxin left review comments

@dontcallmedom dontcallmedom dontcallmedom approved these changes

@wchao1115 wchao1115 Awaiting requested review from wchao1115

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@anssiko @dontcallmedom @huningxin