Repository navigation
Research: XR Support #677
Description
Description
This is a research ticket for gathering information on how to implement XR support in postprocessing.
The rendering workflow in v7 consists of using a RenderPipeline that uses a WebGLRenderer/WebGPURenderer to render individual passes. Three's WebXR support requires the use of WebGLRenderer.setAnimationLoop and it uses a special XR camera for the active XR session. So some parts definitely need to be changed but the actual scope of the changes needs to be specified before VR can be implemented.
Implementation Details
- TBD
Relevant sources
- Unregister the system / unload effects #216 (comment)
- Post-processing with WebXR mrdoob/three.js#18846 (comment)
- Post-processing with WebXR mrdoob/three.js#18846 (comment)
- WebGLRenderer: Improve offscreen rendering in WebXR. mrdoob/three.js#26160
- https://threejs.org/docs/?q=vr#manual/en/introduction/How-to-create-VR-content
- https://threejs.org/docs/?q=xr#api/en/renderers/webxr/WebXRManager
- https://developer.mozilla.org/en-US/docs/Web/API/WebXR_Device_API/Rendering
Additional sources
- https://github.com/orgs/angular-threejs/discussions/7
- https://web.archive.org/web/20220804162612/https://twitter.com/Cody_J_Bennett/status/1482585611781480448
- https://codesandbox.io/p/sandbox/webxr-postprocessing-forked-8hzc9l?file=%2Fsrc%2Findex.js
- Adding support for WebXR react-postprocessing#128
Reacted by Sacha aka. glypse, Fabien Benetou, Jonathan Chemla, Ash, Kris Baumgarter and Micheal Parks
Reacted by Fabien Benetou and Jonathan Chemla
Activity
[-]Research: VR Support[/-][+]Research: XR Support[/+]issimissimo commentedon Jun 18, 2025
vanruesc commentedon Jun 19, 2025
issimissimo commentedon Jun 20, 2025
Let me explain:
1 - I'm using V6, so I'm talking about it right now.
I'm using both "setAnimationLoop" and "requestAnimationFrame" with "postprocessing" in a desktop web app, and both them work.
That's why I had idea that there should be no issue on it for XR, if "setAnimationLoop" is mandatory, and that's why I suspect that the problem is about the xr camera.
2 - By now, my suggestion would be to use another canvas, with another renderer, to suverimpose the "postprocessing" render onto the top of the underliying scene, so to be able to use a standard camera for rendering, instead of the xr camera.
A bit tricky, but I did a test and it work.
Next step is to sync the standard camera with the xr camera, but it's not so hard.
Maybe I'm wrong, or maybe it can give you some idea.
Btw many thanks!!!
vanruesc commentedon Jun 21, 2025
I see. This ticket is about v7 and there are no plans to add XR support to v6. Sorry for the confusion - I've added the v7 milestone now.
Well, yes, no question about that. Postprocessing needs to be aware of the array camera and XR state.
There is no need for an additional renderer. Postprocessing can use the same renderer since it uses its own internal render targets and doesn't actually use a camera to render the effects. (The fullscreenCamera is just a dummy.)
With XR, the scene is rendered twice - once for each eye, with slightly different camera settings and the results are rendered into the left half and the right half of a framebuffer respectively. Postprocessing must again be aware of this and also render the effects for the left eye and the right eye while respecting the different camera settings. Blending the result with the XR output frame is not the problem.
I appreciate your input and I hope this clears things up a bit.
issimissimo commentedon Jun 21, 2025
Yes, I see. Many thanks for your nice and detailed reply.