[Flutter GPU] Design: Why have CommandBuffer? #150828

Keeping in mind that we should only surface concepts in Flutter GPU that are expected to provide demonstrable value, I started contemplating whether the CommandBuffer concept is valuable enough to justify having (again) while researching potential synchronization issues and re-analyzing our current usage of Impeller's HAL.

At some point early on, I had actually removed CommandBuffer from Flutter GPU, but eventually decided to add it back while prototyping. So I wanted to record my thoughts and discuss more openly.

TL;DR, I don't suspect there's any compelling performance reason to keep it, but removing it would limit our options for adding convenient blit functions and such.

The CommandBuffer concept is similar to Metal's MTLCommandBuffer, in that it's the top-level command encoding API. Submitted CommandBuffers cannot be reused. This is a completely different concept to e.g. Vulkan's VkCommandBuffer, which is a reusable list of commands (and may reference secondary command buffers).

Risks of removal

Future usability

As we continue adding more features to the API, having a "command encoder" like CommandBuffer may become a clear winner from a usability standpoint.

For example, we don't need to surface everything as a Pass like with Metal. RenderPass obviously makes a lot of sense for tiler GPUs, but BlitPass? Not really.
Rather than surfacing a BlitPass, we should probably just surface simple blit functions on CommandBuffer for generating mipmaps, copying textures to other textures, copying buffers to textures, etc.

Performance

One (minuscule) benefit of CommandBuffer is slightly less overhead when targeting Metal, given that we'd need to create separate MTLCommandBuffers for every pass otherwise.

If we didn't have a command encoder concept like CommandBuffer, we'd pretty much lose the option to surface pass-less blits in the API.

Adding such blit functions to GpuContext directly would be a Very Bad thing, as each call would end up needing to trigger a queue submission.

Current design

As described in #150816, Flutter GPU's current rasterization workflow looks like this:

1. Get the GPU context.
2. Create a CommandBuffer with GpuContext.createCommandBuffer.
3. Create a RenderPass with CommandBuffer.createRenderPass.
4. Set state/pipelines and bind resources with various RenderPass methods.
5. Append draw commands with RenderPass.draw.
6. Submit the CommandBuffer with CommandBuffer.submit (async). All RenderPasses are encoded in the order with which they were created.

One potential point of friction with this design comes from the fact that we automatically encode each RenderPass on CommandBuffer submit. Which means that renderers are forced to define/create all "subpasses" in advance if they wish to encode everything to a single CommandBuffer.

TODO(bdero): In Impeller's 2D renderer, we've tried to exploit the ability to encode RenderPasses out of order the past by e.g. recording all EntityPass to one CommandBuffer. Cross reference the safety issues that have come up in the past while attempting to do this in Impeller's 2D renderer.

However, Impeller's 2D renderer today does not even rely on the ability to choose the RenderPass encoding order. Impeller's 2D renderer happens to encode every RenderPass to its CommandBuffer in creation order.

• For EntityPass subpasses, InlinePassContext creates and separately submits a new CommandBuffer for each new RenderPass. The optional async completion callback is not used.
• For filters that perform extra RenderPasses, the Filter creates a new CommandBuffer and adds a sequence of RenderPasses to it using the ContentContext::MakeSubpass. This utility encodes the RenderPass immediately, and Filters never use this utility to create subpasses within the same CommandBuffer.

How would removing CommandBuffer change the API?

Without a CommandBuffer, the rasterization workflow would change to look like this:

1. Get the GPU context.
2. Create a RenderPass with GpuContext.createRenderPass.
3. Set state/pipelines and bind resources with various RenderPass methods.
4. Append draw commands with RenderPass.draw.
5. For submission, there are a couple of options:
   • Single submission: Submit the RenderPass with RenderPass.submit (async). All RenderPasses are encoded in the order with which they were created.
   • Batch submission: Submit a list of Passes with GpuContext.submit(List<Pass>)

For 5, I'm leaning towards just having GpuContext.submit(List<Pass>) and nothing else.