-
Notifications
You must be signed in to change notification settings - Fork 29.7k
Description
Our new Gaussian blur performs a downsample pass before computing the blur. In order to achieve acceptable quality results, the appropriate mip levels must be available for sampling (see this comment for more detail).
For ui.Images, the full mipmaps are always available, and so a reasonable result is possible by setting the mip filter to MipFilter::kNearest or MipFilter::kLinear. However, we don't automatically generate mipmaps for layer textures (because generating mipmaps is expensive and the vast majority of the time doing so is unnecessary). This means that the new Gaussian blur will not render backdrop filters or SaveLayer blurs with acceptable quality unless we generate the necessary mip levels.
When an Impeller texture is allocated, the max number of mip levels is specified via TextureDescriptor::mip_count. This means that, ideally, we can set the mip count to the maximum mip value that we'll sample from. Something that's important to note here is that Textures with a mipmap take up 50% more space than textures without one.
So can we work out the optimal mip_count before we allocate the EntityPass backdrop texture? Yes! But we need to ensure that textures with different mip_counts count as unique in the texture cache.
Changes to the FilterContents protocol
So we'll need to add a way to query the max necessary mip level of a filter input from a FilterContents, and then implement this for the new Gaussian blur (this calculation is detailed below under "How to compute the max mip level"). The signature might look something like virtual size_t FilterContents::GetMaxRequiredMipCount() const.
An important observation is that this can be computed without knowing the bounds of the future FilterInput that will be given to the FilterContents later on.
Tracking the optimal mip count
We need to figure out what mip_count is appropriate for the EntityPass texture before it's allocated, of course (which doesn't happen until render time -- see the two locations where CreateRenderTarget is called within entity_pass.cc). To do this, we can track a size_t max_mip_count field in EntityPass and update it as the pass is built by the Canvas.
The two filters we need to worry about are:
- SaveLayer image filters. For these, we need to set the
max_mip_countof the newEntityPasstoFilterContents::GetMaxRequiredMipCount(). - SaveLayer backdrop filters. For these, we need to set the
max_mip_countof the parentEntityPasstoFilterContents::GetMaxRequiredMipCount()if it's greater than the current value ofmax_mip_count.
Generating the mipmaps
We can lazily generate mipmaps in GaussianBlurFilterContents::RenderFilter. If Texture::NeedsMipmapGeneration() returns false for the input snapshot texture, then simply:
- use the Impeller context to create a new command buffer with
context->CreateCommandBuffer(), - create a blit pass with
buffer->CreateBlitPass(), - append a command generate the mipmap with
pass->GenerateMipmap(), - encode the pass, and
- submit the command buffer.
Mipmap lifecycle management
Lastly, we need a way to invalidate mipmaps that no longer accurately represent the content of the base level.
impeller::Texture already has a field to track this: mipmap_generated_. We can probably just add an InvalidateMipmap() that sets this to false. This needs to be called on the EntityPass texture after any RenderPass which attaches the EntityPass texture has been encoded (otherwise unnecessary blit passes may be created for RenderPasses that happen to read from the resolve texture).
How to compute the max mip level for a given Gaussian blur invocation
In the blur, we sample a curve to compute a value that's used to scale the image. I'll refer to this value as the scalefactor. 0 means the resulting image is 0x0 texels, and 1 means the image retains its original size.
So then how do we map it to the mip level?
miplevel=0 is the original image.
miplevel=1 is half the size of miplevel=0.
miplevel=2 is half the size of miplevel=1.
And so on.
Therefore:
For miplevel=0, the perfect scalefactor is 1 (no scale)
For miplevel=1, the perfect scalefactor is 1/2
For miplevel=2, the perfect scalefactor is 1/4
And so on.
And so the relationship is just scalefactor = 1/(2^miplevel).
Solving for miplevel gives us miplevel = log(1/scalefactor)/log(2).
So if we want to sample with the MipFilter set to kNearest, we only need to generate mip level round( log(1/scalefactor)/log(2) ).
And if we want to generate two mip levels and emulate MipFilter=kLinear (probably not necessary for acceptable results), we'll need to generate both mip level floor( log(1/scalefactor)/log(2) ) and mip level ceil( log(1/scalefactor)/log(2) ).