This repository was archived by the owner on Feb 25, 2025. It is now read-only.
Ensure that Scene::toImage renders texture backed images. #6636
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
TL;DR: Offscreen surface is created on the render thread and device to host transfer performed there before task completion on the UI thread. While attempting to snapshot layer trees, the engine was attempting to use the IO thread context. The reasoning was that this would be safe to do because any textures uploaded to the GPU as a result of async texture upload would have originated from this context and hence the handles would be valid in either context. As it turns out, while the handles are valid, Skia does not support this use-case because cross-context images transfer ownership of the image from one context to another. So, when we made the hop from the UI thread to the IO thread (for snapshotting), if either the UI or GPU threads released the last reference to the texture backed image, the image would be invalid. This led to such images being absent from the layer tree snapshot. Simply referencing the images as they are being used on the IO thread is not sufficient because accessing images on one context after their ownership has already been transferred to another is not safe behavior (from Skia's perspective, the handles are still valid in the sharegroup). To work around these issues, it was decided that an offscreen render target would be created on the render thread. The color attachment of this render target could then be transferred as a cross context image to the IO thread for the device to host tranfer. Again, this is currently not quite possible because the only way to create cross context images is from encoded data. Till Skia exposes the functionality to create cross-context images from textures in one context, we do a device to host transfer on the GPU thread. The side effect of this is that this is now part of the frame workload (image compression, which dominate the wall time, is still done of the IO thread). A minor side effect of this patch is that the GPU latch needs to be waited on before the UI thread tasks can be completed before shell initialization.
Member
Author
|
Tested on GPU and software rendering environments. Traces showing the workload on the different threads is attached (debug). Thanks to @andreidiaconu for the reduced test case: |
jason-simmons
approved these changes
Oct 23, 2018
Member
Author
|
I'll follow up with @brianosman to figure out if the texture stealing API in Skia can be made explicit so that we can transfer the texture backed image to the IO thread for device to host transfer. |
engine-flutter-autoroll
added a commit
to engine-flutter-autoroll/flutter
that referenced
this pull request
Oct 23, 2018
flutter/engine@4c79e42...2586e94 git log 4c79e42..2586e94 --no-merges --oneline 2586e94 Support all combinations of GetRectsForRange styles (flutter/engine#6591) e78f86e Fix mac builds. Only Linux and Windows require default GL proc resolvers. (flutter/engine#6641) 52e48ab Fix Windows embedding. Appears that flutter#6523 or flutter#6525 introduced a bug for embedder scenarios causing the window native library to be incorrectly initialized and thus incapable of correctly resolving GL functions. This change fixes that. (flutter/engine#6624) c9197e4 Roll src/third_party/skia 25837bf17019..b46c4d0925ad (6 commits) (flutter/engine#6640) 324c21a Roll src/third_party/skia 1b07dffd979d..25837bf17019 (1 commits) (flutter/engine#6639) 5dfbc0a Roll src/third_party/skia e023ae7c5540..1b07dffd979d (1 commits) (flutter/engine#6638) 4a18dff Roll src/third_party/skia ff1aeb953faf..e023ae7c5540 (1 commits) (flutter/engine#6637) 427915e Allow FlutterViewController to specify whether its FlutterView is opaque (flutter/engine#6570) 20c805c Ensure that Scene::toImage renders texture backed images. (flutter/engine#6636) 25d0507 Roll buildtools to 5a9e1b3a0b84a2871f20f85fde665e54a894ba72 (flutter/engine#6633) 4f17d7e Roll src/third_party/skia 327955b1ba19..ff1aeb953faf (13 commits) (flutter/engine#6635) cdd592f Reland 'Pass null instead of 'none' locale' (flutter/engine#6632) 2cd8920 Roll src/third_party/skia b1a002e850e1..327955b1ba19 (12 commits) (flutter/engine#6631) edfe024 13771 - iOS dictation bug (flutter/engine#6607) cadf440 Roll src/third_party/skia b9998cdceec7..b1a002e850e1 (13 commits) (flutter/engine#6626) The AutoRoll server is located here: https://autoroll.skia.org/r/flutter-engine-flutter-autoroll Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+/master/autoroll/README.md If the roll is causing failures, please contact the current sheriff, who should be CC'd on the roll, and stop the roller if necessary.
goderbauer
pushed a commit
to flutter/flutter
that referenced
this pull request
Oct 23, 2018
flutter/engine@4c79e42...2586e94 git log 4c79e42..2586e94 --no-merges --oneline 2586e94 Support all combinations of GetRectsForRange styles (flutter/engine#6591) e78f86e Fix mac builds. Only Linux and Windows require default GL proc resolvers. (flutter/engine#6641) 52e48ab Fix Windows embedding. Appears that #6523 or #6525 introduced a bug for embedder scenarios causing the window native library to be incorrectly initialized and thus incapable of correctly resolving GL functions. This change fixes that. (flutter/engine#6624) c9197e4 Roll src/third_party/skia 25837bf17019..b46c4d0925ad (6 commits) (flutter/engine#6640) 324c21a Roll src/third_party/skia 1b07dffd979d..25837bf17019 (1 commits) (flutter/engine#6639) 5dfbc0a Roll src/third_party/skia e023ae7c5540..1b07dffd979d (1 commits) (flutter/engine#6638) 4a18dff Roll src/third_party/skia ff1aeb953faf..e023ae7c5540 (1 commits) (flutter/engine#6637) 427915e Allow FlutterViewController to specify whether its FlutterView is opaque (flutter/engine#6570) 20c805c Ensure that Scene::toImage renders texture backed images. (flutter/engine#6636) 25d0507 Roll buildtools to 5a9e1b3a0b84a2871f20f85fde665e54a894ba72 (flutter/engine#6633) 4f17d7e Roll src/third_party/skia 327955b1ba19..ff1aeb953faf (13 commits) (flutter/engine#6635) cdd592f Reland 'Pass null instead of 'none' locale' (flutter/engine#6632) 2cd8920 Roll src/third_party/skia b1a002e850e1..327955b1ba19 (12 commits) (flutter/engine#6631) edfe024 13771 - iOS dictation bug (flutter/engine#6607) cadf440 Roll src/third_party/skia b9998cdceec7..b1a002e850e1 (13 commits) (flutter/engine#6626) The AutoRoll server is located here: https://autoroll.skia.org/r/flutter-engine-flutter-autoroll Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+/master/autoroll/README.md If the roll is causing failures, please contact the current sheriff, who should be CC'd on the roll, and stop the roller if necessary.
Xavjer
pushed a commit
to Xavjer/flutter
that referenced
this pull request
Oct 24, 2018
flutter/engine@4c79e42...2586e94 git log 4c79e42..2586e94 --no-merges --oneline 2586e94 Support all combinations of GetRectsForRange styles (flutter/engine#6591) e78f86e Fix mac builds. Only Linux and Windows require default GL proc resolvers. (flutter/engine#6641) 52e48ab Fix Windows embedding. Appears that flutter#6523 or flutter#6525 introduced a bug for embedder scenarios causing the window native library to be incorrectly initialized and thus incapable of correctly resolving GL functions. This change fixes that. (flutter/engine#6624) c9197e4 Roll src/third_party/skia 25837bf17019..b46c4d0925ad (6 commits) (flutter/engine#6640) 324c21a Roll src/third_party/skia 1b07dffd979d..25837bf17019 (1 commits) (flutter/engine#6639) 5dfbc0a Roll src/third_party/skia e023ae7c5540..1b07dffd979d (1 commits) (flutter/engine#6638) 4a18dff Roll src/third_party/skia ff1aeb953faf..e023ae7c5540 (1 commits) (flutter/engine#6637) 427915e Allow FlutterViewController to specify whether its FlutterView is opaque (flutter/engine#6570) 20c805c Ensure that Scene::toImage renders texture backed images. (flutter/engine#6636) 25d0507 Roll buildtools to 5a9e1b3a0b84a2871f20f85fde665e54a894ba72 (flutter/engine#6633) 4f17d7e Roll src/third_party/skia 327955b1ba19..ff1aeb953faf (13 commits) (flutter/engine#6635) cdd592f Reland &flutter#39;Pass null instead of &flutter#39;none&flutter#39; locale&flutter#39; (flutter/engine#6632) 2cd8920 Roll src/third_party/skia b1a002e850e1..327955b1ba19 (12 commits) (flutter/engine#6631) edfe024 13771 - iOS dictation bug (flutter/engine#6607) cadf440 Roll src/third_party/skia b9998cdceec7..b1a002e850e1 (13 commits) (flutter/engine#6626) The AutoRoll server is located here: https://autoroll.skia.org/r/flutter-engine-flutter-autoroll Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+/master/autoroll/README.md If the roll is causing failures, please contact the current sheriff, who should be CC&flutter#39;d on the roll, and stop the roller if necessary.
|
when can we expect this fix to be available on the beta channel? |
Member
Author
|
@cbracken Can you comment on the roll to the beta channel? This is already on ToT (was rolled in flutter/flutter@f7f7221) |
Member
|
@donarn we don't make forward-looking statements but I've adjusted the milestone for flutter/flutter#17687 to accurately reflect the set of issues that fall into the same release so you can track rough progress. |
|
totally understood. thanks for the information, and for all of your hard work. |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
5 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
TL;DR: Offscreen surface is created on the render thread and device to host
transfer performed there before task completion on the UI thread.
While attempting to snapshot layer trees, the engine was attempting to use the
IO thread context. The reasoning was that this would be safe to do because any
textures uploaded to the GPU as a result of async texture upload would have
originated from this context and hence the handles would be valid in either
context. As it turns out, while the handles are valid, Skia does not support
this use-case because cross-context images transfer ownership of the image from
one context to another. So, when we made the hop from the UI thread to the IO
thread (for snapshotting), if either the UI or GPU threads released the last
reference to the texture backed image, the image would be invalid. This led to
such images being absent from the layer tree snapshot.
Simply referencing the images as they are being used on the IO thread is not
sufficient because accessing images on one context after their ownership has
already been transferred to another is not safe behavior (from Skia's
perspective, the handles are still valid in the sharegroup).
To work around these issues, it was decided that an offscreen render target
would be created on the render thread. The color attachment of this render
target could then be transferred as a cross context image to the IO thread for
the device to host tranfer.
Again, this is currently not quite possible because the only way to create
cross context images is from encoded data. Till Skia exposes the functionality
to create cross-context images from textures in one context, we do a device to
host transfer on the GPU thread. The side effect of this is that this is now
part of the frame workload (image compression, which dominate the wall time,
is still done of the IO thread).
A minor side effect of this patch is that the GPU latch needs to be waited on
before the UI thread tasks can be completed before shell initialization.
Fixes flutter/flutter#17687