Skip to content

Conversation

@gnprice
Copy link
Member

@gnprice gnprice commented Jun 20, 2024

Fixes part of #150524.

As part of #150524, I'll be sending a couple of PRs that simplify and reduce the bookkeeping we do for tracking relayout boundaries. To measure the performance effect of that reduced bookkeeping, I wrote some microbenchmarks that exercise that tracking.

This PR adds the microbenchmarks. That should give them a clean baseline, before the PRs that change how the bookkeeping works and are meant to change the results of the microbenchmarks.

The "reparenting accordion" benchmark is a bit pathological: it involves N GlobalKeys all getting reparented on each frame. It takes quadratic time, and I think we're OK with that. Still, constant-factor improvements and regressions there seem worth knowing about, because real apps may contain smaller versions of the same pattern.

Pre-launch Checklist

If you need help, consider asking for advice on the #hackers-new channel on Discord.

Fixes part of flutter#150524.

As part of flutter#150524, I'll be sending a couple of PRs that simplify
and reduce the bookkeeping we do for tracking relayout boundaries.
To measure the performance effect of that reduced bookkeeping, I
wrote some microbenchmarks that exercise that tracking.

This PR adds the microbenchmarks.  That should give them a clean
baseline, before the PRs that change how the bookkeeping works and
are meant to change the results of the microbenchmarks.

The "reparenting accordion" benchmark is a bit pathological:
it involves N `GlobalKey`s all getting reparented on each frame.
It takes quadratic time, and I think we're OK with that.
Still, constant-factor improvements and regressions there seem
worth knowing about, because real apps may contain smaller
versions of the same pattern.
@gnprice
Copy link
Member Author

gnprice commented Jun 20, 2024

The failures in Mac build_ios_framework_module_test and Mac tool_integration_tests_1_4 were timeouts, and seem like they shouldn't be related — those checks don't sound like they run benchmarks. Re-running those checks.

Copy link
Contributor

@LongCatIsLooong LongCatIsLooong left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As a microbenchmark for relayout boundaries, I feel testing using widgets will be way too noisy. Clearing the relayout boundary for each node takes no more than a few clockcycles I would assume, but reparenting an Element tree is much much more expensive than that.

@LongCatIsLooong
Copy link
Contributor

I guess you could do the same benchmark tests using the rendering layer only, by putting markNeedsDirty and flushLayout in a tight loop. I believe ultimately it's the overall layout performance that we care about, so existing layout benchmarks should reflect the performance impact of the layout boundary implementation change you'd like to make?

@gnprice
Copy link
Member Author

gnprice commented Jun 26, 2024

Sure, I can try reworking these to be just in terms of render objects with no widgets. (In your second comment, do you mean markNeedsLayout, or is there something elsewhere I'm missing?)

That should make the benchmarks even more micro-, so I expect it'll make the measured speedups from my upcoming changes even steeper. I guess I went for widgets initially because I didn't know if the even-more-micro-benchmark version might be seen as less realistic, and so less convincing.

@gnprice
Copy link
Member Author

gnprice commented Jun 26, 2024

Which existing benchmarks are the ones you'd look at for evaluating a layout change?

Last week I tried looking at the benchmark results in Skia Perf, but found it hard to browse among different benchmarks.

@LongCatIsLooong
Copy link
Contributor

LongCatIsLooong commented Jun 27, 2024

do you mean markNeedsLayout

Ah sorry yeah I meant markNeedsLayout

Which existing benchmarks are the ones you'd look at for evaluating a layout change?

Something like microbenchmarks/lib/stocks/layout_bench.dart (https://flutter-flutter-perf.skia.org/e/?begin=1719363564&end=1719449964&queries=sub_result%3Dstock_layout_iteration)? It's kinda macro-y to me.

I think it's probably justifiable to land the relayout boundary change with no additional benchmarks, since it would remove the treewalk no?

@gnprice
Copy link
Member Author

gnprice commented Jun 27, 2024

Cool, thanks. I just tried that benchmark on my draft changes, and any effect seemed to be within the noise (it varied a few percent from run to run, due perhaps to random apps running background tasks on my phone). On the microbenchmarks in this PR, the effect is much bigger, like 40% for relayout_boundary_toggle and 25-30% for the other two.

As you say, clearing (or updating) the relayout boundary for each node should be very fast, so I expect the speedup to only be material when there's a large same-relayout-boundary subtree that the current logic has to sweep through. Also only when it actually does have to do that sweep — so when the relayout boundary changes, or that subtree is removed from the render tree. I'm not sure if the stocks benchmark exercises those cases, but if it does it probably just doesn't have any really large same-relayout-boundary subtrees.

I think it's probably justifiable to land the relayout boundary change with no additional benchmarks, since it would remove the treewalk no?

Yeah, I wouldn't disagree. Indeed it removes the treewalks — the two in layout when the relayout boundary changes, and then the one in dropChild. So just from reading the changes, even without any benchmark results, one would expect it certainly ought to be a speedup, even if usually a small one in practice.

I think converting these to pure rendering-layer terms would be a fun exercise for me, though, so I'll make a go of it even if only for my own edification. But it might be a few days before I get to that.

I believe they'd be the first examples in the tree of a benchmark of that kind, too. So that might be a bonus reason to land at least one of them.

@gnprice
Copy link
Member Author

gnprice commented Aug 20, 2024

Status update: this is in the same state as the related #150905:

I'm still planning to return to this. The summer has been a busy time, but I'm hoping to pick it back up in September or October.

@Piinks
Copy link
Contributor

Piinks commented Oct 22, 2024

(PR triage): Since we have not heard back on this or #150905 I am going to close this to remove it from the review queue. I have added the framework label though, so it does end up in the right team review queue if you decide to reopen it. If you'd like to reopen these PRs, but not have them in the review queue, please mark them as a draft.
Thanks!

@Piinks Piinks closed this Oct 22, 2024
@Piinks Piinks added framework flutter/packages/flutter repository. See also f: labels. team: benchmark Performance issues found by inspecting benchmarks labels Oct 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

framework flutter/packages/flutter repository. See also f: labels. team: benchmark Performance issues found by inspecting benchmarks

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants