Fix abi key for local golden file testing #148072
Conversation
Piinks
added
a: tests
goderbauer
left a comment
goderbauer
left a comment
There was a problem hiding this comment.
LGTM
Thank you for digging into this!
There was a problem hiding this comment.
Is this also relevant for _getKeysJSON?
There was a problem hiding this comment.
It is not, since those keys are sorted by Gold when they are ingested. It matters here because we are using the keys to generate an md5 sum to look up the image, and being out of order means we get the wrong result.
There was a problem hiding this comment.
"alphabetically, after grouping by letter case" is just "sort by Unicode scalar value" which is just the default string sort in Dart, so one way to do this would be to create the map first, then get the keys, sort them, and add the key/value pairs to a new map in the sorted order. That would avoid future bugs when someone adds a new key manually and gets the order wrong. It would cost a few CPU cycles and some memory though.
There was a problem hiding this comment.
ah, just saw the comment below. Yeah, the order here is just the default order for Dart sorting.
void main() {
Map<String, int> test = {'a': 3, 'B': 2, 'A': 1, 'b': 4};
print(test);
print(test.keys.toList()..sort());
Map<String, int> sorted = {};
for (String key in test.keys.toList()..sort()) {
sorted[key] = test[key]!;
}
print(sorted);
}->
{a: 3, B: 2, A: 1, b: 4}
[A, B, a, b]
{A: 1, B: 2, a: 3, b: 4}
There was a problem hiding this comment.
This week I learned Dart does that by default! Awesome! Yeah I'll update this to sort keys so we don't have to worry about it recurring. :)
There was a problem hiding this comment.
Could we add an assert here (or a test elsewhere) that enforces the sorting rules and breaks if this is messed up in the future?
There was a problem hiding this comment.
I can, it would be pretty complicated though (and probably not today) to account for all the cases where we do or don't include a given key. We don't add keys very often, I was kind of hoping the comment would suffice. WDYT?
There was a problem hiding this comment.
It does seem a touch fragile. Would it be worth adding some code that somehow alphabetizes the map keys first? If so, then I would imagine it would be easier to add a few tests for that logic: input(map) output(md5 hash).
There was a problem hiding this comment.
Isn't this just:
assert(() {
List sorted = keys.keys.toList()..sort();
return listEquals(sorted, keys.keys.toList());
}());Although, maybe doing this on every golden check is kinda slowing things down? I'd be okay with leaving this out as well.
Edit: Thinking about this some more, I do think we should guard against this error a little stronger than just a comment...
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
There was a problem hiding this comment.
Another option would be to just throw these key-values into a self-ordering data structure, e.g. https://main-api.flutter.dev/flutter/dart-collection/SplayTreeMap-class.html
There was a problem hiding this comment.
Oh! Will dart sort by case before alphabetical order? I didn't think it would.. let me check
flutter/flutter@d2da1b2...39651e8 2024-05-15 [email protected] Roll Flutter Engine from d35a1a603c80 to bf1c6da0dd31 (1 revision) (flutter/flutter#148369) 2024-05-15 [email protected] Roll Flutter Engine from 55c62ff82c7e to d35a1a603c80 (4 revisions) (flutter/flutter#148367) 2024-05-15 [email protected] Roll Flutter Engine from a1d930a3a84d to 55c62ff82c7e (3 revisions) (flutter/flutter#148365) 2024-05-14 [email protected] Roll Flutter Engine from dfb5871260a6 to a1d930a3a84d (2 revisions) (flutter/flutter#148360) 2024-05-14 [email protected] Roll Flutter Engine from 1b508a071c96 to dfb5871260a6 (1 revision) (flutter/flutter#148356) 2024-05-14 49699333+dependabot[bot]@users.noreply.github.com Bump codecov/codecov-action from 4.3.1 to 4.4.0 (flutter/flutter#148355) 2024-05-14 [email protected] Fix memory leaks in `SnackBar` (flutter/flutter#147212) 2024-05-14 [email protected] Fix memory leaks in open upwards page transition (flutter/flutter#148046) 2024-05-14 [email protected] Fixes semantics ordering when there are multiple TextFields with pref� (flutter/flutter#148267) 2024-05-14 [email protected] Roll Flutter Engine from ae9ff69a0840 to 1b508a071c96 (2 revisions) (flutter/flutter#148351) 2024-05-14 [email protected] Add missing InputDecorator.hintText tests (flutter/flutter#148113) 2024-05-14 [email protected] Fix abi key for local golden file testing (flutter/flutter#148072) 2024-05-14 [email protected] Maintain the same layout constraints for item in the ReorderableList during dragging as before dragging. (flutter/flutter#147863) 2024-05-14 [email protected] test material text field example (flutter/flutter#147864) 2024-05-14 [email protected] Move toggleable to widget layer (flutter/flutter#148272) 2024-05-14 [email protected] Roll Flutter Engine from 08b44d906fab to ae9ff69a0840 (1 revision) (flutter/flutter#148341) 2024-05-14 [email protected] add another print trace to runInView (flutter/flutter#148337) 2024-05-14 [email protected] Roll Flutter Engine from 7bf865774d06 to 08b44d906fab (4 revisions) (flutter/flutter#148338) If this roll has caused a breakage, revert this CL and stop the roller using the controls here: https://autoroll.skia.org/r/flutter-packages Please CC [email protected],[email protected] on the revert to ensure that a human is aware of the problem. To file a bug in Packages: https://github.com/flutter/flutter/issues/new/choose To report a problem with the AutoRoller itself, please file a bug: https://issues.skia.org/issues/new?component=1389291&template=1850622 Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md
flutter/flutter@d2da1b2...39651e8 2024-05-15 [email protected] Roll Flutter Engine from d35a1a603c80 to bf1c6da0dd31 (1 revision) (flutter/flutter#148369) 2024-05-15 [email protected] Roll Flutter Engine from 55c62ff82c7e to d35a1a603c80 (4 revisions) (flutter/flutter#148367) 2024-05-15 [email protected] Roll Flutter Engine from a1d930a3a84d to 55c62ff82c7e (3 revisions) (flutter/flutter#148365) 2024-05-14 [email protected] Roll Flutter Engine from dfb5871260a6 to a1d930a3a84d (2 revisions) (flutter/flutter#148360) 2024-05-14 [email protected] Roll Flutter Engine from 1b508a071c96 to dfb5871260a6 (1 revision) (flutter/flutter#148356) 2024-05-14 49699333+dependabot[bot]@users.noreply.github.com Bump codecov/codecov-action from 4.3.1 to 4.4.0 (flutter/flutter#148355) 2024-05-14 [email protected] Fix memory leaks in `SnackBar` (flutter/flutter#147212) 2024-05-14 [email protected] Fix memory leaks in open upwards page transition (flutter/flutter#148046) 2024-05-14 [email protected] Fixes semantics ordering when there are multiple TextFields with pref� (flutter/flutter#148267) 2024-05-14 [email protected] Roll Flutter Engine from ae9ff69a0840 to 1b508a071c96 (2 revisions) (flutter/flutter#148351) 2024-05-14 [email protected] Add missing InputDecorator.hintText tests (flutter/flutter#148113) 2024-05-14 [email protected] Fix abi key for local golden file testing (flutter/flutter#148072) 2024-05-14 [email protected] Maintain the same layout constraints for item in the ReorderableList during dragging as before dragging. (flutter/flutter#147863) 2024-05-14 [email protected] test material text field example (flutter/flutter#147864) 2024-05-14 [email protected] Move toggleable to widget layer (flutter/flutter#148272) 2024-05-14 [email protected] Roll Flutter Engine from 08b44d906fab to ae9ff69a0840 (1 revision) (flutter/flutter#148341) 2024-05-14 [email protected] add another print trace to runInView (flutter/flutter#148337) 2024-05-14 [email protected] Roll Flutter Engine from 7bf865774d06 to 08b44d906fab (4 revisions) (flutter/flutter#148338) If this roll has caused a breakage, revert this CL and stop the roller using the controls here: https://autoroll.skia.org/r/flutter-packages Please CC [email protected],[email protected] on the revert to ensure that a human is aware of the problem. To file a bug in Packages: https://github.com/flutter/flutter/issues/new/choose To report a problem with the AutoRoller itself, please file a bug: https://issues.skia.org/issues/new?component=1389291&template=1850622 Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md
We have fiddled with this a bunch, and there is definitively no way to reliably include this, so I am removing it. Since CI does not cover all possible values, we can't consistently look up images for local testing. We thought removing it from the look up would work fine, and it did for most tests, but there were still some that could not find an image without it. A brief history on the abi key - added in #143621 - disabled in #148023 - added back in #148072 - we thought there was only an issue with alphabetizing keys - removed from local image look up in #149696 I updated the docs page to also discuss what makes a good key and what does not based on what we learned here.
5 participants
Fixes #148022
Pre-launch Checklist
///).If you need help, consider asking for advice on the #hackers-new channel on Discord.