Skip to content

matchesGoldenFile when imported by integration_test routes to the host system #160043

@matanlurey

Description

@matanlurey

Would fix #143299.

The implementation of matchesGoldenFile is a wrapper around the current goldenFileComparator, and works like this:

if (autoUpdateGoldenFiles) {
  await goldenFileComparator.update(testNameUri, buffer);
  return null;
}
try {
  final bool success = await goldenFileComparator.compare(buffer, testNameUri);
  return success ? null : 'does not match';
} on TestFailure catch (ex) {
  return ex.message;
}

The default goldenFileComparator is TrivialComparator, which is effectively a NOP, but the one used by default by most Flutter users is LocalFileComparator, which both uses dart:ui and dart:io, and the custom comparators written as part of the internal flutter_goldens pacakge, which also uses dart:ui (through flutter_test) and dart:io.

When running as an integration test, access to the local (host) file system is not guaranteed (it will happen to work on desktop, but fail on iOS, Android, and Web). We would like (1) a routing goldenFileComparator that knows to talk to a remote server (i.e. the driver script) and for (2) the actual implementation of the comparator to live (and run) on the host as a plain Dart-VM script that does not have access to dart:ui.

Another option, which is perhaps more palatable, is to continue doing comparisons on the device, and only use the host for IO access (i.e. read/write files, or make calls into Skia Gold). This would be a bit more complicated in some ways and easier in others.

There is something called WebGoldenComparator that claims to do exactly this, so maybe it could conceptually be reused for non-web use cases and renamed ExternalWebGoldenComparator (or such).


My proposal would be to add this behind an experimental runtime flag in order to iterate without breaking users.

Metadata

Metadata

Assignees

Labels

P1High-priority issues at the top of the work listc: proposalA detailed proposal for a change to Flutterf: integration_testThe flutter/packages/integration_test pluginfyi-webFor the attention of Web platform team

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions