Skip to content

Linux linux_host_engine_test flakey on gold usage #181399

New issue
New issue
Closed
#182387
Closed
Linux linux_host_engine_test flakey on gold usage#181399
#182387
Labels
P1High-priority issues at the top of the work listc: flakeTests that sometimes, but not always, incorrectly passe: impellerImpeller rendering backend issues and features requeststeam-engineOwned by Engine teamtriaged-engineTriaged by Engine team

Description

@gaaclarke
gaaclarke
opened on Jan 23, 2026

Examples

log

Running command "/b/s/w/ir/cache/builder/engine/src/out/ci/host_release_test/exe.unstripped/flutter_tester --disable-vm-service --enable-impeller --enable-flutter-gpu --impeller-backend=metal --use-test-fonts --icu-data-file-path=/b/s/w/ir/cache/builder/engine/src/out/ci/host_release_test/icudtl.dat --flutter-assets-dir=/b/s/w/ir/cache/builder/engine/src/out/ci/host_release_test/gen/flutter/lib/ui/assets --disable-asset-fonts /b/s/w/ir/cache/builder/engine/src/out/ci/host_release_test/gen/canvas_test.dart.dill" in "/b/s/w/ir/cache/builder/engine/src"
00:00 �[32m+0�[0m: canvas APIs should not crash�[0m

00:00 �[32m+1�[0m: Simple .toImage�[0m

Skia gold comparison failed: Error when running Skia Gold: /b/s/w/ir/cache/gold/goldctl imgtest init --instance flutter --work-dir /b/s/w/ir/cache/builder/engine/src/out/ci/host_release_test/gen/skia_gold_canvas_test.dart_iplr/temp --commit 0ab063e989918180685b63a05eb401b9fbe8352b --keys-file /b/s/w/ir/cache/builder/engine/src/out/ci/host_release_test/gen/skia_gold_canvas_test.dart_iplr/keys.json --failure-file /b/s/w/ir/cache/builder/engine/src/out/ci/host_release_test/gen/skia_gold_canvas_test.dart_iplr/failures.json --passfail
Skia Gold imgtest init failed.
An error occurred when initializing golden file test with
goldctl.


stdout:
stderr: File /b/s/w/ir/cache/builder/engine/src/out/ci/host_release_test/gen/skia_gold_canvas_test.dart_iplr/keys.json does not contain readable JSON object: unexpected end of JSON input

00:00 �[32m+1�[0m�[31m -1�[0m: Simple .toImage �[1m�[31m[E]�[0m�[0m

  Exception: Failed comparison: canvas_test_toImage.png

  flutter/testing/dart/goldens.dart 59:7  ImageComparer.addGoldenImage.<fn>
  dart:async/zone_root.dart 48:47         _rootRunUnary
  dart:async/zone.dart 733:19             _CustomZone.runUnary
  dart:async/future_impl.dart 243:22      _FutureListener.handleError
  dart:async/future_impl.dart 960:47      Future._propagateToListeners.handleError
  dart:async/future_impl.dart 981:13      Future._propagateToListeners
  dart:async/future_impl.dart 738:5       Future._completeErrorObject
  dart:async/future_impl.dart 742:5       Future._completeError

Activity

  1. gaaclarke
    added
    e: impellerImpeller rendering backend issues and features requests
    team-engineOwned by Engine team
    P1High-priority issues at the top of the work list
    c: flakeTests that sometimes, but not always, incorrectly pass
    on Jan 23, 2026
  2. github-merge-queue
    added a commit that references this issue on Jan 23, 2026

    Adds contents of keys file when a skia gold error occurs. (#181401)

    71ec25b

  3. gaaclarke commented on Jan 26, 2026

    @gaaclarke
    gaaclarke
    on Jan 26, 2026
    MemberAuthor

    I added the json file to the error message in another PR. The json file looks fine:

    Skia gold comparison failed: Error when running Skia Gold: /b/s/w/ir/cache/gold/goldctl imgtest init --instance flutter --work-dir /b/s/w/ir/cache/builder/engine/src/out/ci/host_release_test/gen/skia_gold_canvas_test.dart_iplr/temp --commit 7e5f1053e9b8bf1d59f5ce9c77c16e78bef81d4b --keys-file /b/s/w/ir/cache/builder/engine/src/out/ci/host_release_test/gen/skia_gold_canvas_test.dart_iplr/keys.json --failure-file /b/s/w/ir/cache/builder/engine/src/out/ci/host_release_test/gen/skia_gold_canvas_test.dart_iplr/failures.json --passfail
Skia Gold imgtest init failed.
An error occurred when initializing golden file test with 
goldctl.
keys json:
{"CI":"luci","Platform":"linux","impeller_enabled":"true"}


stdout: 
stderr: File /b/s/w/ir/cache/builder/engine/src/out/ci/host_release_test/gen/skia_gold_canvas_test.dart_iplr/keys.json does not contain readable JSON object: unexpected end of JSON input

    https://logs.chromium.org/logs/flutter/buildbucket/cr-buildbucket/8691815120289301921/+/u/test:_Host_Tests_for_host_release/stdout

  4. gaaclarke commented on Jan 26, 2026

    @gaaclarke
    gaaclarke
    on Jan 26, 2026
    MemberAuthor

    I looked at the bots that are failing and didn't notice any commonality. The odd thing is that when a bot flakes, it flakes consistently. It will fail 3 times in a row.

  5. gaaclarke commented on Jan 26, 2026

    @gaaclarke
    gaaclarke
    on Jan 26, 2026
    MemberAuthor

    I've filed a bug against goldctl. I'm not ruling out that we changed something that makes it flakey. I suspect their error reporting could be fixed though which would give us information about what is happening. The error now that the JSON is malformed is demonstrably not true: b/478879171

  6. gaaclarke
    added
    triaged-engineTriaged by Engine team
    on Jan 27, 2026

  7. gaaclarke commented on Jan 27, 2026

    @gaaclarke
    gaaclarke
    on Jan 27, 2026
    MemberAuthor

    The failure can also look like this:

    Skia gold comparison failed: Error when running Skia Gold: /b/s/w/ir/cache/gold/goldctl imgtest add --work-dir /b/s/w/ir/cache/builder/engine/src/out/ci/host_release_test/gen/skia_gold_text_test.dart_iplr/temp --test-name engine.text_test_fontWeightVariable --png-file /b/s/w/ir/cache/builder/engine/src/out/ci/host_release_test/gen/skia_gold_text_test.dart_iplr/text_test_fontWeightVariable.png --passfail --add-test-optional-key image_matching_algorithm:fuzzy --add-test-optional-key fuzzy_max_different_pixels:900 --add-test-optional-key fuzzy_pixel_delta_threshold:0
Skia Gold received an unapproved image in post-submit
testing. Golden file images in flutter/engine are triaged
in pre-submit during code review for the given PR.

Visit https://flutter-gold.skia.org// to view and approve
the image(s), or revert the associated change. For more
information, visit the wiki:
https://github.com/flutter/flutter/wiki/Writing-a-golden-file-test-for-package:flutter


stdout:
stderr: Error running command: ''invalid test config: field "gitHash", "commit_id", or "change_list_id" must be set.. At jsonio.go:153 goldclient.go:326 goldclient.go:280 cmd_imgtest.go:363 cmd_imgtest.go:294 command.go:860 command.go:974 command.go:902 command.go:895 main.go:54 proc.go:267 asm_amd64.s:1650''

    https://logs.chromium.org/logs/flutter/buildbucket/cr-buildbucket/8691501221705557905/+/u/test:_Host_Tests_for_host_release/stdout

  8. flutter-zl
    added a commit that references this issue on Feb 10, 2026

    Adds contents of keys file when a skia gold error occurs. (flutter#18…

    5a65b48
  9. github-merge-queue
    added a commit that references this issue on Feb 13, 2026

    Adds impeller backend to golden workspace name (#182387)

    d802cab
  10. auto-submit
    closed this as completedin #182387on Feb 13, 2026
  11. rickhohler
    added a commit that references this issue on Feb 19, 2026

    Adds impeller backend to golden workspace name (flutter#182387)

    1f7d817

  12. github-actions commented on Feb 28, 2026

    @github-actions
    github-actionsBot
    on Feb 28, 2026 – with GitHub Actions

    This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of flutter doctor -v and a minimal reproduction of the issue.

  13. github-actions
    locked as resolved and limited conversation to collaborators on Feb 28, 2026
  14. mboetger
    added a commit that references this issue on Mar 26, 2026

    Adds impeller backend to golden workspace name (flutter#182387)

    e12ece6
  15. Nakiha
    added a commit that references this issue on Sep 28, 2026

    Adds impeller backend to golden workspace name (#182387)

    2a99a6c
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    P1High-priority issues at the top of the work listc: flakeTests that sometimes, but not always, incorrectly passe: impellerImpeller rendering backend issues and features requeststeam-engineOwned by Engine teamtriaged-engineTriaged by Engine team

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions