Skip to content

Is --run-skipped intended to work on flutter/flutter? #158972

Open

Description

@matanlurey

Both dart test and flutter test support a --run-skipped, which is intended to mean "run tests otherwise that would be skipped". Ideally it would be used to temporarily (either locally or on a bringup: true task) run tests that are flaking or failing for reasons that we don't want to affect the mainline tree or development experience.

For example:

cd flutter/packages/flutter_tools
dart test test/general.shard --run-skipped

However, sometimes we use skip to mean the test can't run in this configuration, such as:

test('...', () {
  // ...
}, skip: !Platform.isWindows ? 'Test only runs on windows' : false);

In that case, --run-skipped not only can never work, but now the feature cannot be used across the codebase.

For Dart (dart test-instrumented) tests, we could use @TestOn instead:

@TestOn('windows')
library;

// ...

void main() {
  test('...', () {
  
  });
}

However, my understanding is this feature does not work in flutter test, so it would not work everywhere/we would diverge.

It would be awesome to have some instruction, outside of commenting code in/out, how we could run skipped tests but also not run tests that are never intended to run in a given configuration or platform. Ideally (to me) that would mean using --run-skipped and perhaps using/supporting @TestOn broadly, but that is obviously more work.

Activity

  1. added
    a: tests"flutter test", flutter_test, or one of our tests
    c: contributor-productivityTeam-specific productivity, code health, technical debt.
    c: proposalA detailed proposal for a change to Flutter
    c: tech-debtTechnical debt, code quality, testing, etc.
    on Nov 15, 2024
  2. natebosch commented on Nov 15, 2024

    @natebosch
    Contributor

    It should work to do testOn: 'windows' instead of a skip: argument.

  3. added
    in triagePresently being triaged by the triage team
    toolAffects the "flutter" command-line tool. See also t: labels.
    team-toolOwned by Flutter Tool team
    and removed
    in triagePresently being triaged by the triage team
    toolAffects the "flutter" command-line tool. See also t: labels.
    on Nov 15, 2024
  4. added
    P3Issues that are less important to the Flutter project
    triaged-toolTriaged by Flutter Tool team
    on Dec 3, 2024
  5. added
    tool-still-validIssues that have been deemed still valid as part of the Flutter Tools issue cleanup initiative.
    on Jul 11, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    P3Issues that are less important to the Flutter projecta: tests"flutter test", flutter_test, or one of our testsc: contributor-productivityTeam-specific productivity, code health, technical debt.c: proposalA detailed proposal for a change to Flutterc: tech-debtTechnical debt, code quality, testing, etc.team-toolOwned by Flutter Tool teamtool-still-validIssues that have been deemed still valid as part of the Flutter Tools issue cleanup initiative.triaged-toolTriaged by Flutter Tool team

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions