[Bug]: Behavior of maxFailures with last-run.json and CI Test Reruns with --last-failed #33974

Version

1.49.1

Steps to reproduce

I encountered an issue while using Playwright with the maxFailures configuration in our CI pipeline. Here's the scenario:

1. We have maxFailures set to 3.
2. When running 70 tests, if the 4th test fails 3 times, Playwright stops execution as per the maxFailures threshold.
3. The last-run.json file then only contains information about the failed tests (in this case, the 4th test).
4. When rerunning the job with the --last-failed flag, Playwright only runs the failed test(s) and skips the rest of the tests that were not executed in the initial run.

Is this the expected behavior of Playwright with maxFailures?
Are there alternative configurations or recommendations to ensure all tests are executed in reruns while still using the maxFailures feature?

Expected behavior

I would expect the --last-failed flag to respect the test configuration and include skipped tests from the initial run in the rerun. This would ensure comprehensive test coverage during subsequent executions.

Actual behavior

When rerunning the job with the --last-failed flag, Playwright only runs the failed test(s) and skips the rest of the tests that were not executed in the initial run.
This behavior creates an issue because skipped tests remain untested during reruns.

Additional context

No response

Environment

System:
    OS: macOS 15.2
    CPU: (10) arm64 Apple M1 Pro
    Memory: 81.59 MB / 16.00 GB
  Binaries:
    Node: 20.12.2 - ~/.nvm/versions/node/v20.12.2/bin/node
    npm: 10.5.0 - ~/.nvm/versions/node/v20.12.2/bin/npm
    pnpm: 9.14.2 - /opt/homebrew/bin/pnpm
  Languages:
    Bash: 3.2.57 - /bin/bash
  npmPackages:
    @playwright/test: ^1.49.1 => 1.49.1