Skip to content

e2e tests are unstable #3203

@khassel

Description

@khassel

No new problem, our e2e test are failing unrelated to the tested code.

Was already discussed in some PR's so I decided to make an issue for this.

My observations:

  • not related to native fetch, was the same with node-fetch before

  • most fails are with node v20 (but not exclusive)

  • seems to be a timing problem, very hard to reproduce locally

  • looks like different tests interfere, maybe the next one is started before the predecessor stopped

  • some strange errors are mocked in tests/e2e/helpers/mock-console.js:

    const mockError = (err) => {
      if (
        err.includes("ECONNREFUSED") ||
        err.includes("ECONNRESET") ||
        err.includes("socket hang up") ||
        err.includes("exports is not defined") ||
        err.includes("write EPIPE") ||
        err.includes("AggregateError") ||
        err.includes("ERR_SOCKET_CONNECTION_TIMEOUT")
      ) {
        jest.fn();
      } else {
        console.dir(err);
      }
    };
    
    global.console = {
      log: jest.fn(),
      dir: console.dir,
      error: mockError,
      warn: console.warn,
      info: jest.fn(),
      debug: console.debug
    };

    So removing mockError here will show these errors and may can lead to the real error

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions