After moving to vitest we should look into its Browser Mode, which makes use of some of the above (playwright and WebdriverIO, with playwright being the preferred tool as it supports parallel execution).

Can be used with a GUI or headless. If we end up using this we can remove the canvas package from our dev dependencies.

To test Piximi application, we wanted to write Piximi test cases in such a way that would resemble how our application is used. We experimented with React Testing Library, Vitest, and Playwright.

React Testing Library

We tried to use React Testing Library to test individual components, rendering each component and interacting with the component to simulate user interaction.An example of this would be render our NewProjectListItem which we can use to start a new project.

render(
       <Provider store={store}>
           <NewProjectListItem  />
       </Provider> 
  );

But doing so made the duration of each test case to be 30 seconds. In the performance breakdown of each test case, which gives information about time taken to transform, setup, collect, and prepare, we saw that collect took around ~25 seconds as the components were heavy. We could try to use mock testing but then we would not be resembling user behaviour for testing.

Vitest Browser Mode

The application already integrates Vitest, and Vitest provides a browser mode option that gives access to DOM elements and their objects. It also works in the same way as React Testing Library: first, we render, and then we simulate user actions. The test cases are written in such a way that each one tests a single flow, for example, opening a particular example project or creating a new kind.

When we run the test cases together, only the first one passes. The subsequent ones fail with a timeout saying the test ID is not found. However, if we run each test case individually, they all pass. This suggests that we are not on the home page when the second test runs. The first test case has an onClick, so the issue could be one of the following: either the render() function is not working properly and is reusing the DOM from the first test case, meaning the expected test ID is missing, or there is a navigation issue and the page is not resetting to the home page as it should.

To find out the issue, whether the issue was with respect to navigation or render , the below example was used.

export const TestApplication = () => {
  return (
    <BrowserRouter basename={"/"}>
      <Routes>
        <Route path="/" element={<ComponentA />} />
        <Route path="compB" element={<ComponentB />} />
      </Routes>
    </BrowserRouter>
  );
};

const ComponentA = () => {
  const navigate = useNavigate();
  const [dummyText, setDummyText] = useState("before click");
  const handleNav = () => {
    navigate("compB");
  };
  return (
    <div>
      <button data-testid="nav" onClick={handleNav}>
        nav
      </button>
      <button
        data-testid="textButton"
        onClick={() => setDummyText("after click")}
      >
        {dummyText}
      </button>
      <a data-testid="comp-a">This is component A</a>
    </div>
  );
};

const ComponentB = () => {
  const navigate = useNavigate();
  const handleNav = () => {
    navigate("/");
  };
  return (
    <div>
      <button data-testid="nav" onClick={handleNav}>
        nav
      </button>
      <a data-testid="comp-b">This is component B</a>
    </div>
  );
};

test("stest Comp A button text", async () => {
  const page = render(<TestApplication />);
  const testText = page.getByTestId("comp-a");
  const textButton = page.getByTestId("textButton");
  await expect.element(textButton).toHaveTextContent("before click");
  await textButton.click();
  await expect.element(textButton).toHaveTextContent("after click");
  await expect.element(testText).toBeVisible();
});

test("stest Comp A button text reset + nav", async () => {
  const page = render(<TestApplication />);
  const testText = page.getByTestId("comp-a");
  const navButton = page.getByTestId("nav");
  const textButton = page.getByTestId("textButton");
  await expect.element(textButton).toHaveTextContent("before click");
  await expect.element(testText).toBeVisible();
  await navButton.click();
  const testTextB = page.getByTestId("comp-b");
  await expect.element(testTextB).toBeVisible();
});

test("stest Comp A", async () => {
  const page = render(<TestApplication />);
  console.log(window.location.href);
  const testText = page.getByTestId("comp-a");
  const navButton = page.getByTestId("nav");
  await expect.element(testText).toBeVisible();
  await navButton.click();
});

The second test case passes. It checks that the textButton has the value “before click,” and since the value is present, the test case succeeds. However, the third test case fails with a timeout. When we print window.location.href, it shows the same URL as in the second test case. This suggests that the application is not navigating back to the home page between tests.

To address this, we would need to add code in different parts of our React application to ensure navigation to the home page. However, adding such code purely for testing purposes, when it does not contribute to the actual functionality of the application, is not considered good practice.

We experimented with multiple configurations, such as isolating each test case and using pool: ‘forks ’ in the Vitest configuration to run each test case in a separate process, but none of these solutions worked. As a result, we were not able to use Vitest browser mode effectively.

Playwright

Playwright is extensively used to do end-to-end testing and is suitable to simulate user actions. It’s kind of similar to React Testing Library and Vitest browser mode, as we need the application or component to be rendered and need access to the DOM. Playwright launches the whole application (so essentially renders it) in the browser and gives access to the DOM. The test cases to simulate user behaviour and test are written in Playwright in the Piximi application.

Test cases in Playwright start by navigating the page to a URL. After that, the test can interact with the page elements. So, we need to launch our application on localhost first and then run our test cases. A sample test case would look like this:

test("Documenation", async () => {
  const browser = await chromium.launch();
  const context = await browser.newContext();
  const page = await context.newPage();
  await page.goto("http://localhost:3000/");
  const documentationButton = page.getByTestId("documentation");
  await expect(documentationButton).toHaveAttribute(
    "href",
    "https://documentation.piximi.app"
  );
  const [newTab] = await Promise.all([
    page.context().waitForEvent("page"),
    documentationButton.click(),
  ]);
  await newTab.waitForLoadState();
  await expect(newTab).toHaveURL("https://documentation.piximi.app/intro.html");
});

The above test case checks if the Documentation link is working properly and if it’s navigating to the correct URL.

To add this to our workflow, a new GitHub workflow is added in playwright.yml. This workflow launches the application on localhost and runs the Playwright test cases.