Description

This PR adds a new annotations field to the TestCase and an annotate method attached to the TestContext. In the reporter, you can get annotations from testCase.annotations() or in onTestCaseAnnotate event. onTestCaseAnnotate is asynchronous and doesn't block test execution.

interface TestCaseAnnotation {
  message: string
  type: string
  location?: {
    line: number
    column: number
  }
  attachment?: {
    contentType?: string
    path?: string
    body?: string | Uint8Array
  }
}

// extend https://vitest.dev/advanced/api/test-case
interface TestCase {
  annotations(): TestCaseAnnotation[]
}

interface Reporter {
  onTestCaseAnnotate(testCase: TestCase, annotation: TestCaseAnnotation): void
}

Reporters display these annotations differently:

• verbose reporter in non-tty mode prints annotations after every test immediately
• default reporter prints them if there was an error in the test
• tap reporter prints them in # comment
• junit prints them inside properties/attachment
• github-actions shows the annotation simillar to errors
• ui reporter prints them in a "Report" tab and inside the test file (inlined)
• vscode extension will also show these annotations inline

Annotations can also have attachments that will be shown in UI. These attachments are stored in .vitest-attachments folder.

To add an annotation, you call ctx.annotate method:

test('name', ({ annotate }) => {
  // path is relative to the root of the project
  await annotate('hello, world!', 'info', { path: './som-png.png' })
  await annotate('simple message')
  await annotate('this branch shouldn\'t be called, need investigation', 'warning')
})

UI example (WIP):

Please don't delete this checklist! Before submitting the PR, please make sure you do the following:

• It's really useful if your PR references an issue where it is discussed ahead of time. If the feature is substantial or introduces breaking changes without a discussion, PR might be closed.
• Ideally, include a test that fails without this PR but passes with it.
• Please, don't make changes to pnpm-lock.yaml unless you introduce a new test example.

Tests

• Run the tests with pnpm test:ci.

Documentation

• If you introduce new functionality, document it. You can run documentation with pnpm run docs command.

Changesets

• Changes in changelog are generated from PR name. Please, make sure that it explains your changes in an understandable manner. Please, prefix changeset messages with feat:, fix:, perf:, docs:, or chore:.