Skip to content

Comments

feat(vitest): add github actions reporter#5093

Merged
sheremet-va merged 27 commits intovitest-dev:mainfrom
hi-ogawa:feat-github-actions-reporter
Feb 7, 2024
Merged

feat(vitest): add github actions reporter#5093
sheremet-va merged 27 commits intovitest-dev:mainfrom
hi-ogawa:feat-github-actions-reporter

Conversation

@hi-ogawa
Copy link
Contributor

@hi-ogawa hi-ogawa commented Feb 1, 2024 •
edited
Loading

Description

The idea is to reuse the entire printError util used by "default" reporter:

vitest/packages/vitest/src/node/reporters/base.ts

Line 371 in 8cc3eef

await this.ctx.logger.printError(error, { project })

It looks like this mostly works with minor adjustment/fix of printError side (though ideally it might make sense to refactor current printError to cover this kind of use case better).

There are a few minor differences from sapphi-red's vitest-github-actions-reporter:

example output

In this commit c89aa5f, I tested demo test cases on CI and it looks like this:

Reveal screenshot

image

Also comparison to default reporter:

image

Also the same annotations appear in the summary https://github.com/vitest-dev/vitest/actions/runs/7749165463

image

todo / tbd

  • test unit
  • test on github
  • doc
  • does workspace sub-directory work?
    • it should be fine since full file path is in the output and github will figure it out.
  • enable/disable automatically by detecting GITHUB_ACTIONS env var?
    • jest requires github-actions reporter explicitly but disable it automatically if GITHUB_ACTIONS is not defined. I did the same here.
  • jest now also does "log folding". it might be nice to have but it looks more tricky (for example, what if tests are in parallel?) I'll explore it separately later Add new default reporter for github actions jestjs/jest#13626
  • enable it on Vitest's own CI?

references

Please don't delete this checklist! Before submitting the PR, please make sure you do the following:

  • It's really useful if your PR references an issue where it is discussed ahead of time. If the feature is substantial or introduces breaking changes without a discussion, PR might be closed.
  • Ideally, include a test that fails without this PR but passes with it.
  • Please, don't make changes to pnpm-lock.yaml unless you introduce a new test example.

Tests

  • Run the tests with pnpm test:ci.

Documentation

  • If you introduce new functionality, document it. You can run documentation with pnpm run docs command.

Changesets

  • Changes in changelog are generated from PR name. Please, make sure that it explains your changes in an understandable manner. Please, prefix changeset messages with feat:, fix:, perf:, docs:, or chore:.

@netlify
Copy link

netlify bot commented Feb 1, 2024 •
edited
Loading

✅ Deploy Preview for fastidious-cascaron-4ded94 ready!

Name Link
🔨 Latest commit 4f43d5a
🔍 Latest deploy log https://app.netlify.com/sites/fastidious-cascaron-4ded94/deploys/65c2dd0330ec360008dfafba
😎 Deploy Preview https://deploy-preview-5093--fastidious-cascaron-4ded94.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

@hi-ogawa hi-ogawa marked this pull request as ready for review February 1, 2024 10:11
hi-ogawa
hi-ogawa commented Feb 1, 2024
View reviewed changes
@sheremet-va
Copy link
Member

sheremet-va commented Feb 5, 2024 •
edited
Loading

Should we enable it by default if GitHub Actions env is found? What does Bun do in this situation for example? The problem here is that it is impossible to disable then 🤔

@hi-ogawa
Copy link
Contributor Author

hi-ogawa commented Feb 5, 2024

Should we enable it by default if GitHub Actions env is found? What does Bun do in this situation for example? The problem here is that it is impossible to disable then 🤔

So far I've been checking only Jest as a comparison (and make it somewhat compatible). Good point, I'll also check how Bun does this.

@hi-ogawa
Copy link
Contributor Author

hi-ogawa commented Feb 6, 2024

Indeed, Bun automatically enables github actions style output based on GITHUB_ACTIONS=true. It was introduced by this PR:

To opt out, users would probably need to manually set GITHUB_ACTIONS=false or unset it on their CI workflow. I don't think Vitest can introduce this feature in this way, so I think it's better to make it opt-in like Jest (at least for starter).

To me, automatically disabling by checking non-existence of GITHUB_ACTIONS already felt not so intuitive, but I think I get this is a tiny nice-to-have trick since otherwise users would need to write a few lines to achieve this like in:

https://github.com/sapphi-red/vitest-github-actions-reporter?tab=readme-ov-file#usage

export default {
  test: {
    reporters: process.env.GITHUB_ACTIONS
      ? ['default', "github-actions"]
      : 'default'
  }
}

@sheremet-va
Copy link
Member

sheremet-va commented Feb 6, 2024

I don't think Vitest can introduce this feature in this way, so I think it's better to make it opt-in like Jest (at least for starter).

Why do you think we cannot introduce it? Is it a breaking change?

This was referenced Feb 6, 2024
Merged
Closed
renovate bot referenced this pull request in JoshuaKGoldberg/eslint-plugin-expect-type Feb 27, 2024
@renovate
chore(deps): update dependency vitest to v1.3.1 (#185