Skip to content

Comments

fix(ui): auto reload coverage iframe after test run#5242

Merged
sheremet-va merged 3 commits intovitest-dev:mainfrom
hi-ogawa:feat-ui-reload-coverge
Feb 20, 2024
Merged

fix(ui): auto reload coverage iframe after test run#5242
sheremet-va merged 3 commits intovitest-dev:mainfrom
hi-ogawa:feat-ui-reload-coverge

Conversation

@hi-ogawa
Copy link
Contributor

@hi-ogawa hi-ogawa commented Feb 20, 2024 •
edited
Loading

Description

I tested a simple iframe reloading and tested by pnpm -C examples/basic test:ui --coverage:

Show demo
2024-02-20.13-35-10.webm

But actually I'm worrying that there might be some race condition here:

vitest/packages/vitest/src/node/core.ts

Lines 529 to 530 in a479115

await this.report('onFinished', this.state.getFiles(specs), this.state.getUnhandledErrors())
await this.reportCoverage(allTestsRun)

If reportCoverage takes significant time, then reloading iframe during onFinished might end up fetching old coverage html.

To avoid this race condition, it might be necessary to either:

  • add new reporter hook (like onReportCoverageFinished) just for this, or
  • move up reportCoverage before onFinished (this would likely mess up terminal output for coverage text reporter) or
  • provide a dedicated "iframe reload" button on UI, so users can at least reload it manually.

I'm leaning towards the 1st option, but what do you think?

Please don't delete this checklist! Before submitting the PR, please make sure you do the following:

  • It's really useful if your PR references an issue where it is discussed ahead of time. If the feature is substantial or introduces breaking changes without a discussion, PR might be closed.
  • Ideally, include a test that fails without this PR but passes with it.
  • Please, don't make changes to pnpm-lock.yaml unless you introduce a new test example.

Tests

  • Run the tests with pnpm test:ci.

Documentation

  • If you introduce new functionality, document it. You can run documentation with pnpm run docs command.

Changesets

  • Changes in changelog are generated from PR name. Please, make sure that it explains your changes in an understandable manner. Please, prefix changeset messages with feat:, fix:, perf:, docs:, or chore:.

@netlify
Copy link

netlify bot commented Feb 20, 2024 •
edited
Loading

✅ Deploy Preview for fastidious-cascaron-4ded94 ready!

Name Link
🔨 Latest commit 468c057
🔍 Latest deploy log https://app.netlify.com/sites/fastidious-cascaron-4ded94/deploys/65d46dc142ee350008b3b370
😎 Deploy Preview https://deploy-preview-5242--fastidious-cascaron-4ded94.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

@hi-ogawa hi-ogawa changed the title feat(ui): reload coverage iframe after test re-run feat(ui): auto reload coverage iframe after test run Feb 20, 2024
@AriPerkkio
Copy link
Member

AriPerkkio commented Feb 20, 2024

Using the html reporter from Istanbul for UI has always been quite hacky solution. In long term it would be best to create new custom coverage reporter that extended the html one. This reporter would integrate with browser through web sockets.

@hi-ogawa
Copy link
Contributor Author

hi-ogawa commented Feb 20, 2024 •
edited
Loading

Using the html reporter from Istanbul for UI has always been quite hacky solution. In long term it would be best to create new custom coverage reporter that extended the html one. This reporter would integrate with browser through web sockets.

That's an interesting idea, but it feels a bit overkill? With the custom reporter, we can have access to onEnd to notify UI, but then that would still require managing websocket rpc and probably need adding some reporter hook at the core level.

For now, I went with adding onFinishedReportCoverage reporter hook.

@hi-ogawa hi-ogawa marked this pull request as ready for review February 20, 2024 06:39
@sheremet-va sheremet-va added this to the 1.4.0 milestone Feb 20, 2024
sheremet-va
sheremet-va reviewed Feb 20, 2024
View reviewed changes
packages/vitest/src/types/reporter.ts Outdated
onCollected?: (files?: File[]) => Awaitable<void>
onFinished?: (files?: File[], errors?: unknown[]) => Awaitable<void>
/** @internal */
onFinishedReportCoverage?: () => Awaitable<void>
Copy link
Member

@sheremet-va sheremet-va Feb 20, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there any other way? I don't like exposing this even with @internal flag 🤔

Copy link
Contributor Author

@hi-ogawa hi-ogawa Feb 20, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's necessary to notify websocket client in one way or another.

Actually there's one thing possible, which is to expose only WebSocketEvents.onFinishedReportCoverage without Reporter.onFinishedReportCoverage and it would look like this 468c057

Copy link
Member

@sheremet-va sheremet-va Feb 20, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like it 👍🏻

@sheremet-va
Copy link
Member

sheremet-va commented Feb 20, 2024

I think it's more of a bug fix than a feature.

@sheremet-va sheremet-va changed the title feat(ui): auto reload coverage iframe after test run fix(ui): auto reload coverage iframe after test run Feb 20, 2024
@sheremet-va sheremet-va merged commit 5376d5b into vitest-dev:main Feb 20, 2024
@hi-ogawa hi-ogawa deleted the feat-ui-reload-coverge branch February 20, 2024 23:08
Brooooooklyn referenced this pull request in toeverything/AFFiNE Feb 21, 2024
@Brooooooklyn
chore: bump up all non-major dependencies (#5793