test(frontend): Improve reliability of visual regression tests by Twixes · Pull Request #14422 · PostHog/posthog

Twixes · 2023-02-27T17:39:04Z

Problem

Noticed on #14286 that the Exporter stories are tricky to take snapshots of reliably due to graph resizing.

Changes

Making some minor improvements to make sure there's no flakiness anywhere.

posthog-bot · 2023-02-27T17:52:24Z

📸 UI snapshots have been updated

1 snapshot changes in total. 0 added, 1 modified, 0 deleted:

chromium: 0 added, 1 modified, 0 deleted (diff for shard 2)
webkit: 0 added, 0 modified, 0 deleted
firefox: 0 added, 0 modified, 0 deleted

Triggered by this commit.

👉 Review this PR's diff of snapshots.

posthog-bot · 2023-02-27T18:02:22Z

📸 UI snapshots have been updated

1 snapshot changes in total. 0 added, 1 modified, 0 deleted:

chromium: 0 added, 1 modified, 0 deleted (diff for shard 2)
webkit: 0 added, 0 modified, 0 deleted
firefox: 0 added, 0 modified, 0 deleted

Triggered by this commit.

👉 Review this PR's diff of snapshots.

posthog-bot · 2023-02-27T18:41:01Z

📸 UI snapshots have been updated

1 snapshot changes in total. 0 added, 1 modified, 0 deleted:

chromium: 0 added, 1 modified, 0 deleted (diff for shard 1)
webkit: 0 added, 0 modified, 0 deleted
firefox: 0 added, 0 modified, 0 deleted

Triggered by this commit.

👉 Review this PR's diff of snapshots.

posthog-bot · 2023-02-28T17:00:38Z

📸 UI snapshots have been updated

2 snapshot changes in total. 0 added, 2 modified, 0 deleted:

chromium: 0 added, 1 modified, 0 deleted (diff for shard 1)
webkit: 0 added, 1 modified, 0 deleted (diff for shard 2)
firefox: 0 added, 0 modified, 0 deleted

Triggered by this commit.

👉 Review this PR's diff of snapshots.

posthog-bot · 2023-02-28T20:41:04Z

📸 UI snapshots have been updated

2 snapshot changes in total. 0 added, 2 modified, 0 deleted:

chromium: 0 added, 1 modified, 0 deleted (diff for shard 2)
webkit: 0 added, 1 modified, 0 deleted (diff for shard 2)
firefox: 0 added, 0 modified, 0 deleted

Triggered by this commit.

👉 Review this PR's diff of snapshots.

posthog-bot · 2023-02-28T21:05:43Z

📸 UI snapshots have been updated

1 snapshot changes in total. 0 added, 1 modified, 0 deleted:

chromium: 0 added, 1 modified, 0 deleted (diff for shard 2)
webkit: 0 added, 0 modified, 0 deleted
firefox: 0 added, 0 modified, 0 deleted

Triggered by this commit.

👉 Review this PR's diff of snapshots.

posthog-bot · 2023-02-28T21:19:15Z

📸 UI snapshots have been updated

2 snapshot changes in total. 0 added, 2 modified, 0 deleted:

chromium: 0 added, 2 modified, 0 deleted (diff for shard 2)
webkit: 0 added, 0 modified, 0 deleted
firefox: 0 added, 0 modified, 0 deleted

Triggered by this commit.

👉 Review this PR's diff of snapshots.

posthog-bot · 2023-03-01T11:33:18Z

📸 UI snapshots have been updated

1 snapshot changes in total. 0 added, 1 modified, 0 deleted:

chromium: 0 added, 1 modified, 0 deleted (diff for shard 1)
webkit: 0 added, 0 modified, 0 deleted
firefox: 0 added, 0 modified, 0 deleted

Triggered by this commit.

👉 Review this PR's diff of snapshots.

posthog-bot · 2023-03-01T11:48:40Z

📸 UI snapshots have been updated

1 snapshot changes in total. 0 added, 1 modified, 0 deleted:

chromium: 0 added, 1 modified, 0 deleted (diff for shard 1)
webkit: 0 added, 0 modified, 0 deleted
firefox: 0 added, 0 modified, 0 deleted

Triggered by this commit.

👉 Review this PR's diff of snapshots.

Twixes · 2023-03-01T13:52:17Z

100% reliability is really difficult because of Chart.js being occasionally quirky in a hard-to-debug way, but this should be an improvement.

posthog-bot · 2023-03-01T18:57:21Z

📸 UI snapshots have been updated

5 snapshot changes in total. 0 added, 5 modified, 0 deleted:

chromium: 0 added, 3 modified, 0 deleted (diff for shard 2)
webkit: 0 added, 2 modified, 0 deleted (diff for shard 2)
firefox: 0 added, 0 modified, 0 deleted

Triggered by this commit.

👉 Review this PR's diff of snapshots.

thmsobrmlr

🚢 -it

posthog-bot · 2023-03-02T12:19:09Z

📸 UI snapshots have been updated

2 snapshot changes in total. 0 added, 2 modified, 0 deleted:

chromium: 0 added, 2 modified, 0 deleted (diff for shard 1)
webkit: 0 added, 0 modified, 0 deleted
firefox: 0 added, 0 modified, 0 deleted

Triggered by this commit.

👉 Review this PR's diff of snapshots.

posthog-bot · 2023-03-02T12:31:29Z

📸 UI snapshots have been updated

1 snapshot changes in total. 0 added, 1 modified, 0 deleted:

chromium: 0 added, 1 modified, 0 deleted (diff for shard 1)
webkit: 0 added, 0 modified, 0 deleted
firefox: 0 added, 0 modified, 0 deleted

Triggered by this commit.

👉 Review this PR's diff of snapshots.

posthog-bot · 2023-03-02T18:18:05Z

📸 UI snapshots have been updated

1 snapshot changes in total. 0 added, 1 modified, 0 deleted:

chromium: 0 added, 1 modified, 0 deleted (diff for shard 2)
webkit: 0 added, 0 modified, 0 deleted
firefox: 0 added, 0 modified, 0 deleted

Triggered by this commit.

👉 Review this PR's diff of snapshots.

posthog-bot · 2023-03-02T19:13:30Z

📸 UI snapshots have been updated

1 snapshot changes in total. 0 added, 1 modified, 0 deleted:

chromium: 0 added, 1 modified, 0 deleted (diff for shard 2)
webkit: 0 added, 0 modified, 0 deleted
firefox: 0 added, 0 modified, 0 deleted

Triggered by this commit.

👉 Review this PR's diff of snapshots.

Twixes and others added 2 commits February 27, 2023 18:35

test(frontend): Ensure 100% reliability of visual regression tests

3ced51f

Update UI snapshots for chromium (2)

f9ecbcb

Update UI snapshots for chromium (2)

c99fa2c

Twixes and others added 2 commits February 27, 2023 19:25

Use OptiPNG in CI

ccd998b

Update UI snapshots for chromium (1)

7db681d

Twixes and others added 5 commits February 28, 2023 17:44

Don't rely on pre-commit hook for image compression

2dc8bcd

Allow waiting for selector instead of constant delay

77ebea4

Merge branch 'master' into vrt-full-reliability

176a3dd

Update UI snapshots for webkit (2)

f3d77d8

Update UI snapshots for chromium (1)

12eaae0

github-actions bot added 2 commits February 28, 2023 17:09

Update UI snapshots for chromium (2)

f452c35

Update UI snapshots for webkit (2)

91fa94f

mariusandra mentioned this pull request Feb 28, 2023

chore: upgrade d3 #14442

Merged

Twixes added 3 commits February 28, 2023 20:39

Add extra insight rendering conditions

0eea48a

Set strict timeouts for visual regression test jobs

027f19e

Optimize OptiPNG step

b94d656

Twixes and others added 2 commits February 28, 2023 21:51

Fix histogram selector

74e5164

Update UI snapshots for chromium (2)

0e3131f

github-actions bot and others added 2 commits February 28, 2023 21:13

Update UI snapshots for chromium (2)

20e0877

Ensure non-blank canvas

df5845d

Twixes mentioned this pull request Feb 28, 2023

chore: added waitforloader for the monaco editor #14464

Closed

Twixes added 2 commits February 28, 2023 23:07

Fix blank canvas detection and auto-detect waitForLoadersToDisappear

c7ed3e5

Use Spinner for loading state in MonacoEditor

82d9020

Update UI snapshots for chromium (1)

fcba0d8

Twixes added 2 commits March 1, 2023 13:09

Merge branch 'master' into vrt-full-reliability

392536e

Fix timeout-minutes

d41e0f8

Twixes changed the title ~~test(frontend): Ensure 100% reliability of visual regression tests~~ test(frontend): Improve reliability of visual regression tests Mar 1, 2023

Twixes requested a review from thmsobrmlr March 1, 2023 13:52

Twixes and others added 3 commits March 1, 2023 19:36

Merge branch 'master' into vrt-full-reliability

68fb36e

Update UI snapshots for chromium (2)

f1ff534

Update UI snapshots for webkit (2)

a5c5c70

thmsobrmlr approved these changes Mar 2, 2023

View reviewed changes

Twixes and others added 2 commits March 2, 2023 13:04

Merge branch 'master' into vrt-full-reliability

2d3552b

Update UI snapshots for chromium (1)

1e9ca7f

Update UI snapshots for chromium (1)

c51e2c2

Twixes and others added 4 commits March 2, 2023 15:57

Merge branch 'master' into vrt-full-reliability

b450bc8

Merge branch 'master' into vrt-full-reliability

b87d418

Add extra wait for canvases

00e7504

Update UI snapshots for chromium (2)

7878403

Twixes and others added 2 commits March 2, 2023 19:18

Merge branch 'master' into vrt-full-reliability

f4a9c23

Update UI snapshots for chromium (2)

b950927

Merge branch 'master' into vrt-full-reliability

37887ca

Twixes enabled auto-merge (squash) March 2, 2023 21:04

Twixes merged commit 631ba8c into master Mar 2, 2023

Twixes deleted the vrt-full-reliability branch March 2, 2023 21:11

Conversation

Twixes commented Feb 27, 2023

Problem

Changes

Uh oh!

posthog-bot commented Feb 27, 2023

📸 UI snapshots have been updated

Uh oh!

posthog-bot commented Feb 27, 2023

📸 UI snapshots have been updated

Uh oh!

posthog-bot commented Feb 27, 2023

📸 UI snapshots have been updated

Uh oh!

posthog-bot commented Feb 28, 2023

📸 UI snapshots have been updated

Uh oh!

posthog-bot commented Feb 28, 2023

📸 UI snapshots have been updated

Uh oh!

posthog-bot commented Feb 28, 2023

📸 UI snapshots have been updated

Uh oh!

posthog-bot commented Feb 28, 2023

📸 UI snapshots have been updated

Uh oh!

posthog-bot commented Mar 1, 2023

📸 UI snapshots have been updated

Uh oh!

posthog-bot commented Mar 1, 2023

📸 UI snapshots have been updated

Uh oh!

Twixes commented Mar 1, 2023

Uh oh!

posthog-bot commented Mar 1, 2023

📸 UI snapshots have been updated

Uh oh!

thmsobrmlr left a comment

Choose a reason for hiding this comment

Uh oh!

posthog-bot commented Mar 2, 2023

📸 UI snapshots have been updated

Uh oh!

posthog-bot commented Mar 2, 2023

📸 UI snapshots have been updated

Uh oh!

posthog-bot commented Mar 2, 2023

📸 UI snapshots have been updated

Uh oh!

posthog-bot commented Mar 2, 2023

📸 UI snapshots have been updated

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants