Skip to content

[Testing:Forum] Implement Image Diff For Markdown#10260

Closed
04vmatsibekker wants to merge 49 commits into
mainfrom
image-diff
Closed

[Testing:Forum] Implement Image Diff For Markdown#10260
04vmatsibekker wants to merge 49 commits into
mainfrom
image-diff

Conversation

@04vmatsibekker

Copy link
Copy Markdown
Contributor

Please check if the PR fulfills these requirements:

  • Tests for the changes have been added/updated (if possible)
  • Documentation has been updated/added if relevant
  • Screenshots are attached to Github PR if visual/UI changes were made

What is the current behavior?

Can't test markdown.

What is the new behavior?

Implemented Image Diff so we can take a screenshot of the forum page and then test it looks correct

Other information?

@codecov

codecov Bot commented Mar 15, 2024

Copy link
Copy Markdown

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 20.35%. Comparing base (3102c53) to head (4eddee1).
Report is 6 commits behind head on main.

Additional details and impacted files

Impacted file tree graph

@@             Coverage Diff              @@
##               main   #10260      +/-   ##
============================================
- Coverage     22.07%   20.35%   -1.72%     
- Complexity     8673     8675       +2     
============================================
  Files           224      200      -24     
  Lines         30764    28178    -2586     
============================================
- Hits           6792     5737    -1055     
+ Misses        23972    22441    -1531     
Flag Coverage Δ
autograder ?
migrator ?
php 19.09% <ø> (-0.01%) ⬇️
python_submitty_utils ?
submitty_daemon_jobs 90.94% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

@github-actions

github-actions Bot commented Jun 8, 2024

Copy link
Copy Markdown

This PR has been inactive (no commits and no review comments) for 12 days. If there is no new activity in the next 48 hours, this PR will be labeled as Abandoned PR - Needs New Owner and either another developer will finish the PR or it will be closed.

@github-actions github-actions Bot added the Abandoned PR - Needs New Owner No activity on PR for more than 2 weeks -- seeking new owner to complete label Jun 8, 2024
@williamjallen williamjallen removed the Abandoned PR - Needs New Owner No activity on PR for more than 2 weeks -- seeking new owner to complete label Jul 4, 2024
@github-actions github-actions Bot added the Abandoned PR - Needs New Owner No activity on PR for more than 2 weeks -- seeking new owner to complete label Jul 7, 2024
@manzil-infinity180

manzil-infinity180 commented Jul 24, 2024

Copy link
Copy Markdown
Contributor

Removed the visual testing for autograding section (markdown_test.spec.js)
I will test that section in separate pr

@IDzyre IDzyre left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks close, some small changes.

Comment thread .github/workflows/submitty_ci.yml Outdated
Comment thread site/public/js/notebook_builder/widgets/widget.js Outdated
@IDzyre

IDzyre commented Jul 31, 2024

Copy link
Copy Markdown
Member

Once the tests pass in both CI and locally, I'll do a final review, but the progress looks good.

@manzil-infinity180

manzil-infinity180 commented Jul 31, 2024

Copy link
Copy Markdown
Contributor

Once the tests pass in both CI and locally, I'll do a final review, but the progress looks good.

is that threshold value is good like i used 0.02 in this test?

@IDzyre

IDzyre commented Jul 31, 2024

Copy link
Copy Markdown
Member

Ideally it would be 0 thresh hold, but we can discuss it more in the meeting.

@github-actions github-actions Bot added the Abandoned PR - Needs New Owner No activity on PR for more than 2 weeks -- seeking new owner to complete label Aug 27, 2024
@IDzyre

IDzyre commented Apr 28, 2025

Copy link
Copy Markdown
Member

Seeing the comments at the bottom of the JS package, it would be difficult to get this working locally and in CI without using docker, which we don't plan to use/support. If we want to get it to work in just CI we could, but it would probably be hard to maintain.

Please notice
Be aware that despite forcing a screenshot resolution to a particular height and width for a test, if this test is run on different computers (i.e a 13" Mac vs PC attached to a 30" monitor), the results will be different. So it's extremely important that you standardize where the tests will run, both locally and CI.
One way to handle this is by running it with docker container. This project tests use a docker container to run the tests so it could be used as an example.

@bmcutler

Copy link
Copy Markdown
Member

Closing this PR because unfortunately, due to differences in screen resolution (and perhaps also browser and native machine OS) the idea of performing automatic image comparisons will be challenging. It will be difficult for developers to debug problems and maintain these tests long term.

Closing this PR for now.
Perhaps this testing will be better supported and easier to do in the future, and we can re-implement/re-open this PR.

@bmcutler bmcutler closed this Jun 13, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Abandoned PR - Needs New Owner No activity on PR for more than 2 weeks -- seeking new owner to complete markdown Testing / Continuous Integration (CI)

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

5 participants