[Bugfix:Developer] Cypress test for autograding teams by Rkoester47 · Pull Request #12347 · Submitty/Submitty

Rkoester47 · 2026-01-23T15:10:55Z

Why is this Change Important & Necessary?

Fixes #11920
An issue where team submissions would cause an error, resolved by PR #11919 . Currently there is no cypress test to catch the error in the future.

What is the New Behavior?

This test checks to make sure no such error happens again for team autograding submissions

What steps should a reviewer take to reproduce or test the bug or new feature?

Review the code and make sure the test is passing.

Automated Testing & Documentation

Other information

This is not a breaking change.

codecov · 2026-01-23T15:12:40Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 21.66%. Comparing base (9cb25d6) to head (5cb9ff0).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff            @@
##               main   #12347   +/-   ##
=========================================
  Coverage     21.66%   21.66%           
  Complexity     9637     9637           
=========================================
  Files           268      268           
  Lines         36222    36222           
  Branches        487      487           
=========================================
  Hits           7847     7847           
  Misses        27892    27892           
  Partials        483      483

Flag	Coverage Δ
autograder	`21.32% <ø> (ø)`
js	`2.04% <ø> (ø)`
migrator	`100.00% <ø> (ø)`
php	`20.68% <ø> (ø)`
python_submitty_utils	`80.08% <ø> (ø)`
submitty_daemon_jobs	`91.13% <ø> (ø)`

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Team autograding submission cypress tests now checks for the warning/error message to be displayed. Now working on regular solo autograding submissions.

Finished new tests for team and non team submissions, and uncommented older tests

williamschen23

You should not need to copy frame.cpp and frame_hardcoded.cpp in order to test for autograding. The entire more_autograding_examples is copied into the /fixtures directory. Look at how https://github.com/Submitty/Submitty/blob/2d6d58826819e8165cbc197ce0222e2eced44db4/site/cypress/e2e/Cypress-Autograding-Tutorial/electronic_gradeable_tutorial.spec.js does it.

Tests now use submission files already present.

Rkoester47 · 2026-02-06T22:30:02Z

You should not need to copy frame.cpp and frame_hardcoded.cpp in order to test for autograding. The entire more_autograding_examples is copied into the /fixtures directory. Look at how https://github.com/Submitty/Submitty/blob/2d6d58826819e8165cbc197ce0222e2eced44db4/site/cypress/e2e/Cypress-Autograding-Tutorial/electronic_gradeable_tutorial.spec.js does it.

Thank you for pointing this out. I deleted the files I should not have copied over, and rewrote the tests to use copy_of_more_autograding_examples when checking for the "Something went wrong" banner.

williamschen23

This only checks if we can submit files and that nothing immediately went wrong. This is not sufficient to check if there is an actual error in autograding, as we don't get the result from the autograder and check that. The code that should be written should more closely follow https://github.com/Submitty/Submitty/blob/b823c839857cc491f287fc029aa0df809570e402/site/cypress/e2e/Cypress-Autograding-Development/electronic_gradeable_development.spec.js or https://github.com/Submitty/Submitty/blob/b823c839857cc491f287fc029aa0df809570e402/site/cypress/e2e/Cypress-Autograding-Tutorial/electronic_gradeable_tutorial.spec.js, using the functions to submit the files and check the results.

Team Autograding test now checks that all members see the changes made by re-uploads and regrades.

williamschen23 · 2026-03-15T00:01:39Z

Hmmmm it looks like check_refresh is broken for team gradeables. If you get check_refresh working, you should be able to use runTests like the other autograding ones. Otherwise you will have to have a lot of workarounds for doing cypress testing on teams.

check_refresh refers to the 5 second polling that the submission page goes through and tries to find when a submission finishes. When check_refresh finishes, the page refreshes, which is why the other tests work.

Lowered the timeout time and added reload.

…ubmitty/Submitty into cypress-test-autograding-teams

### Why is this Change Important & Necessary? Fixes Submitty#11920 An issue where team submissions would cause an error, resolved by PR Submitty#11919 . Currently there is no cypress test to catch the error in the future. ### What is the New Behavior? This test checks to make sure no such error happens again for team autograding submissions ### What steps should a reviewer take to reproduce or test the bug or new feature? Review the code and make sure the test is passing. ### Automated Testing & Documentation ### Other information This is not a breaking change. --------- Co-authored-by: Rkoester47 <[email protected]>

Rkoester47 added 4 commits January 20, 2026 14:43

Fixed Typo and started new cypress test

41ed688

Added some submission steps

b2badf1

Cleaned up autograding test

278a047

copied frame_hardcoded.cpp to cypress

f395ff5

github-project-automation Bot added this to Submitty Development Jan 23, 2026

github-project-automation Bot moved this to Seeking Reviewer in Submitty Development Jan 23, 2026

automateprojectmangement Bot moved this from Seeking Reviewer to Work in Progress in Submitty Development Jan 23, 2026

Rkoester47 added 4 commits January 25, 2026 22:33

Finished Team submission, started non team

a8380e8

Team autograding submission cypress tests now checks for the warning/error message to be displayed. Now working on regular solo autograding submissions.

Uncommented previous tests

818e201

Finished new tests for team and non team submissions, and uncommented older tests

Formatting fixes

5c78b76

Merge branch 'main' into cypress-test-autograding-teams

e7687e1

Rkoester47 marked this pull request as ready for review January 26, 2026 16:13

automateprojectmangement Bot moved this from Work in Progress to Seeking Reviewer in Submitty Development Jan 26, 2026

williamschen23 requested changes Jan 31, 2026

View reviewed changes

github-project-automation Bot moved this from Seeking Reviewer to Work in Progress in Submitty Development Jan 31, 2026

Deleted unnecessarily copied files

b823c83

Tests now use submission files already present.

Rkoester47 changed the title ~~[BugFix:Developer] Cypress test for autograding teams~~ [Bugfix:Developer] Cypress test for autograding teams Feb 6, 2026

Rkoester47 requested a review from williamschen23 February 10, 2026 22:44

automateprojectmangement Bot moved this from Work in Progress to In Review in Submitty Development Feb 10, 2026

williamschen23 requested changes Feb 12, 2026

View reviewed changes

github-project-automation Bot moved this from In Review to Work in Progress in Submitty Development Feb 12, 2026

github-actions Bot added the Abandoned PR - Needs New Owner No activity on PR for more than 2 weeks -- seeking new owner to complete label Feb 27, 2026

Rkoester47 and others added 5 commits March 12, 2026 20:17

Added test steps

c272fa8

Team Autograding test now checks that all members see the changes made by re-uploads and regrades.

Lint fixes

7e9ae1d

More Linting fixes

48d2b34

Javascript Lint

0925710

Fixed newline issue

b8bfb5c

Rkoester47 removed the Abandoned PR - Needs New Owner No activity on PR for more than 2 weeks -- seeking new owner to complete label Mar 14, 2026

Increase timeout time

f772473

Rkoester47 and others added 7 commits March 16, 2026 20:21

Increasing wait time

c3d4edd

Added debugging for failing CI run

fc87dc0

Merge branch 'main' into cypress-test-autograding-teams

8a185a8

Removed debugging and added reload

555bcf5

Lowered the timeout time and added reload.

Merge branch 'cypress-test-autograding-teams' of https://github.com/S…

0ed320d

…ubmitty/Submitty into cypress-test-autograding-teams

Updated tests

2ae186a

Fixed scoring checks on tests

5cb9ff0

Rkoester47 moved this from Work in Progress to Ready to Merge in Submitty Development Mar 21, 2026

bmcutler merged commit 9735664 into main Mar 22, 2026
25 checks passed

bmcutler deleted the cypress-test-autograding-teams branch March 22, 2026 02:01

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[Bugfix:Developer] Cypress test for autograding teams#12347

[Bugfix:Developer] Cypress test for autograding teams#12347
bmcutler merged 22 commits into
mainfrom
cypress-test-autograding-teams

Rkoester47 commented Jan 23, 2026

Uh oh!

codecov Bot commented Jan 23, 2026 •

edited

Loading

Uh oh!

williamschen23 left a comment •

edited

Loading

Uh oh!

Rkoester47 commented Feb 6, 2026

Uh oh!

williamschen23 left a comment •

edited

Loading

Uh oh!

williamschen23 commented Mar 15, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

Rkoester47 commented Jan 23, 2026

Why is this Change Important & Necessary?

What is the New Behavior?

What steps should a reviewer take to reproduce or test the bug or new feature?

Automated Testing & Documentation

Other information

Uh oh!

codecov Bot commented Jan 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

williamschen23 left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Rkoester47 commented Feb 6, 2026

Uh oh!

williamschen23 left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

williamschen23 commented Mar 15, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

codecov Bot commented Jan 23, 2026 •

edited

Loading

williamschen23 left a comment •

edited

Loading

williamschen23 left a comment •

edited

Loading