Skip to content

[Bugfix:Autograding] Fix regrade team detection#12737

Merged
bmcutler merged 2 commits into
Submitty:mainfrom
srikaaviya:fix/regrade-team-detection
May 1, 2026
Merged

[Bugfix:Autograding] Fix regrade team detection#12737
bmcutler merged 2 commits into
Submitty:mainfrom
srikaaviya:fix/regrade-team-detection

Conversation

@srikaaviya

@srikaaviya srikaaviya commented Apr 2, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Why is this Change Important & Necessary?

Fixes #12423

The regrade.py script used an '_' heuristic to distinguish individual user submissions from team submissions:

This incorrectly classifies usernames containing underscores (e.g. jane_doe) as team submissions, causing them to be incorrectly queued for regrading.

What is the New Behavior?

Instead of checking for '_', the fix reads user_assignment_settings.json inside the submission directory and checks for the presence of a "team_history" key.

This key is written exclusively by TeamController.php when a team directory is first created.
Individual user directories never contain this key.
If the file is missing, the code safely defaults to treating the submitter as an individual user.

What steps should a reviewer take to reproduce or test the bug or new feature?

  1. In the development VM, find a gradeable with a user whose username contains an underscore (e.g. student_user)
  2. Before fix: is_team will be true and user will be ""
  3. After fix: is_team will be false and user will be "student_user"

Automated Testing & Documentation

No automated tests currently exist for regrade.py. A follow-up issue should be created to add unit tests for this script.

Other information

  • Not a breaking change
  • No migrations required
  • No security concerns

@srikaaviya
[Bugfix:Autograding] Fix regrade team detection
05604de 
Fixes Submitty#12423

* Replaced the fragile '_' heuristic used to identify team submissions.
* The script now checks for the 'team_history' key in user_assignment_settings.json.
* Ensures users with underscores (e.g. jane_doe) are not wrongly queued as teams.
* Safely defaults to treating missing files as individual submissions.
@github-project-automation github-project-automation Bot moved this to Seeking Reviewer in Submitty Development Apr 2, 2026
@codecov

codecov Bot commented Apr 2, 2026 •
edited
Loading

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 21.68%. Comparing base (c52d47a) to head (b0d8a3f).
⚠️ Report is 41 commits behind head on main.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff            @@
##               main   #12737   +/-   ##
=========================================
  Coverage     21.68%   21.68%           
  Complexity     9778     9778           
=========================================
  Files           268      268           
  Lines         36504    36504           
  Branches        487      487           
=========================================
  Hits           7915     7915           
  Misses        28106    28106           
  Partials        483      483
Flag Coverage Δ
autograder 21.32% <ø> (ø)
js 2.04% <ø> (ø)
migrator 100.00% <ø> (ø)
php 20.71% <ø> (ø)
python_submitty_utils 80.08% <ø> (ø)
submitty_daemon_jobs 91.13% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@QuackHonk QuackHonk self-assigned this Apr 3, 2026
QuackHonk
QuackHonk requested changes Apr 3, 2026
View reviewed changes

@QuackHonk QuackHonk left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code looks good, but I'm having trouble testing it. Is there a dummy account already existing with a hyphen in it? Did you create your own account? If so, how?

@github-project-automation github-project-automation Bot moved this from Seeking Reviewer to Work in Progress in Submitty Development Apr 3, 2026
John-Roy123
John-Roy123 requested changes Apr 3, 2026
View reviewed changes

@John-Roy123 John-Roy123 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When testing this change as an instructor user by creating a new student in the "Manage Students" page, the page crashes when I try to create a new student.

Image

@QuackHonk QuackHonk removed their assignment Apr 3, 2026
@srikaaviya

srikaaviya commented Apr 3, 2026

Copy link
Copy Markdown
Contributor Author

The code looks good, but I'm having trouble testing it. Is there a dummy account already existing with a hyphen in it? Did you create your own account? If so, how?

You can test this entirely through the UI in your Vagrant:

  1. Log in to the VM as instructor.
  2. Go to Manage Students and click "Add New Student". Create an account with an underscore (not hyphen) in the User ID, for example: student_user.
  3. Go to an individual gradeable (e.g. Open Homework).
  4. Because you are an instructor, use the "Submit on behalf of a student" feature to submit a dummy text file as student_user.
  5. Now in your terminal, run regrade:
    sudo /usr/local/submitty/bin/regrade.py/var/local/submitty/courses/s26/sample/submissions/open_homework/student_user

Look at the output of the queue JSON. Before this PR, it wrongly flagged it as a team. After this PR, it correctly recognizes it as an individual.

@srikaaviya

srikaaviya commented Apr 3, 2026

Copy link
Copy Markdown
Contributor Author

When testing this change as an instructor user by creating a new student in the "Manage Students" page, the page crashes when I try to create a new student.

Image

I guess the crash you're experiencing on the "Manage Students" page is unrelated to this PR. My changes were contained within bin/regrade.py python script.

I was actually able to successfully create a student named student_user on my own Vagrant VM without any crashing.

Looking at your screenshot, I noticed you tried to use the ID adams_g. Is it possible that adams_g already exists in your database from a previous test?

@github-actions github-actions Bot added the Abandoned PR - Needs New Owner No activity on PR for more than 2 weeks -- seeking new owner to complete label Apr 29, 2026
Eli-J-Schwartz
Eli-J-Schwartz approved these changes May 1, 2026
View reviewed changes

@Eli-J-Schwartz Eli-J-Schwartz left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just tested this, on both an individual and a team submission. On the individual regrade, it correctly detected that the submission was by an individual, even when I submitted with a username that included an underscore _. On a regrade of a team submission, it again correctly detected that the submission was by a team.

@bmcutler
bmcutler merged commit 4cee7f5 into Submitty:main May 1, 2026
24 of 25 checks passed
@github-project-automation github-project-automation Bot moved this from Work in Progress to Done in Submitty Development May 1, 2026
prestoncarman added a commit that referenced this pull request May 21, 2026
@prestoncarman
Merge branch 'main' into feature/submitty-cli
180cdc8 
* main: (50 commits)
  [Bugfix:Developer] Fix Broken CI (#12851)
  [Bugfix:System] Restore TCLAPP (#12850)
  [Bugfix:Submission] Fix non-VCS repo error (#12608)
  [Feature:Submission] Jupyter warnings outside notebook (#12523)
  [Bugfix:InstructorUI] Removed feature flag for Bulk Late Days (#12826)
  [Bugfix:Forum] Fix stats header rendering (#12698)
  [Bugfix:Autograding] Fix negative number tolerance (#12569)
  [Bugfix:HelpQueue] Always use Abbreviated Names in OH (#12820)
  [Bugfix:Autograding] Fix regrade team detection (#12737)
  [Dependency] Bump Lichen from v23.09.00 to v26.04.01 (#12823)
  [Dependency] Bump RainbowGrades from v25.10.00 to v26.04.01 (#12824)
  [Feature:Developer] Remove misc Java utils and TCLAPP (#12819)
  [Bugfix:TAGrading] Fix 2-panel open-document select (#12577)
  [Dependency] Bump AnalysisToolsTS from v23.10.00 to v26.04.00 (#12822)
  [Bugfix:Developer] Fix bump_repo permissions (#12821)
  [Bugfix:InstructorUI] Change Colors to match in Docker UI table (#12739)
  [Bugfix:Forum] Reduce Page Caching (#12339)
  [Bugfix:System] File lost when running install clean fix (#12158)
  [Bugfix:Submission] No File Rubric submit all user groups (#12812)
  [Dependency] Bump twig/markdown-extra from 3.23.0 to 3.24.0 in /site (#12721)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@John-Roy123 John-Roy123 John-Roy123 requested changes
@QuackHonk QuackHonk QuackHonk requested changes
@Eli-J-Schwartz Eli-J-Schwartz Eli-J-Schwartz approved these changes

Assignees

No one assigned

Labels

Abandoned PR - Needs New Owner No activity on PR for more than 2 weeks -- seeking new owner to complete

Projects

Submitty Development
Archived in project

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

regrade.py incorrectly identifies usernames with underscores as teams

6 participants

@srikaaviya @John-Roy123 @Eli-J-Schwartz @QuackHonk @bmcutler @dagemcn