Skip to content

[Feature:Notifications] Released grade notification#10358

Merged
bmcutler merged 103 commits into
mainfrom
grade-notifications
Jun 20, 2025
Merged

[Feature:Notifications] Released grade notification#10358
bmcutler merged 103 commits into
mainfrom
grade-notifications

Conversation

@jeffrey-cordero

@jeffrey-cordero jeffrey-cordero commented Apr 15, 2024

Copy link
Copy Markdown
Contributor

Please check if the PR fulfills these requirements:

  • Tests for the changes have been added/updated (if possible)
  • Documentation has been updated/added if relevant
  • Screenshots are attached to Github PR if visual/UI changes were made

What is the current behavior?

Currently, students are not notified when a score for a visible gradable is released. It is up to the instructor to release an announcement for the given assignment.

What is the new behavior?

Now, all users who wish can receive notifications and/or emails regarding all released scores for any visible electronic gradeable scores for the course they are registered in. Notification settings can be updated as usual in the notification settings page to toggle these notifications as users please.

image

Instructors can now also view the total notifications sent within the dates section. If they wish to resend them for a future release date, a confirmation is displayed on the screen when changing to a future date for gradeables with released notifications.

image

Other information?

This is implemented as a cron job, a scheduled task, to run every hour. To test this feature, you must run the course migration as the notification state of the gradable is added to the gradable table. You can initially run submitty_install to run the proper migration. From here, you can run the following commands after ensuring at least one gradable has pending notifications.

$ sudo su submitty_daemon
$ python3 /usr/local/submitty/sbin/send_notification.py
$ cat /var/local/submitty/logs/notifications/20250319.txt
   [2025-03-21 17:12:14.007012] Successfully updated notification status for 0 submissions

   [2025-03-21 17:14:40.049533] (sample): Sent 81 site, 79 email notifications
   [2025-03-21 17:14:40.101745] Successfully updated notification status for 83 submissions

   [2025-03-21 17:24:45.058778] (sample): Sent 46 site, 45 email notifications
   [2025-03-21 17:24:45.095482] Successfully updated notification status for 46 submissions

To re-release notifications as an instructor for an existing gradable with released notifications, a confirmation will pop up on the edit dates page if the release date is being changed to a future date, where OK should be clicked to reset the notification status of recipients.

[Submitty sample] Grade Released: Grading Homework PDF
Your grade is now available for Grades Released Homework in course 
SAMPLE.

Click here for more info: http://localhost:1511/courses/s25/sample/gradeable/grading_homework_pdf

--
NOTE: This is an automated email notification, which is unable to receive replies.
Please refer to the course syllabus for contact information for your teaching staff.
Update your email notification settings for this course here: http://localhost:1511/courses/s25/sample/notifications/settings

image

@jeffrey-cordero jeffrey-cordero linked an issue Apr 15, 2024 that may be closed by this pull request
@jeffrey-cordero jeffrey-cordero changed the title Grade notifications [Feature: Notifications] Notification for grade scores Apr 15, 2024
@jeffrey-cordero jeffrey-cordero changed the title [Feature: Notifications] Notification for grade scores [Feature:Notifications] Released scores notification Apr 15, 2024
@jeffrey-cordero jeffrey-cordero changed the title [Feature:Notifications] Released scores notification [Feature:Notifications] Released grade notification Apr 15, 2024
@jeffrey-cordero
jeffrey-cordero marked this pull request as ready for review April 15, 2024 18:16
@jeffrey-cordero
jeffrey-cordero marked this pull request as draft April 16, 2024 18:46
@jeffrey-cordero
jeffrey-cordero marked this pull request as ready for review April 16, 2024 18:55

{% if electronic and gradeable.isStudentView() and release_date <= "now"|date(date_format, timezone_string) %}
<p style="max-width:600px; margin-top:5px;" id="gradeable-notifications-message">
Note: Gradeable notifications are released at the start of every hour for completed electronic gradeables with student views. <span id="notification-total">{{ gradeable.getNotificationsSent() }}</span> have been sent so far.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we also add a note about how many students are un-notified/un-sent and will be notified/sent at the top of the hour?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can update this to be in the form Note: Grades released notifications are sent and emailed at the start of every hour for students with complete TA/Manual grades. <x> notifications have been sent and <y> are awaiting to be sent.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

" are awaiting to be sent at the top of the hour."
if is non-zero. If is zero, leave out this phrase.

@bmcutler

Copy link
Copy Markdown
Member

@jeffrey-cordero
I've updated the text for the UI. Please proofread and make sure it sounds good and is consistent.
And then please update the images in the PR description.

@jeffrey-cordero
jeffrey-cordero requested a review from bmcutler June 11, 2025 19:28
@automateprojectmangement automateprojectmangement Bot moved this from Work in Progress to In Review in Submitty Development Jun 11, 2025
@bmcutler
bmcutler merged commit 45a5d6d into main Jun 20, 2025
24 checks passed
@bmcutler
bmcutler deleted the grade-notifications branch June 20, 2025 15:20
bmcutler pushed a commit that referenced this pull request Jun 25, 2025
### Why is this Change Important & Necessary?
<!-- Include any GitHub issue that is fixed/closed using "Fixes
#<number>" or "Closes #<number>" syntax.
Alternately write "Partially addresses #<number>" or "Related to
#<number>" as appropriate. -->

Our email format, such as the subject and footer, can be found within
the web server and the course notification cron job script, which can be
troublesome to maintain as we continue to alter their contents. A more
maintainable approach would be to normalize the given footer or subject
contents before sending them, and not store redundant content in the
master database, as they can be implied from the original subject or the
potential term and course. Fixes #11734.

### What is the New Behavior?
<!-- Include before & after screenshots/videos if the user interface has
changed. -->

The subject and footer contents are now calculated within the email cron
job script and added to the finalized email before being delivered over
the network. You can verify the original email content within the master
database (`SELECT subject, body FROM emails ORDER by created DESC LIMIT
1;` and the finalized email content within the individual logs
(`/var/local/submitty/logs/emails/mailboxes/<email>/<id>[email protected]`).

```bash
root@ubuntu:/usr/local/submitty# cat /var/local/submitty/logs/emails/mailboxes/[email protected]/[email protected]
Content-Type: text/plain; charset=utf-8
TO: [email protected]
From: submitty@vagrant
reply-to: do-not-reply@vagrant
Subject: [Submitty sample]: New Announcement: Announcement


An Instructor or Teaching Assistant made an announcement in:
SAMPLE

Announcement

Announcement Content...

Author: Quinn I.
Click here for more info: http://localhost:1511/courses/s25/sample/forum/threads/10

--
NOTE: This is an automated email notification, which is unable to receive replies.
Please refer to the course syllabus for contact information for your teaching staff.
Update your email notification settings for this course here: http://localhost:1511/courses/s25/sample/notifications/settings

root@ubuntu:/usr/local/submitty# cat /var/local/submitty/logs/emails/mailboxes/[email protected]/[email protected]
Content-Type: text/plain; charset=utf-8
TO: [email protected]
From: submitty@vagrant
reply-to: do-not-reply@vagrant
Subject: [Submitty sample] Grade Released: Grades Released Homework


Your grade is now available for Grades Released Homework in course 
SAMPLE.

Click here for more info: http://localhost:1511/courses/s25/sample/gradeable/grades_released_homework

--
NOTE: This is an automated email notification, which is unable to receive replies.
Please refer to the course syllabus for contact information for your teaching staff.
Update your email notification settings for this course here: http://localhost:1511/courses/s25/sample/notifications/settings
```

Additionally, this pull request resolves the issue of invalid email
error logs displayed on the email status page, which currently shows a
site error. The current implementation attempts to access the email
error via array access, but it's an email entity; thus, the Doctrine
getter method should be used.

Before


![image](https://github.com/user-attachments/assets/b1007ebb-ed2a-4935-ad10-77fc27a1438f)

After


![image](https://github.com/user-attachments/assets/31ca5693-6a69-41f4-9e27-70e0050379aa)


### What steps should a reviewer take to reproduce or test the bug or
new feature?

1. Release an instructor/admin announcement or grade notifications (see
#10358).
2. Wait for the emails to be automatically released (100 per min. ~ not
applicable to grade notifications unless the cron job ran recently), or
run `python3 /usr/local/submitty/sbin/send_email.py` using the
`submitty_daemon` user as many times as needed.
3. Verify the original and finalized email content using the sources
noted above.
4. For testing the email errors, run `UPDATE emails SET error = 'x';`
and notice the difference in the email status page. Revert the change
via `UPDATE emails SET error = '';`

### Automated Testing & Documentation
<!-- Is this feature sufficiently tested by unit tests and end-to-end
tests?
If this PR does not add/update the necessary automated tests, write a
new GitHub issue and link it below.
Is this feature sufficiently documented on submitty.org?
Link related PRs or new GitHub issue to update documentation. -->

Email unit tests have been updated to ignore the footer content.

### Other information
<!-- Is this a breaking change?  
Does this PR include migrations to update existing installations?  
Are there security concerns with this PR? -->

N/A
bmcutler pushed a commit that referenced this pull request Jul 10, 2025
### Why is this Change Important & Necessary?
<!-- Include any GitHub issue that is fixed/closed using "Fixes
#<number>" or "Closes #<number>" syntax.
Alternately write "Partially addresses #<number>" or "Related to
#<number>" as appropriate. -->

The initial sample data insertion for gradeables doesn't account for the
existing notification state. Therefore, all gradeables released in the
past will have their release flag initialized as false by default,
causing the first cron job script run to release them in a large batch
for new system installations. Related to #10358.


![notification](https://github.com/user-attachments/assets/6b72ea9d-169c-4837-9844-6bed30432f0c)


### What is the New Behavior?
<!-- Include before & after screenshots/videos if the user interface has
changed. -->

The installation scripts related to inserting electronic gradeable data
for the testing courses will have the notification state set to true if
the gradeable release date is in the past, similar to how the initial
migration command handled it.

### What steps should a reviewer take to reproduce or test the bug or
new feature?

1. Visit a course database through the CLI to reset notification states
or skip the following steps through verification via `vagrant up`
 - `$ su postgres`
 - `$ psql`
- `$ \c submitty_f25_sample` (update `f25` to suit your existing system)
- `$ UPDATE electronic_gradeable_version SET g_notification_sent =
FALSE;`
3. Run the `recreate_sample_courses` command as the root user
4. Using the initial steps in step 1, visit the testing database and run
the following command to ensure the script worked as intended.
- `SELECT * FROM electronic_gradeable_version WHERE g_notification_sent
IS TRUE;` (should be non-empty)

### Automated Testing & Documentation
<!-- Is this feature sufficiently tested by unit tests and end-to-end
tests?
If this PR does not add/update the necessary automated tests, write a
new GitHub issue and link it below.
Is this feature sufficiently documented on submitty.org?
Link related PRs or new GitHub issue to update documentation. -->

N/A

### Other information
<!-- Is this a breaking change?  
Does this PR include migrations to update existing installations?  
Are there security concerns with this PR? -->

N/A
williamschen23 pushed a commit that referenced this pull request Jul 18, 2025
<!-- Include any GitHub issue that is fixed/closed using "Fixes
Alternately write "Partially addresses #<number>" or "Related to

The initial sample data insertion for gradeables doesn't account for the
existing notification state. Therefore, all gradeables released in the
past will have their release flag initialized as false by default,
causing the first cron job script run to release them in a large batch
for new system installations. Related to #10358.

![notification](https://github.com/user-attachments/assets/6b72ea9d-169c-4837-9844-6bed30432f0c)

<!-- Include before & after screenshots/videos if the user interface has
changed. -->

The installation scripts related to inserting electronic gradeable data
for the testing courses will have the notification state set to true if
the gradeable release date is in the past, similar to how the initial
migration command handled it.

new feature?

1. Visit a course database through the CLI to reset notification states
or skip the following steps through verification via `vagrant up`
 - `$ su postgres`
 - `$ psql`
- `$ \c submitty_f25_sample` (update `f25` to suit your existing system)
- `$ UPDATE electronic_gradeable_version SET g_notification_sent =
FALSE;`
3. Run the `recreate_sample_courses` command as the root user
4. Using the initial steps in step 1, visit the testing database and run
the following command to ensure the script worked as intended.
- `SELECT * FROM electronic_gradeable_version WHERE g_notification_sent
IS TRUE;` (should be non-empty)

<!-- Is this feature sufficiently tested by unit tests and end-to-end
tests?
If this PR does not add/update the necessary automated tests, write a
new GitHub issue and link it below.
Is this feature sufficiently documented on submitty.org?
Link related PRs or new GitHub issue to update documentation. -->

N/A

<!-- Is this a breaking change?
Does this PR include migrations to update existing installations?
Are there security concerns with this PR? -->

N/A
bmcutler added a commit that referenced this pull request Aug 9, 2025
### Why is this Change Important & Necessary?
<!-- Include any GitHub issue that is fixed/closed using "Fixes
#<number>" or "Closes #<number>" syntax.
Alternately write "Partially addresses #<number>" or "Related to
#<number>" as appropriate. -->

Notifications for grade releases were originally introduced within
#10358. Still, students have asked for more types of notifications tied
to gradeables, such as receiving one when submissions are open for a
given gradeable.

### What is the New Behavior?
<!-- Include before & after screenshots/videos if the user interface has
changed. -->

This extends the existing hourly cron job to check for electronic
gradeables with pending submission-related notifications ready for
release to all active course users. By default, site notifications are
enabled, while email notifications are disabled to help minimize email
fatigue. For assignments with prerequisites, such as a crash course, the
student will only be notified if the minimum autograding score for the
gradeable dependency is met.

<img width="1587" height="281" alt="image"
src="https://github.com/user-attachments/assets/ce779258-a6f4-4200-8458-d5af01b95bb2"
/>


This is the site notification, which compacts the due date and late day
information into a single line.

<img width="1664" height="312" alt="image"
src="https://github.com/user-attachments/assets/694357ec-0225-47a7-868e-7bb348c15f85"
/>


This is the email notification, including all relevant information on a
separate line for readability.

```
[Subject]
Submissions Open: release_testing
[Content]
Submissions are now being accepted for "release_testing" in course SAMPLE.

Deadline: 2025-08-01 @ 09:13 AM
Late Days: 2 remaining, 5 allowed

Click here for more info: http://localhost:1511/courses/f25/sample/gradeable/release_testing
```

Additionally, the Email Status page may have multiple entries for a
gradeable as the content includes the unique students' remaining late
days, along with the applicable late days, and deadline for the
assignment.

<img width="1575" height="167" alt="image"
src="https://github.com/user-attachments/assets/8a1b0c91-e345-4531-a3af-47a69a5924e0"
/>


### What steps should a reviewer take to reproduce or test the bug or
new feature?

1. Run the required migration via `submitty_install`
2. Create a new electronic gradeable for an active course
4. Release the electronic gradeable for students to submit
5. Run `sudo -u "submitty_daemon" python3 sbin/send_notification.py`
6. Visit the notification tab (email setting is off by default; thus,
this should be manually set for testing)

### Automated Testing & Documentation
<!-- Is this feature sufficiently tested by unit tests and end-to-end
tests?
If this PR does not add/update the necessary automated tests, write a
new GitHub issue and link it below.
Is this feature sufficiently documented on submitty.org?
Link related PRs or new GitHub issue to update documentation. -->

I will create an issue for us to introduce a new Cypress file,
`notifications.spec.js`, within the `Cypress-Feature` workflow to handle
all types of notifications, including verification of their release
through the cron job script.

### Other information
<!-- Is this a breaking change?  
Does this PR include migrations to update existing installations?  
Are there security concerns with this PR? -->

Existing variable and function names related to score notifications have
been refactored for improved readability and to better support
integration of additional notification types.

---------

Co-authored-by: Barb Cutler <[email protected]>
Co-authored-by: Barb Cutler <Barb Cutler>
bmcutler pushed a commit that referenced this pull request Apr 10, 2026
### Why is this Change Important & Necessary?

Students previously had to manually click a notification or visit the
notification page to mark gradeable release and grade release
notifications when visiting the respective page via a link, which is a
poor-quality-of-life experience.

### What is the New Behavior?

When a student visits a gradeable page, any unseen grading notifications
for that gradeable (submission open or grade release) are automatically
marked as seen in a single database update.

### What steps should a reviewer take to reproduce or test the bug or
new feature?

1. Run the respective course migration.
2. Trigger a gradeable release or grade release notification via
`send_notification.py` (`udo -u "submitty_daemon" python3
sbin/send_notification.py`: #11897 and #10358 have more in-depth
explanations).
3. Log in as the notified student (notification bell should show
unread).
4. Navigate to the gradeable page.
5. Verify the notification is now marked as seen (bell clears,
notification panel shows read).

### Automated Testing & Documentation
<!-- Is this feature sufficiently tested by unit tests and end-to-end
tests?
If this PR does not add/update the necessary automated tests, write a
new GitHub issue and link it below.
Is this feature sufficiently documented on submitty.org?
Link related PRs or new GitHub issue to update documentation. -->

Unit tests have been added, but an issue will be made to extend this
notification feature to end-to-end testing coverage.

### Other information

N/A

---------

Co-authored-by: Justin Manion <[email protected]>
GarvitKhandelwal31 pushed a commit to GarvitKhandelwal31/Submitty that referenced this pull request Apr 13, 2026
…itty#12658)

### Why is this Change Important & Necessary?

Students previously had to manually click a notification or visit the
notification page to mark gradeable release and grade release
notifications when visiting the respective page via a link, which is a
poor-quality-of-life experience.

### What is the New Behavior?

When a student visits a gradeable page, any unseen grading notifications
for that gradeable (submission open or grade release) are automatically
marked as seen in a single database update.

### What steps should a reviewer take to reproduce or test the bug or
new feature?

1. Run the respective course migration.
2. Trigger a gradeable release or grade release notification via
`send_notification.py` (`udo -u "submitty_daemon" python3
sbin/send_notification.py`: Submitty#11897 and Submitty#10358 have more in-depth
explanations).
3. Log in as the notified student (notification bell should show
unread).
4. Navigate to the gradeable page.
5. Verify the notification is now marked as seen (bell clears,
notification panel shows read).

### Automated Testing & Documentation
<!-- Is this feature sufficiently tested by unit tests and end-to-end
tests?
If this PR does not add/update the necessary automated tests, write a
new GitHub issue and link it below.
Is this feature sufficiently documented on submitty.org?
Link related PRs or new GitHub issue to update documentation. -->

Unit tests have been added, but an issue will be made to extend this
notification feature to end-to-end testing coverage.

### Other information

N/A

---------

Co-authored-by: Justin Manion <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

Notifications for grade release

7 participants