Skip to content

[Feature:InstructorUI] Add bulk upload redactions#11539

Merged
bmcutler merged 88 commits into
mainfrom
manual-redactions
Jul 31, 2025
Merged

[Feature:InstructorUI] Add bulk upload redactions#11539
bmcutler merged 88 commits into
mainfrom
manual-redactions

Conversation

@lavalleeale

@lavalleeale lavalleeale commented Mar 12, 2025

Copy link
Copy Markdown
Contributor

Why is this Change Important & Necessary?

Images of test PDFs frequently contain a student's information, so now users have the option to redact certain parts of certain pages of tests.

What is the New Behavior?

An instructor can upload redactions json files to specify areas of the bulk split pdf that should not be visible to graders.

What steps should a reviewer take to reproduce or test the bug or new feature?

To add redactions to a gradeable visit the update page on the rubric tab. The JSON can only be uploaded when " And do you expect each specific problem/item/component to appear on a specific page number of the PDF document? " is turned on

Automated Testing & Documentation

Other information

See #11538

What is the current behavior?

Other information?

Documentation in Submitty/submitty.github.io#682

This pull request introduces a new feature to handle redactions for gradeable submissions, along with updates to related database migrations, job processing, and API endpoints. The key changes include adding a gradeable_redaction table, implementing a redaction processing workflow, and modifying the PDF image generation process to apply redactions.

Database Changes:

  • Added a new gradeable_redaction table to store redaction data, including constraints for valid coordinate ranges (migration/migrator/data/course_tables.sql).
  • Created a migration script to add the gradeable_redaction table and constraints (migration/migrator/migrations/course/20250312145730_add_redactions.py).

Job Processing Updates:

  • Updated the generate_pdf_images job to accept redactions and output redacted images with a checkered pattern (sbin/submitty_daemon_jobs/submitty_jobs/generate_pdf_images.py).
  • Added a new RegenerateBulkImages job to regenerate images for all submissions in a bulk upload, applying redactions (sbin/submitty_daemon_jobs/submitty_jobs/regenerate_bulk_images.py).
  • Integrated the RegenerateBulkImages job into the job processing pipeline (sbin/submitty_daemon_jobs/submitty_jobs/jobs.py).

API and Controller Enhancements:

  • Added endpoints to retrieve and update redactions for a gradeable in AdminGradeableController, including validation and triggering the regeneration job (site/app/controllers/admin/AdminGradeableController.php).
  • Updated the SubmissionController to include the Redaction model (site/app/controllers/student/SubmissionController.php).

Test and Example Updates:

  • Removed assertions for page image generation in bulk PDF split tests, as this functionality is now handled by the redaction workflow (sbin/submitty_daemon_jobs/tests/test_bulk_pdf_split.py).
  • Added an example redactions.json file to demonstrate redaction data format (more_autograding_examples/bulk_upload_pdfs/submissions/redactions.json).

These changes collectively enable the system to manage redactions effectively, ensuring sensitive information in gradeable submissions can be obscured as needed.

@codecov

codecov Bot commented Mar 12, 2025

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 0% with 117 lines in your changes missing coverage. Please review.
✅ Project coverage is 21.60%. Comparing base (166aadd) to head (ed6ebde).
⚠️ Report is 2 commits behind head on main.

Additional details and impacted files

Impacted file tree graph

@@             Coverage Diff              @@
##               main   #11539      +/-   ##
============================================
- Coverage     21.73%   21.60%   -0.13%     
- Complexity     9335     9363      +28     
============================================
  Files           268      268              
  Lines         35847    35914      +67     
  Branches        474      474              
============================================
- Hits           7793     7761      -32     
- Misses        27584    27683      +99     
  Partials        470      470              
Flag Coverage Δ
autograder 21.31% <ø> (ø)
js 2.07% <ø> (ø)
migrator 100.00% <ø> (ø)
php 20.59% <0.00%> (-0.08%) ⬇️
python_submitty_utils 80.08% <ø> (ø)
submitty_daemon_jobs 90.72% <ø> (+1.83%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

img = Image.open(image_filename)
draw = ImageDraw.Draw(img)
draw.rectangle(redaction.coordinates, fill="black")
img.save(image_filename,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

might be better to save the image after all the redactions are completed on it in case something goes wrong with redacting

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also it'd be good if you could show an image example on the description for your PR

@github-project-automation github-project-automation Bot moved this from Seeking Reviewer to Work in Progress in Submitty Development Mar 25, 2025
@lavalleeale lavalleeale self-assigned this Apr 1, 2025
@github-actions github-actions Bot added the Abandoned PR - Needs New Owner No activity on PR for more than 2 weeks -- seeking new owner to complete label Apr 16, 2025

@williamschen23 williamschen23 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

comments from quickyl looking at code

Comment thread migration/migrator/migrations/course/20250312145730_add_redactions.py Outdated
Comment thread sbin/submitty_daemon_jobs/submitty_jobs/bulk_qr_split.py Outdated
Comment thread sbin/submitty_daemon_jobs/submitty_jobs/bulk_qr_split.py Outdated
Comment thread sbin/submitty_daemon_jobs/submitty_jobs/bulk_upload_split.py Outdated
Comment thread site/app/controllers/admin/AdminGradeableController.php Outdated
@lavalleeale
lavalleeale requested a review from bmcutler July 28, 2025 15:09
@automateprojectmangement automateprojectmangement Bot moved this from Work in Progress to In Review in Submitty Development Jul 28, 2025

@jeffrey-cordero jeffrey-cordero left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The bulk upload redactions functionality still looks good to me. The code looks slightly more readable and maintainable since my last review. One potential note I'd add is making an issue for converting this into a Vue component when it gets merged.

@bmcutler bmcutler left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • please update PR description to follow new format.

  • In addition to the high level 'why' - please add explaination of what & why for each of the significant changes in the processing of bulk upload submissions (why lines are deleted & relocated from qr_split, etc)

Hmm, install failed so I can't test right away. Might be user error on my part.

@github-project-automation github-project-automation Bot moved this from In Review to Work in Progress in Submitty Development Jul 29, 2025
@bmcutler bmcutler moved this from Work in Progress to Awaiting Maintainer Review in Submitty Development Jul 30, 2025
@bmcutler

Copy link
Copy Markdown
Member

@lavalleeale

Redacting worked ok -- the files produced looked pretty good.

But when I went to try grading as a limited access grader I encounter a few problems (may not all be from this PR, sme might be the recent refactor/vue PRs?)

  • When I specified a 2 panel layout, I couldn't place something on the right side, it wasn't in the drop down menu
Screenshot 2025-07-29 at 11 30 03 PM Screenshot 2025-07-29 at 11 30 14 PM
  • upload.pdf didn't load when I was a limited access grader -- even though I was in 'unblinded mode' grading for limited access grader.
Screenshot 2025-07-29 at 11 32 20 PM
  • the assigned page per gradeable component always tried to open the upload.pdf. I think we opened the png if ithat It used to open the png if that was the last thing opened. I dont remember if we planned to save this for a future PR? Anyway combined with the previous failure to load the upload.pdf this is a problem for limited access grader convenience.

  • I can't resize the panels anymore, they are stuck.

Comment thread site/app/controllers/student/SubmissionController.php Outdated
@bmcutler
bmcutler merged commit b67a5d7 into main Jul 31, 2025
24 checks passed
@bmcutler
bmcutler deleted the manual-redactions branch July 31, 2025 22:58
@github-project-automation github-project-automation Bot moved this from Awaiting Maintainer Review to Done in Submitty Development Jul 31, 2025
bmcutler pushed a commit that referenced this pull request Aug 8, 2025
### Why is this Change Important & Necessary?
<!-- Include any GitHub issue that is fixed/closed using "Fixes
#<number>" or "Closes #<number>" syntax.
Alternately write "Partially addresses #<number>" or "Related to
#<number>" as appropriate. -->
#11539 added this line. This line does not work for checkpoint
gradeables as `gradeable.js` does not get included. This would cause all
the dates to fail as the event listeners below it are in control of the
dates

### What is the New Behavior?
<!-- Include before & after screenshots/videos if the user interface has
changed. -->
We should only load bulk uploads if we have an electronic gradeable

### What steps should a reviewer take to reproduce or test the bug or
new feature?
Visit the checkpoint gradeable edit page. Notice error in console. Go to
dates, notice that you cannot change any as a refresh will clear it

### Automated Testing & Documentation
<!-- Is this feature sufficiently tested by unit tests and end-to-end
tests?
If this PR does not add/update the necessary automated tests, write a
new GitHub issue and link it below.
Is this feature sufficiently documented on submitty.org?
Link related PRs or new GitHub issue to update documentation. -->
#11982

### Other information
<!-- Is this a breaking change?  
Does this PR include migrations to update existing installations?  
Are there security concerns with this PR? -->
GarvitKhandelwal31 added a commit to GarvitKhandelwal31/Submitty that referenced this pull request Feb 23, 2026
- Replace pdf2image with PyMuPDF for faster PDF processing
- Add pattern caching for instant redaction rendering
- Implement parallel processing for bulk operations (1.7x speedup)
- Add file caching to skip unchanged submissions
- Optimize memory usage (70% reduction)
- Remove external poppler dependency

Resolves performance concerns from PR Submitty#11539
bmcutler pushed a commit that referenced this pull request Jun 3, 2026
**Why is this Change Important & Necessary?**
Fixes #11867. When testing PR #11539 (PDF Redactions), performance
issues were found in PDF image generation with redactions enabled. This
PR improves performance by switching the rendering backend and adds
safety fixes for concurrent bulk regeneration and caching.


**What is the New Behavior?**
1)PyMuPDF Integration: Replaced the previous PDF rendering approach with
PyMuPDF (fitz) for faster page rendering.
2) Caching with Redaction Awareness: Adds a cache keyed on the PDF
contents and redaction data, so changing redactions correctly forces
regeneration.
3) Daemon support updates.

**What steps should a reviewer take to reproduce or test the new
feature?**
1. Test it on a gradeable with pdf redactions enabled. I tested on this
gradeable Bulk Upload Scanned Exam.
2. Upload the json redactions file
You can use this- 
[
  {
    "page": 1,
    "x1": 0.25,
    "y1": 0.25,
    "x2": 0.75,
    "y2": 0.75
  },
  {
    "page": 2,
    "x1": 0.1,
    "y1": 0.1,
    "x2": 0.9,
    "y2": 0.9
  }
]
3. I have tested on a 28 page pdf file. 
4. Watch the daemon logs (sudo journalctl -u
submitty_daemon_jobs_handler.service -f)
5. Performance on my system-
On main it took around 10 seconds
On the PR's branch it took around 5.27 second 
<img width="958" height="170" alt="Screenshot 2026-06-03 143240"
src="https://github.com/user-attachments/assets/6c0c1a95-6759-431c-bf9a-2bb6f3e2528b"
/>
6. Modify redactions and regenerate again, images should update (cache
invalidated).

**Automated Testing & Documentation**
No new  tests were added.
**Other information**
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

9 participants