Skip to content

[Bugfix:System] Vagrant workers ssh fix#12798

Merged
bmcutler merged 13 commits into
mainfrom
worker-ssh-fix
Apr 30, 2026
Merged

[Bugfix:System] Vagrant workers ssh fix#12798
bmcutler merged 13 commits into
mainfrom
worker-ssh-fix

Conversation

@hyeenaa

@hyeenaa hyeenaa commented Apr 17, 2026

Copy link
Copy Markdown
Contributor

solves #12796

Why is this Change Important & Necessary?

as linked in the attached issue, worker machines cannot be directly accessed from the main vagrant machine. This is obviously a problem considering this is the primary way vagrant workers are accessed. This PR makes vagrant workers accessible from the host vagrant machine.

The main problem present is that the authorized keys that are initialized when workers are being provisioned are for the root user. After this, the same configuration for authorization is overwritten by install_system.sh. The current refresh_vagrant_workers functionality expects the root creds, but instead gets the vagrant creds, and fails.

What is the New Behavior?

Now, when ssh worker-x (where x is whatever worker you're trying to connect to) is run on the submitty_daemon user, workers are connected and a ssh connection succeeds.

What steps should a reviewer take to reproduce or test the bug or new feature?

as linked in the issue, to test the problem:

pre-requirements: have a running submitty development machine on vagrant

  1. create a vagrant worker (vagrant workers generate, vagrant workers up)
  2. ssh into development vagrant machine (vagrant ssh)
  3. run refresh_vagrant_workers in any directory
  4. See error

to test my solution, swap to the attached branch and repeat the steps above and observe that you can now connect to the aforementioned worker:

  1. repeat steps above
  2. run sudo su submitty_daemon
  3. run ssh worker-x, with x as whatever work you're attempting to access.

Automated Testing & Documentation

Other information

This is not a breaking change. Securitywise, it shouldn't be a problem, but that doesn't mean it's not. Since we're dealing with ssh that can always open up potential vectors, but to my knowledge my changes shouldn't affect any security concerns.

@codecov

codecov Bot commented Apr 17, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 21.67%. Comparing base (bc7cdf9) to head (3efdb80).
⚠️ Report is 6 commits behind head on main.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff            @@
##               main   #12798   +/-   ##
=========================================
  Coverage     21.67%   21.67%           
  Complexity     9807     9807           
=========================================
  Files           268      268           
  Lines         36722    36722           
  Branches        490      490           
=========================================
  Hits           7960     7960           
  Misses        28276    28276           
  Partials        486      486           
Flag Coverage Δ
autograder 21.32% <ø> (ø)
js 2.02% <ø> (ø)
migrator 100.00% <ø> (ø)
php 20.73% <ø> (ø)
python_submitty_utils 80.08% <ø> (ø)
submitty_daemon_jobs 91.13% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@Rkoester47 Rkoester47 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I replicated the bug on main and then followed the steps on this branch. I can confirm that functionally, the changes described on this PR perform as described. I was able to go through all of the testing steps and encountered no errors while on this branch.

@github-project-automation github-project-automation Bot moved this from Seeking Reviewer to Awaiting Maintainer Review in Submitty Development Apr 21, 2026

@Farhanxikram Farhanxikram left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I found no errors while going through the described testing steps , and was able to ssh into workers successfully, awesome job!

Comment thread .setup/bin/refresh_vagrant_workers.py Outdated
w = subprocess.run(['su', DAEMON_USER, '-c', f"scp -i /tmp/worker_keys/{name} -o StrictHostKeyChecking=no ~/.ssh/id_rsa.pub root@{data['ip_addr']}:/tmp/workerkey"])
w = subprocess.run(['su', DAEMON_USER, '-c', f"ssh -i /tmp/worker_keys/{name} -o StrictHostKeyChecking=no root@{data['ip_addr']} \"chown {SUPERVISOR_USER}:{SUPERVISOR_USER} /tmp/workerkey && su {SUPERVISOR_USER} -c \\\"mkdir -p ~/.ssh && mv /tmp/workerkey ~/.ssh/authorized_keys\\\"\""])
if w.returncode == 0:
w = subprocess.call(['su', DAEMON_USER, '-c', f"scp -i /tmp/worker_keys/{name} -o StrictHostKeyChecking=no ~/.ssh/id_rsa.pub root@{data['ip_addr']}:/tmp/workerkey"])

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why did we switch to .call? .call is the legacy function. .run is the preferred method.
Other than this change, I can't tell if anything else was changed on these lines.

@bmcutler
bmcutler merged commit 5868c3a into main Apr 30, 2026
23 of 24 checks passed
@bmcutler
bmcutler deleted the worker-ssh-fix branch April 30, 2026 21:52
@github-project-automation github-project-automation Bot moved this from Awaiting Maintainer Review to Done in Submitty Development Apr 30, 2026
bmcutler pushed a commit that referenced this pull request Apr 30, 2026
### Why is this Change Important & Necessary?
#12798 was merged despite
failing Python CI.

### What is the New Behavior?
Fixes the Python CI.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

4 participants