Skip to content

Explicit error if devices: auto & >1 GPU available and if devices >1 - #532

Merged
bsharma58 merged 2 commits into
mainfrom
fix/gpu-config-explicit-error
Jun 30, 2026
Merged

bsharma58 merged 2 commits into
mainfrom
fix/gpu-config-explicit-error

Conversation

@bsharma58

Copy link
Copy Markdown
Contributor

Pull Request

Description

Summary of changes:

  • Check available CUDA GPU count before rejecting trainer configs.
  • Reject devices: "auto" only when multiple GPUs are available.
  • Reject explicit multi-GPU configs such as devices: 2 and devices: [0, 1] when multiple GPUs are available.
  • Allow safe configs such as devices: 1, devices: [0], and accelerator: cpu.
  • No extra dependencies required

Change motivated by non explicit error previously being raised and discussion during ML chat

Fixes

Testing:

  • uv run python -m pytest tests/models/test_validation.py

  • test validate gpu config rejects multi gpu configs when mutliple gpus available

  • test validate gpu config allows configs when only one gpu available

  • test validate gpu config allows single device configs

  • Yes

Checklist:

  • My code follows OCF's coding style guidelines
  • I have performed a self-review of my own code
  • I have made corresponding changes to the documentation
  • I have added tests that prove my fix is effective or that my feature works
  • I have checked my code and corrected any misspellings

@bsharma58
bsharma58 requested review from Sukh-P and felix-e-h-p June 24, 2026 16:03
@bsharma58 bsharma58 self-assigned this Jun 24, 2026
Comment thread pvnet/utils.py Resolved

@felix-e-h-p felix-e-h-p left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good, thanks for this! One small change if all OK.

@felix-e-h-p felix-e-h-p left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe one small change in tests also

Comment thread tests/models/test_validation.py Resolved
- Check available CUDA GPU count before rejecting trainer configs.
- Reject `devices: "auto"` only when multiple GPUs are available.
- Reject explicit multi-GPU configs such as `devices: 2` and `devices: [0, 1]` when multiple GPUs are available.
- Allow safe configs such as `devices: 1`, `devices: [0]`, and `accelerator: cpu`.

Testing:

- `uv run python -m pytest tests/models/test_validation.py`
- test validate gpu config rejects multi gpu configs when mutliple gpus available
- test validate gpu config allows configs when only one gpu available
- test validate gpu config allows single device configs
@bsharma58
bsharma58 requested a review from felix-e-h-p June 26, 2026 13:09
@bsharma58
bsharma58 merged commit 54ee949 into main Jun 30, 2026
6 checks passed
@bsharma58
bsharma58 deleted the fix/gpu-config-explicit-error branch June 30, 2026 12:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants