Skip to content

Conversation

@apalmas-saifh
Copy link
Contributor

Fix evaluation batch size validation to use num_generations_eval when specified

Problem:

The validation check in GRPOConfig.__post_init__ (lines 830-836) (

if (self.per_device_eval_batch_size * num_processes) % self.num_generations != 0:
)
always validates the evaluation batch size divisibility against num_generations, even when num_generations_eval is explicitly set.

Solution:

Modified the check to use num_generations_eval when it's set, falling back to num_generations only when num_generations_eval is None.

Impact:

  • Fixes incorrect validation errors when users set different generation counts for training vs evaluation
  • Allows proper use of the num_generations_eval parameter

Changes:

  • Added logic to determine the correct generation count for evaluation validation
  • Updated error message to be more accurate

Before submitting

  • This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
  • Did you read the contributor guideline,
    Pull Request section?
  • Was this discussed/approved via a GitHub issue? Please add a link
    to it if that's the case.
  • Did you make sure to update the documentation with your changes?
  • Did you write any new necessary tests?

Copy link
Member

@qgallouedec qgallouedec left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks!

@HuggingFaceDocBuilderDev

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@qgallouedec qgallouedec merged commit 2f62f3d into huggingface:main Dec 12, 2025
9 of 11 checks passed
qgallouedec pushed a commit that referenced this pull request Dec 12, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants