Skip to content

Validate inputs in GroupNorm and InstanceNorm - #3653

Merged
angeloskath merged 1 commit into
ml-explore:mainfrom
ishtihoss:norm-input-validation
Jun 11, 2026
Merged

angeloskath merged 1 commit into
ml-explore:mainfrom
ishtihoss:norm-input-validation

Conversation

@ishtihoss

Copy link
Copy Markdown
Contributor

Proposed changes

Two cases where the norm layers accept bad inputs and return meaningless results instead of raising:

  1. InstanceNorm on an input with no spatial dimensions returns all zeros. For a (N, C) or (C,) input the reduction axes tuple is empty, so mx.mean returns the input itself and mx.var returns 0, and the normalized output is exactly zero everywhere:
import mlx.core as mx
import mlx.nn as nn

nn.InstanceNorm(dims=8)(mx.random.normal((4, 8)))
# all zeros, no error
  1. GroupNorm does not check that num_groups divides dims. With say num_groups=4, dims=6 the reshape can still succeed depending on the spatial size, and the groups end up mixing channel and spatial positions, so the statistics are garbage. The pytorch_compatible=True path floor-divides the group size and silently drops the intended grouping too. PyTorch raises for this at construction.

Both now raise a ValueError, same idea as #3615 did for SinusoidalPositionalEncoding. Added tests to the existing group norm and instance norm tests.

Checklist

Put an x in the boxes that apply.

  • I have read the CONTRIBUTING document
  • I have run pre-commit run --all-files to format my code / installed pre-commit prior to committing changes
  • I have added tests that prove my fix is effective or that my feature works
  • I have updated the necessary documentation (if needed)

@ishtihoss
ishtihoss force-pushed the norm-input-validation branch from 5fc83d8 to 9521e2f Compare June 10, 2026 22:06

@zcbenz zcbenz left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@angeloskath
angeloskath merged commit a6ec712 into ml-explore:main Jun 11, 2026
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants