Repository navigation
feat(numpy_ops): support nanmin, nanmax, nanvar, and nanstd in tf.experimental.numpy - #130338
Open
Yogyabhilash wants to merge 3 commits into
Open
Yogyabhilash wants to merge 3 commits into
Yogyabhilash wants to merge 3 commits into
Conversation
Contributor
There was a problem hiding this comment.
Code Review
This pull request introduces several NumPy-compatible math operations to TensorFlow, specifically implementing nanmin, nanmax, nanvar, and nanstd, alongside adding complex number support to isnan and corresponding unit tests. The review feedback identifies two critical issues: first, isnan returns a raw tf.Tensor instead of a tf.experimental.numpy.ndarray when handling complex numbers; second, nanstd raises a TypeError when an integer dtype is specified because math_ops.sqrt is called on an integer tensor. Both issues can be resolved using the provided code suggestions to wrap the tensor in np_array_ops.asarray and defer integer casting in nanstd respectively.
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem & Motivation
tf.experimental.numpyprovides drop-in NumPy parity for TensorFlow users, including reductions likenansum,nanprod, andnanmean. However, several essential NaN-handling reduction operations from NumPy were missing:nanmin: finding the minimum value ignoring NaNs across specified axes.nanmax: finding the maximum value ignoring NaNs across specified axes.nanvar: computing the variance ignoring NaNs with degrees-of-freedom (ddof) support and complex dtype support.nanstd: computing standard deviation ignoring NaNs.np_math_ops.isnanfailed with anInvalidArgumentErroron complex dtypes because TensorFlow's underlyingIsNankernel does not support complex dtypes directly.When migrating NumPy workloads that operate on datasets with missing/NaN values, users encountered
AttributeError: module 'tensorflow.experimental.numpy' has no attribute 'nanmin'(ornanmax,nanvar,nanstd), andisnanraised errors on complex numbers.Substantive Changes
tensorflow/python/ops/numpy_ops/np_math_ops.py:isnan:isinf: evaluates real and imaginary components separately (math_ops.logical_or(isnan(real(x)), isnan(imag(x)))).nanmin&nanmax:@tf_export.tf_export('experimental.numpy.nanmin')andnanmax.amin/amaxfor exact dtypes (integers, booleans) without overhead.+inf(fornanmin) or-inf(fornanmax) before computing reductions.reduce_all(nan_mask)and restoresNaNfor slices containing only NaNs, conforming to NumPy's exact specification.initialscalar parameter for bounded min/max reductions.outandwherearguments with informativeValueError.nanvar&nanstd:@tf_export.tf_export('experimental.numpy.nanvar')andnanstd.np_array_ops.var.working_dtypecasting whendtypeis provided.nanmeanand calculates squared deviations (usingreal(diff * conj(diff))for complex dtypes to ensure real variance).dof = count - ddof; producesNaNwhendof <= 0(e.g. all-NaN slice or single element withddof=1), matching NumPy.nanstdassqrt(nanvar(...)).tensorflow/python/ops/numpy_ops/np_math_ops_test.py:testNanMinMax,testNanVarStd, andtestIsNanComplex.axis=None,axis=0,axis=-1,keepdims=True,keepdims=False,initial, andddof=0/ddof=1.outandwherearguments.Verification
All newly added functionality and unit tests were verified against NumPy:
testIsNanComplex: Passed forcomplex128arrays with both real and imaginary NaNs.testNanMinMax: Passed across float32, float64, int32, bool, all-NaN slices, keepdims, and initial parameters.testNanVarStd: Passed across float32, float64, int32, complex128, all-NaN slices, ddof=0, and ddof=1.