Fix make_tensor_proto to normalize non-native NumPy byte order#121394
Conversation
|
Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA). View this failed invocation of the CLA check for more information. For the most up to date status, view the checks section at the bottom of the pull request. |
There was a problem hiding this comment.
Code Review
This pull request updates make_tensor_proto in tensorflow/python/framework/tensor_util.py to normalize numpy arrays with non-native byte orders to the host's native byte order. It also adds comprehensive unit tests in tensor_util_test.py to verify this behavior for 32-bit integers, 32-bit floats, and 64-bit integers. No review comments were provided, and there is no additional feedback.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
dmiltr3
left a comment
There was a problem hiding this comment.
Hi @Ijtihed,
Thanks for submitting this fix! The logic to normalize non-native NumPy arrays before serialization looks correct and targets the root cause of the silent data corruption.
Could you add a unit test case to your PR to verify this behavior and prevent future regressions? Specifically, the test should cover various numeric data types (like int32, float32, and int64) initialized with a forced non-native endianness to assert they serialize and deserialize correctly.
For your convenience, here is a test implementation targeting tensorflow/python/framework/tensor_util_test.py that you can directly copy into your PR:
def testNonNativeByteOrder(self):
for dtype in [np.int32, np.float32, np.int64]:
arr = np.array([1, 2, 3], dtype=dtype)
arr_swapped = arr.astype(arr.dtype.newbyteorder("S"))
self.assertFalse(arr_swapped.dtype.isnative)
t = tensor_util.make_tensor_proto(arr_swapped)
a = tensor_util.MakeNdarray(t)
self.assertTrue(a.dtype.isnative)
self.assertEqual(arr.dtype.type, a.dtype.type)
self.assertAllClose(arr, a)
Once you've added the tests, let us know and we'll proceed with reviewing and importing it internally!
0f8836f to
9c2ccc3
Compare
|
@dmiltr3 Done. thanks! |
|
@dmiltr3 lmk if anything else is needed from my side! Thanks! |
9404127
into
tensorflow:master
tf.make_tensor_proto()serializes incorrect tensor values when given a NumPy array with non-native byte order (e.g., big-endian int32 on a little-endian host).nparray.tobytes()emits bytes in the array's dtype byte order but TensorFlow assumes native byte order when deserializingtensor_content.This adds a byte order normalization step before serialization using
nparray.dtype.isnativeandastype(newbyteorder('=')). Only non-native arrays are converted so native arrays are untouched.Fixes #55789