Skip to content

Fix make_tensor_proto to normalize non-native NumPy byte order#121394

Merged
copybara-service[bot] merged 1 commit into
tensorflow:masterfrom
Ijtihed:fix/byte-order-make-tensor-proto
Jul 7, 2026
Merged

Fix make_tensor_proto to normalize non-native NumPy byte order#121394
copybara-service[bot] merged 1 commit into
tensorflow:masterfrom
Ijtihed:fix/byte-order-make-tensor-proto

Conversation

@Ijtihed

@Ijtihed Ijtihed commented Jun 17, 2026

Copy link
Copy Markdown
Contributor

tf.make_tensor_proto() serializes incorrect tensor values when given a NumPy array with non-native byte order (e.g., big-endian int32 on a little-endian host). nparray.tobytes() emits bytes in the array's dtype byte order but TensorFlow assumes native byte order when deserializing tensor_content.

This adds a byte order normalization step before serialization using nparray.dtype.isnative and astype(newbyteorder('=')). Only non-native arrays are converted so native arrays are untouched.

Fixes #55789

@google-ml-butler google-ml-butler Bot added the size:S CL Change Size: Small label Jun 17, 2026
@google-cla

google-cla Bot commented Jun 17, 2026

Copy link
Copy Markdown

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates make_tensor_proto in tensorflow/python/framework/tensor_util.py to normalize numpy arrays with non-native byte orders to the host's native byte order. It also adds comprehensive unit tests in tensor_util_test.py to verify this behavior for 32-bit integers, 32-bit floats, and 64-bit integers. No review comments were provided, and there is no additional feedback.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

@keerthanakadiri
keerthanakadiri requested a review from a team June 17, 2026 06:52
@google-ml-butler google-ml-butler Bot added the awaiting review Pull request awaiting review label Jun 17, 2026
@keerthanakadiri keerthanakadiri added the python Pull requests that update Python code label Jun 17, 2026
@github-project-automation github-project-automation Bot moved this to Assigned Reviewer in PR Queue Jun 17, 2026
@keerthanakadiri keerthanakadiri added the prtype:bugfix PR to fix a bug label Jun 17, 2026

@dmiltr3 dmiltr3 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @Ijtihed,

Thanks for submitting this fix! The logic to normalize non-native NumPy arrays before serialization looks correct and targets the root cause of the silent data corruption.

Could you add a unit test case to your PR to verify this behavior and prevent future regressions? Specifically, the test should cover various numeric data types (like int32, float32, and int64) initialized with a forced non-native endianness to assert they serialize and deserialize correctly.

For your convenience, here is a test implementation targeting tensorflow/python/framework/tensor_util_test.py that you can directly copy into your PR:

def testNonNativeByteOrder(self):
  for dtype in [np.int32, np.float32, np.int64]:
    arr = np.array([1, 2, 3], dtype=dtype)
    arr_swapped = arr.astype(arr.dtype.newbyteorder("S"))
    self.assertFalse(arr_swapped.dtype.isnative)

    t = tensor_util.make_tensor_proto(arr_swapped)
    a = tensor_util.MakeNdarray(t)
    self.assertTrue(a.dtype.isnative)
    self.assertEqual(arr.dtype.type, a.dtype.type)
    self.assertAllClose(arr, a)

Once you've added the tests, let us know and we'll proceed with reviewing and importing it internally!

@github-project-automation github-project-automation Bot moved this from Assigned Reviewer to Reviewer Requested Changes in PR Queue Jun 18, 2026
@Ijtihed
Ijtihed force-pushed the fix/byte-order-make-tensor-proto branch from 0f8836f to 9c2ccc3 Compare June 18, 2026 19:26
@Ijtihed

Ijtihed commented Jun 18, 2026

Copy link
Copy Markdown
Contributor Author

@dmiltr3 Done. thanks!

@dmiltr3
dmiltr3 self-requested a review June 18, 2026 20:02
@google-ml-butler google-ml-butler Bot added kokoro:force-run Tests on submitted change ready to pull PR ready for merge process labels Jun 18, 2026
@github-project-automation github-project-automation Bot moved this from Reviewer Requested Changes to Approved by Reviewer in PR Queue Jun 18, 2026
@kokoro-team kokoro-team removed the kokoro:force-run Tests on submitted change label Jun 18, 2026
@Ijtihed

Ijtihed commented Jun 19, 2026

Copy link
Copy Markdown
Contributor Author

@dmiltr3 lmk if anything else is needed from my side! Thanks!

@keerthanakadiri keerthanakadiri removed the awaiting review Pull request awaiting review label Jun 23, 2026
@nithyak0204 nithyak0204 added ready to pull PR ready for merge process and removed ready to pull PR ready for merge process labels Jul 7, 2026
@copybara-service
copybara-service Bot merged commit 9404127 into tensorflow:master Jul 7, 2026
16 of 17 checks passed
@github-project-automation github-project-automation Bot moved this from Approved by Reviewer to Merged in PR Queue Jul 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

prtype:bugfix PR to fix a bug python Pull requests that update Python code ready to pull PR ready for merge process size:S CL Change Size: Small

Projects

Status: Merged

Development

Successfully merging this pull request may close these issues.

tf.make_tensor_proto() does not respect byte order of numpy input array

6 participants