Skip to content

Conversation

@steven-johnson
Copy link
Contributor

No description provided.

The .npy format is NumPy's native format for storing multidimensional arrays (aka tensors/buffers). Being able to load/save in this format makes it (potentially) a lot easier to interchange data with the Python ecosystem, as well as providing a file format that support floating-point data more robustly than any of the others that we current support.

This adds load/save support for a useful subset:
- We support the int/uint/float types common in Halide (except for f16/bf16 for now)
- We don't support reading or writing files that are in `fortran_order`
- We don't support any object/struct/etc files, only numeric primitives
- We only support loading files that are in the host's endianness (typically little-endian)

Note that at present this doesn't support f16 / bf16 formats, but that could likely be added with minimal difficulty.

The tricky bit of this is that the reading code has to parse a (limited) Python dict in text form. Please review that part carefully.

TODO: we could probably add this as an option for `debug_to_file()` without too much pain in a followup PR.
Built on top of #8175, this adds .npy as an option. This is actually pretty great because it's easy to do something like

```
ss = numpy.load("my_file.npy")
print(ss)
```

in Python and get nicely-formatted output, which can sometimes be a lot easier for debugging that inserting lots of print() statements (see #8176)

Did a drive-by change to the correctness test to use this format instead of .mat.
@steven-johnson steven-johnson requested a review from abadams April 8, 2024 22:54
Base automatically changed from srj/npy-debug-to-file to main April 11, 2024 18:04
@steven-johnson
Copy link
Contributor Author

PTAL

@steven-johnson
Copy link
Contributor Author

Finally clean, PTAL

@steven-johnson
Copy link
Contributor Author

Monday Morning Review Ping

@steven-johnson steven-johnson merged commit d55d82b into main Apr 29, 2024
@steven-johnson steven-johnson deleted the srj/debug-to-file-api branch April 29, 2024 16:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants