Skip to content

python310Packages.tensorflow-bin: 2.13.0 -> 2.15.0#258546

Merged
samuela merged 1 commit intoNixOS:masterfrom
GaetanLepage:tensorflow-bin
Nov 30, 2023
Merged

python310Packages.tensorflow-bin: 2.13.0 -> 2.15.0#258546
samuela merged 1 commit intoNixOS:masterfrom
GaetanLepage:tensorflow-bin

Conversation

@GaetanLepage
Copy link
Contributor

@GaetanLepage GaetanLepage commented Oct 2, 2023 •
edited
Loading

Description of changes

Update tensorflow-bin to 2.15.0.
Changelog: https://github.com/tensorflow/tensorflow/releases/tag/v2.15.0

cc @jyp @abbradar

Things done

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 23.11 Release Notes (or backporting 23.05 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

@github-actions github-actions bot added the 6.topic: python Python is a high-level, general-purpose programming language. label Oct 2, 2023
@GaetanLepage
Copy link
Contributor Author

GaetanLepage commented Oct 2, 2023

Currently, the installation fails:

AttributeError: module 'ml_dtypes' has no attribute 'float8_e4m3b11'. Did you mean: 'float8_e4m3fn'?
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "<string>", line 1, in <lambda>
  File "/nix/store/pzf6dnxg8gf04xazzjdwarm7s03cbrgz-python3-3.10.12/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/nix/store/4pcdqpn2asn90qk1r8ckf27bim9g34a4-python3.10-tensorflow-2.14.0/lib/python3.10/site-packages/tensorflow/__init__.py", line 38, in <module>
    from tensorflow.python.tools import module_util as _module_util
  File "/nix/store/4pcdqpn2asn90qk1r8ckf27bim9g34a4-python3.10-tensorflow-2.14.0/lib/python3.10/site-packages/tensorflow/python/__init__.py", line 42, in <module>
    from tensorflow.python.saved_model import saved_model
  File "/nix/store/4pcdqpn2asn90qk1r8ckf27bim9g34a4-python3.10-tensorflow-2.14.0/lib/python3.10/site-packages/tensorflow/python/saved_model/saved_model.py", line 20, in <module>
    from tensorflow.python.saved_model import builder
  File "/nix/store/4pcdqpn2asn90qk1r8ckf27bim9g34a4-python3.10-tensorflow-2.14.0/lib/python3.10/site-packages/tensorflow/python/saved_model/builder.py", line 23, in <module>
    from tensorflow.python.saved_model.builder_impl import _SavedModelBuilder
  File "/nix/store/4pcdqpn2asn90qk1r8ckf27bim9g34a4-python3.10-tensorflow-2.14.0/lib/python3.10/site-packages/tensorflow/python/saved_model/builder_impl.py", line 26, in <module>
    from tensorflow.python.framework import dtypes
  File "/nix/store/4pcdqpn2asn90qk1r8ckf27bim9g34a4-python3.10-tensorflow-2.14.0/lib/python3.10/site-packages/tensorflow/python/framework/dtypes.py", line 39, in <module>
    _np_float8_e5m2 = pywrap_ml_dtypes.float8_e5m2()
TypeError: Unable to convert function return value to a Python type! The signature was
        () -> handle

While I don't exactly see the link between the AttributeError and the rest of the traceback, I guess that this is related to a recent deprecation in ml-dtypes.
Tensorflow has pinned ml-dtypes to 2.0.0.
This has also been reported in jax and tensorflow-onnx.

@GaetanLepage GaetanLepage mentioned this pull request Oct 2, 2023
Merged
12 tasks
@alyssais
Copy link
Member

alyssais commented Oct 2, 2023

@ofborg eval

@samuela
Copy link
Member

samuela commented Oct 2, 2023

shall we mark as draft for the time being?

@GaetanLepage GaetanLepage marked this pull request as draft October 2, 2023 08:37
@ofborg ofborg bot added 10.rebuild-darwin: 11-100 This PR causes between 11 and 100 packages to rebuild on Darwin. 10.rebuild-linux: 11-100 This PR causes between 11 and 100 packages to rebuild on Linux. labels Oct 2, 2023
@mihaimaruseac
Copy link
Member

mihaimaruseac commented Oct 2, 2023

While I don't exactly see the link between the AttributeError and the rest of the traceback, I guess that this is related to a recent deprecation in ml-dtypes.
Tensorflow has pinned ml-dtypes to 2.0.0.
This has also been reported in jax and tensorflow-onnx.

TF adjacent developer here. The issue here is that ml_dtypes released a backwards incompatible version as TF started the builds for the final release (the release itself took too long, for reasons that are offtopic for this). The main branch of the repository already has a fix but the release team decided to not waste time looking for the fix and instead just add the upper bound on ml-dtypes for this release only.

@samuela
Copy link
Member

samuela commented Oct 2, 2023

Thanks for explaining @mihaimaruseac ! Sounds like we can just wait until the next release then

@GaetanLepage
Copy link
Contributor Author

GaetanLepage commented Oct 3, 2023

Thanks @mihaimaruseac !
Indeed, we will wait for the next release.

@GaetanLepage
Copy link
Contributor Author

GaetanLepage commented Oct 14, 2023 •
edited
Loading

tensorflow builds fine for me on x86, with and without CUDA.
If all the versions/flavors build fine, I would plead to merge this ASAP.
Indeed, several packages are broken as they depend on TF. Furthermore, this could let us update many packages in the python ecosystem.

EDIT: This comment was supposed to be posted on #255998

@mihaimaruseac
Copy link
Member

mihaimaruseac commented Oct 14, 2023

Note that due to some confusion, master branch now also have the pin:

https://github.com/tensorflow/tensorflow/blob/c8b5e2b0d9d91ff199a5eeb30ec2f5d33a5d92c6/tensorflow/tools/pip_package/setup.py#L92

Since 2.15 branch has been cut already, this likely means that TensorFlow 2.15 will suffer from the same problem.

@mihaimaruseac mihaimaruseac mentioned this pull request Nov 22, 2023
Merged
13 tasks
@GaetanLepage GaetanLepage changed the title python310Packages.tensorflow-bin: 2.13.0 -> 2.14.0 python310Packages.tensorflow-bin: 2.13.0 -> 2.15.0 Nov 25, 2023
@GaetanLepage GaetanLepage marked this pull request as ready for review November 25, 2023 23:53
@GaetanLepage
Copy link
Contributor Author

GaetanLepage commented Nov 27, 2023

Result of nixpkgs-review pr 258546 run on x86_64-linux 1

34 packages built:
  • libretranslate (python311Packages.libretranslate)
  • libretranslate.dist (python311Packages.libretranslate.dist)
  • python310Packages.argos-translate-files
  • python310Packages.argos-translate-files.dist
  • python310Packages.argostranslate
  • python310Packages.argostranslate.dist
  • python310Packages.ctranslate2
  • python310Packages.ctranslate2.dist
  • python310Packages.faster-whisper
  • python310Packages.faster-whisper.dist
  • python310Packages.libretranslate
  • python310Packages.libretranslate.dist
  • python310Packages.tensorflow-bin
  • python310Packages.tensorflow-bin.dist
  • python310Packages.translatehtml
  • python310Packages.translatehtml.dist
  • python311Packages.argos-translate-files
  • python311Packages.argos-translate-files.dist
  • python311Packages.argostranslate
  • python311Packages.argostranslate.dist
  • python311Packages.ctranslate2
  • python311Packages.ctranslate2.dist
  • python311Packages.faster-whisper
  • python311Packages.faster-whisper.dist
  • python311Packages.tensorflow-bin
  • python311Packages.tensorflow-bin.dist
  • python311Packages.translatehtml
  • python311Packages.translatehtml.dist
  • tts
  • tts.dist
  • whisper-ctranslate2
  • whisper-ctranslate2.dist
  • wyoming-faster-whisper
  • wyoming-faster-whisper.dist

samuela
samuela approved these changes Nov 27, 2023
View reviewed changes
Copy link
Member

@samuela samuela left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

diff LGTM!

@GaetanLepage
Copy link
Contributor Author

GaetanLepage commented Nov 29, 2023

Any change request ? I guess that this is ready to be merged :)

@samuela
Copy link
Member

samuela commented Nov 30, 2023

LGTM, if no one objects I'll merge tomorrow?

@ConnorBaker ConnorBaker added the 6.topic: cuda Parallel computing platform and API label Nov 30, 2023
@mihaimaruseac
Copy link
Member

mihaimaruseac commented Nov 30, 2023

Looks good to me too, ok with merging

@samuela samuela merged commit 71443dc into NixOS:master Nov 30, 2023
@samuela
Copy link
Member

samuela commented Nov 30, 2023

Thanks @GaetanLepage !

@GaetanLepage GaetanLepage deleted the tensorflow-bin branch November 30, 2023 21:15
@zeuner zeuner mentioned this pull request Dec 8, 2023
Open
13 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@samuela samuela samuela approved these changes

@SomeoneSerge SomeoneSerge SomeoneSerge approved these changes

@abbradar abbradar Awaiting requested review from abbradar

@ConnorBaker ConnorBaker Awaiting requested review from ConnorBaker

+1 more reviewer

@mihaimaruseac mihaimaruseac mihaimaruseac approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

6.topic: cuda Parallel computing platform and API 6.topic: python Python is a high-level, general-purpose programming language. 10.rebuild-darwin: 11-100 This PR causes between 11 and 100 packages to rebuild on Darwin. 10.rebuild-linux: 11-100 This PR causes between 11 and 100 packages to rebuild on Linux.

Projects

CUDA Team
Status: ✅ Done

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@GaetanLepage @alyssais @samuela @mihaimaruseac @SomeoneSerge @ConnorBaker