Issue Type

Build/Install

Source

source

Tensorflow Version

2.9.1, 2.8.2, 2.7.3, 2.7.0

Custom Code

No

OS Platform and Distribution

macOS 12.4 (Apple M1 Pro)

Mobile device

No response

Python version

3.9.13

Bazel version

5.2.0, 4.2.2

GCC/Compiler version

Apple Clang 13.1.6

CUDA/cuDNN version

N/A

GPU model and memory

N/A

Current Behaviour?

I'm trying to update our Spack build recipe with the latest versions of TensorFlow but I'm unable to get anything to build. I've tried multiple versions of both TF and Bazel.

With TensorFlow 2.9.1 and Bazel 5.2.0 I get:

ERROR: /private/var/folders/j1/68dlgpr91vlgs26vty2c8xk80000gn/T/ajstewart/spack-stage/spack-stage-py-tensorflow-2.9.1-oljb6pzisjhfdevgnogv3hwizwhgj2rs/spack-src/tensorflow/tools/pip_package/BUILD:72:10: While resolving toolchains for target //tensorflow/tools/pip_package:simple_console: No matching toolchains found for types @bazel_tools//tools/cpp:toolchain_type. Maybe --incompatible_use_cc_configure_from_rules_cc has been flipped and there is no default C++ toolchain added in the WORKSPACE file? See https://github.com/bazelbuild/bazel/issues/10134 for details and migration instructions.

With TensorFlow 2.9.1 and Bazel 4.2.2 I get:

ERROR: /private/var/folders/j1/68dlgpr91vlgs26vty2c8xk80000gn/T/spackqkjcmrsf/08f21de77d854b22a26da21255c81581/external/build_bazel_rules_swift/swift/internal/swift_protoc_gen_aspect.bzl:457:13: 'lambda' not supported, declare a function instead

With TensorFlow 2.8.2/2.7.3/2.7.0 and Bazel 4.2.2 I get:

bazel-out/darwin_arm64-opt/bin/tensorflow/core/protobuf/error_codes.pb.h:10:10: fatal error: 'google/protobuf/port_def.inc' file not found
#include <google/protobuf/port_def.inc>
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.

Multiple users have reported this last build issue: spack/spack#31229.

Standalone code to reproduce the issue

spack/spack#31615 is the PR where I'm working on our build recipe. This issue can be reproduced by first cloning my PR branch:

$ git clone https://github.com/adamjstewart/spack.git
$ cd spack
$ git switch packages/py-tensorflow
$ . share/spack/setup-env.sh

Then, try to install TF using one of the following commands:

$ spack install py-tensorflow
$ spack install [email protected] ^[email protected]
$ spack install [email protected] ^[email protected]
$ spack install [email protected] ^[email protected]
...

Unfortunately, all of these fail at the moment. You don't have to try my PR branch, the develop branch also fails. But the develop branch hasn't been updated since 2.7.0 since none of our developers have managed to get newer versions of TF to build on any system, let alone M1. Would any TF developers be interested in helping maintain this build recipe?

You can see the full build recipe and patches we currently require by running spack edit py-tensorflow or by looking at the above PR.

Relevant log output

TensorFlow 2.9.1 and Bazel 5.2.0:

• build log
• build env

TensorFlow 2.9.1 and Bazel 4.2.2:

• build log
• build env

TensorFlow 2.8.2 and Bazel 4.2.2:

• build log
• build env