Add latest version of bazel by Sinan81 · Pull Request #13813 · spack/spack

Sinan81 · 2019-11-20T21:07:03Z

Split off of #13112

Installed bazel 1.2.0 with Clang 11.0.0, Python 3.7.4, and JDK 1.8.0_231-b11 on macOS 10.15.1. Unit tests pass.

Add a lot of new versions
Add patches for all versions
Make sure patches apply to all versions
Make sure some versions actually build
Make sure some packages can be built with this bazel installation
Remove complexity in patches for passing Spack env vars
Remove complexity in setup_dependent_package
Add unit tests

Fixes #12398
Fixes #11850
Closes #11120
Closes #10422

@jcftang @aweits @obreitwi @rubendibattista @dodo47 @healther

alalazo · 2019-11-22T17:33:17Z

@adamjstewart I don't really know how bazel works, but would it be possible to set:

module.bazel = Executable('bazel')

in bazel and have the packages using it specifying the arguments like:

bazel('build', make_jobs)

? If there are no issues that would be the most straightforward solution.

…nding cc_env patch

adamjstewart · 2019-11-24T02:29:51Z

var/spack/repos/builtin/packages/bazel/package.py

-    version('0.4.4', sha256='d52a21dda271ae645711ce99c70cf44c5d3a809138e656bbff00998827548ebb')
-
-    depends_on('java@8:', type=('build', 'link', 'run'))
-    depends_on('zip')


It's unclear if zip/unzip are needed for linking or just to unzip the download tarball...

adamjstewart · 2019-11-24T06:10:17Z

Okay, I added a ton of new versions and made sure are patches correctly apply to every version. This does not mean we are patching all the files that we need to though. Last step is to actually test a few bazel installations.

adamjstewart · 2019-11-24T06:11:22Z

var/spack/repos/builtin/packages/bazel/package.py

+
+    # https://docs.bazel.build/versions/master/install-compile-source.html#bootstrap-bazel
+    depends_on('java@8', type=('build', 'link', 'run'))
+    depends_on('python', type=('build', 'link', 'run'))


Docs for 0.17-1.2 say both Python 2 and 3 work, no docs prior to 0.17... (can we not support ancient versions of bazel and TF?)

That is totally fine for me!

adamjstewart · 2019-11-24T06:12:09Z

var/spack/repos/builtin/packages/bazel/package.py

-    patch('fix_env_handling-0.9.0.patch', when='@0.9.0:0.12.0')
-    patch('fix_env_handling-0.13.0.patch', when='@0.13.0:0.13.999')
-    patch('fix_env_handling-0.17.2.patch', when='@0.14.0:')
-    patch('link.patch')


I dropped this patch as the fix_env_handling patches apply correctly to every version, and I couldn't find a version where link.patch worked. This may need to be resurrected.

adamjstewart · 2019-11-24T20:06:36Z

var/spack/repos/builtin/packages/jdk/package.py

    version('1.8.0_131-b11', sha256='62b215bdfb48bace523723cdbb2157c665e6a25429c73828a32f00e587301236', curl_options=curl_options,
            url='http://download.oracle.com/otn-pub/java/jdk/8u131-b11/d54c1d3a095b4ff2b6607d096fa80163/jdk-8u131-linux-x64.tar.gz')

-    provides('java')


adamjstewart · 2019-11-24T20:07:45Z

Okay, I think this is finally ready to merge! I built a dozen or so versions of the package. The unit tests passed for me. We may need additional tweaks once we start working on TF, but this package is a lot better than it used to be.

adamjstewart · 2019-11-24T21:26:52Z

var/spack/repos/builtin/packages/bazel/bazelconfiguration-0.3.patch

+      if (envName.startsWith("SPACK_")) {
+        builder.put(envName, env.get(envName));
+      }
+    }


Previously, we enumerated through every Spack environment variable and passed it to the build if it was set. This was extremely error prone, as the exact env vars Spack sets in build_environment.py and uses in the compiler wrappers changes frequently. About half of the env vars we were setting no longer exist, and the other half weren't being passed.

With this newer patch, bazel will pass all SPACK_* env vars to the build environment. I haven't written Java in a loooong time, but I think I did things right.

Great improvement!

adamjstewart · 2019-11-24T21:27:34Z

var/spack/repos/builtin/packages/bazel/cc_configure-0.3.0.patch

@@ -12,12 +12,8 @@
 +  ]
 +
 +  env = repository_ctx.os.environ
-+  if "SPACK_DEPENDENCIES" in env:
-+    for dep in env["SPACK_DEPENDENCIES"].split(":"):


SPACK_DEPENDENCIES no longer exists, long live SPACK_INCLUDE_DIRS!

adamjstewart · 2019-11-24T22:33:22Z

Local tests pass, but ftpmirror.gnu.org is down so the build tests are currently failing. We'll either have to wait for ftpmirror.gnu.org to come back online, or switch the download URLs.

healther

Looks good to me, far better than our version :)

* use bazel commit in spack#13112, and add version 0.24.1, and corresponding cc_env patch * undo preferred java version by dodo47 * patch for v0.26 * Update install steps * Add patches for more versions * Add unit tests * Update patches for new Spack env vars * env is already defined, use spackEnv

ax3l added new-version java patch update-package labels Nov 20, 2019

adamjstewart changed the title ~~use bazel commit in #13112, and add version 0.24.1, and corresponding…~~ Add latest version of bazel Nov 20, 2019

adamjstewart removed the java label Nov 20, 2019

This comment has been minimized.

Sign in to view

Sinan81 added 3 commits November 23, 2019 20:03

use bazel commit in spack#13112, and add version 0.24.1, and correspo…

eea9e1c

…nding cc_env patch

undo preferred java version by dodo47

e68cbd0

patch for v0.26

2343a4f

adamjstewart force-pushed the package/bazel branch from b73a04a to 2343a4f Compare November 24, 2019 02:05

Update install steps

2d61002

adamjstewart added the WIP label Nov 24, 2019

adamjstewart changed the title ~~Add latest version of bazel~~ [WIP] Add latest version of bazel Nov 24, 2019

adamjstewart reviewed Nov 24, 2019

View reviewed changes

adamjstewart self-assigned this Nov 24, 2019

Add patches for more versions

fa4e79f

adamjstewart reviewed Nov 24, 2019

View reviewed changes

adamjstewart mentioned this pull request Nov 24, 2019

Version concretization failing for virtual dependencies #13870

Closed

Add unit tests

33c6a67

adamjstewart changed the title ~~[WIP] Add latest version of bazel~~ Add latest version of bazel Nov 24, 2019

adamjstewart removed the WIP label Nov 24, 2019

adamjstewart reviewed Nov 24, 2019

View reviewed changes

Update patches for new Spack env vars

9a492c7

adamjstewart reviewed Nov 24, 2019

View reviewed changes

env is already defined, use spackEnv

07f1781

healther approved these changes Nov 25, 2019

View reviewed changes

adamjstewart merged commit 448e09e into spack:develop Nov 25, 2019

Conversation

Sinan81 commented Nov 20, 2019 • edited by adamjstewart Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

alalazo commented Nov 22, 2019

Uh oh!

Choose a reason for hiding this comment

Uh oh!

adamjstewart commented Nov 24, 2019

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

adamjstewart commented Nov 24, 2019

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

adamjstewart commented Nov 24, 2019

Uh oh!

healther left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Sinan81 commented Nov 20, 2019 •

edited by adamjstewart

Loading