Skip to content

ARROW-12590: [C++][R] Update copies of Homebrew files to reflect recent updates [WIP] - #12157

Closed
jonkeane wants to merge 9 commits into
apache:masterfrom
jonkeane:macos-autobrew
Closed

jonkeane wants to merge 9 commits into
apache:masterfrom
jonkeane:macos-autobrew

Conversation

@jonkeane

@jonkeane jonkeane commented Jan 14, 2022 •

Copy link
Copy Markdown
Member

This is not totally resolved (I'm getting illegal op codes on the macos 11 job). And might not be the right approach regardless. The crux of this is that our (source) nightly builds bundle the apache-arrow.rb formula, and that formula works with autobrew/homebrew-core which isn't compatible with macos 11. (Of course, we need to keep apache-arrow.rb around and use autobrew/homebrew-core for macos 10.11/10.13 which cran uses!) The approach here is to also bundle (and test!) the formula that is in autobrew/homebrew-cran on a modern macos to ensure that that works.

These should only really matter for nightly (source) builds (or someone who otherwise tries to use these formulae) since releases will use the autobrew/scripts script without using brew at all.

I have confirmed that without the formula from autobrew/homebrew-cran, running on macos 11 runners fails with:

Details
==> Installing thrift dependency: boost
==> Downloading https://dl.bintray.com/boostorg/release/1.67.0/source/boost_1_67_0.tar.bz2
curl: (22) The requested URL returned error: 502 Bad Gateway
Error: Error: An exception occured within a child process:
  DownloadError: Failed to download resource "boost"
Download failed: https://dl.bintray.com/boostorg/release/1.67.0/source/boost_1_67_0.tar.bz2

==> Installing dependencies for apache-arrow: boost, thrift, zstd
==> Installing apache-arrow dependency: boost
/usr/bin/sandbox-exec -f /private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/hbtmp/homebrew20220113-77625-5t1tdb.sb nice /private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/build-apache-arrow/Library/Homebrew/vendor/portable-ruby/2.3.7/bin/ruby -W0 -I /private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/build-apache-arrow/Library/Homebrew/vendor/bundle-standalone/bundler/../ruby/2.3.0/gems/ruby-macho-2.0.0/lib:/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/build-apache-arrow/Library/Homebrew/vendor/bundle-standalone/bundler/../ruby/2.3.0/gems/plist-3.4.0/lib:/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/build-apache-arrow/Library/Homebrew/vendor/bundle-standalone/bundler/:/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/build-apache-arrow/Library/Homebrew/vendor/bundle-standalone/bundler/../ruby/2.3.0/gems/backports-3.11.4/lib:/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/build-apache-arrow/Library/Homebrew/vendor/bundle-standalone/bundler/../ruby/2.3.0/gems/activesupport-5.2.1/lib:/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/build-apache-arrow/Library/Homebrew/vendor/bundle-standalone/bundler/../ruby/2.3.0/gems/tzinfo-1.2.5/lib:/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/build-apache-arrow/Library/Homebrew/vendor/bundle-standalone/bundler/../ruby/2.3.0/gems/thread_safe-0.3.6/lib:/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/build-apache-arrow/Library/Homebrew/vendor/bundle-standalone/bundler/../ruby/2.3.0/gems/minitest-5.11.3/lib:/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/build-apache-arrow/Library/Homebrew/vendor/bundle-standalone/bundler/../ruby/2.3.0/gems/i18n-1.1.0/lib:/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/build-apache-arrow/Library/Homebrew/vendor/bundle-standalone/bundler/../ruby/2.3.0/gems/concurrent-ruby-1.0.5/lib:/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/build-apache-arrow/Library/Homebrew/vendor/portable-ruby/2.3.7/lib/ruby/gems/2.3.0/gems/did_you_mean-1.0.0/lib:/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/build-apache-arrow/Library/Homebrew/vendor/portable-ruby/2.3.7/lib/ruby/site_ruby/2.3.0:/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/build-apache-arrow/Library/Homebrew/vendor/portable-ruby/2.3.7/lib/ruby/site_ruby/2.3.0/x86_64-darwin9.0:/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/build-apache-arrow/Library/Homebrew/vendor/portable-ruby/2.3.7/lib/ruby/site_ruby/2.3.0/universal-darwin9.0:/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/build-apache-arrow/Library/Homebrew/vendor/portable-ruby/2.3.7/lib/ruby/site_ruby:/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/build-apache-arrow/Library/Homebrew/vendor/portable-ruby/2.3.7/lib/ruby/vendor_ruby/2.3.0:/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/build-apache-arrow/Library/Homebrew/vendor/portable-ruby/2.3.7/lib/ruby/vendor_ruby/2.3.0/x86_64-darwin9.0:/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/build-apache-arrow/Library/Homebrew/vendor/portable-ruby/2.3.7/lib/ruby/vendor_ruby/2.3.0/universal-darwin9.0:/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/build-apache-arrow/Library/Homebrew/vendor/portable-ruby/2.3.7/lib/ruby/vendor_ruby:/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/build-apache-arrow/Library/Homebrew/vendor/portable-ruby/2.3.7/lib/ruby/2.3.0:/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/build-apache-arrow/Library/Homebrew/vendor/portable-ruby/2.3.7/lib/ruby/2.3.0/x86_64-darwin9.0:/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/build-apache-arrow/Library/Homebrew/vendor/portable-ruby/2.3.7/lib/ruby/2.3.0/universal-darwin9.0:/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/build-apache-arrow/Library/Homebrew -- /private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/build-apache-arrow/Library/Homebrew/build.rb /private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/build-apache-arrow/Library/Taps/autobrew/homebrew-core/Formula/boost.rb --verbose
==> Downloading https://dl.bintray.com/boostorg/release/1.67.0/source/boost_1_67_0.tar.bz2
/usr/bin/curl -q --show-error --insecure --user-agent Homebrew/1.X.Y\ \(Macintosh\;\ Intel\ Mac\ OS\ X\ 11.6.2\)\ curl/7.64.1 --fail --silent --location --remote-time --continue-at 0 --output /var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/downloads/d08cab6533547504db34623190d93eb217086eb76b2f39ad79d3e9301b40b41e--boost_1_67_0.tar.bz2.incomplete https://dl.bintray.com/boostorg/release/1.67.0/source/boost_1_67_0.tar.bz2
curl: (22) The requested URL returned error: 502 Bad Gateway
Error: Error: An exception occured within a child process:
  DownloadError: Failed to download resource "boost"
Download failed: https://dl.bintray.com/boostorg/release/1.67.0/source/boost_1_67_0.tar.bz2

created /var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T//build-apache-arrow/lib/libbrewaws-c-auth.a
created /var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T//build-apache-arrow/lib/libbrewaws-c-cal.a
created /var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T//build-apache-arrow/lib/libbrewaws-c-common.a
created /var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T//build-apache-arrow/lib/libbrewaws-c-compression.a
created /var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T//build-apache-arrow/lib/libbrewaws-c-event-stream.a
created /var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T//build-apache-arrow/lib/libbrewaws-c-http.a
created /var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T//build-apache-arrow/lib/libbrewaws-c-io.a
created /var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T//build-apache-arrow/lib/libbrewaws-c-mqtt.a
created /var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T//build-apache-arrow/lib/libbrewaws-c-s3.a
created /var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T//build-apache-arrow/lib/libbrewaws-checksums.a
created /var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T//build-apache-arrow/lib/libbrewaws-cpp-sdk-cognito-identity.a
created /var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T//build-apache-arrow/lib/libbrewaws-cpp-sdk-config.a
created /var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T//build-apache-arrow/lib/libbrewaws-cpp-sdk-core.a
created /var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T//build-apache-arrow/lib/libbrewaws-cpp-sdk-identity-management.a
created /var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T//build-apache-arrow/lib/libbrewaws-cpp-sdk-s3.a
created /var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T//build-apache-arrow/lib/libbrewaws-cpp-sdk-sts.a
created /var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T//build-apache-arrow/lib/libbrewaws-cpp-sdk-transfer.a
created /var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T//build-apache-arrow/lib/libbrewaws-crt-cpp.a
created /var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T//build-apache-arrow/lib/libbrewtesting-resources.a
created /var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T//build-apache-arrow/lib/libbrewy.a
created /var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T//build-apache-arrow/lib/libbrewlz4.a
created /var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T//build-apache-arrow/lib/libbrewcrypto.a
created /var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T//build-apache-arrow/lib/libbrewssl.a
created /var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T//build-apache-arrow/lib/libbrewsnappy.a
------------------------- NOTE ---------------------------
There was an issue preparing the Arrow C++ libraries.
See https://arrow.apache.org/docs/r/articles/install.html
---------------------------------------------------------
ERROR: configuration failed for package ‘arrow’
* removing ‘/Users/runner/work/crossbow/crossbow/arrow/r/check/arrow.Rcheck/arrow’

1 error ✖ | 0 warnings ✔ | 0 notes ✔

Note: this also does not (yet) get the formulae in sync or add a check that they are still in sync with the homebrew / autobrew repos.

@jonkeane

Copy link
Copy Markdown
Member Author

@github-actions crossbow submit homebrew-r-autobrew

@github-actions

Copy link
Copy Markdown

@github-actions

Copy link
Copy Markdown

⚠️ Ticket has not been started in JIRA, please click 'Start Progress'.

@github-actions

Copy link
Copy Markdown

Revision: 5054ddce78d1d8811a4b1d8830a8a09deda31e25

Submitted crossbow builds: ursacomputing/crossbow @ actions-1397

Task Status
homebrew-r-autobrew Github Actions

@jonkeane

jonkeane commented Feb 4, 2022

Copy link
Copy Markdown
Member Author

@github-actions crossbow submit homebrew-r-autobrew

@github-actions

github-actions Bot commented Feb 4, 2022

Copy link
Copy Markdown

Revision: b26b082

Submitted crossbow builds: ursacomputing/crossbow @ actions-1586

Task Status
homebrew-r-autobrew Github Actions

@jonkeane

jonkeane commented Feb 7, 2022

Copy link
Copy Markdown
Member Author

@github-actions crossbow submit homebrew-r-autobrew

@github-actions

github-actions Bot commented Feb 7, 2022

Copy link
Copy Markdown

Revision: 1592565

Submitted crossbow builds: ursacomputing/crossbow @ actions-1601

Task Status
homebrew-r-autobrew Github Actions

kou pushed a commit that referenced this pull request Aug 17, 2022
…nt updates (#13769)

Building on #12157

Lead-authored-by: Jacob Wujciak-Jens <[email protected]>
Co-authored-by: Jonathan Keane <[email protected]>
Signed-off-by: Sutou Kouhei <[email protected]>
zagto pushed a commit to zagto/arrow that referenced this pull request Oct 7, 2022
…nt updates (apache#13769)

Building on apache#12157

Lead-authored-by: Jacob Wujciak-Jens <[email protected]>
Co-authored-by: Jonathan Keane <[email protected]>
Signed-off-by: Sutou Kouhei <[email protected]>
@paleolimbot

Copy link
Copy Markdown
Member

I came across this in a "browsing old PRs tagged with R" mission...is this still relevant?

@jonkeane

jonkeane commented Jan 5, 2023

Copy link
Copy Markdown
Member Author

this also does not (yet) get the formulae in sync or add a check that they are still in sync with the homebrew / autobrew repos.

Getting to a point where ^^^ happens would be very helpful for keeping up with changes downstream + making sure we don't break anything. And part of that would be to do many of the updates here (if they are still in the downstream sources...). But I don't think I'll be able to get back to this to do so, so I'll close it out — but we should still come back to that issue

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants