This issue/PR has been automatically marked as stale because it has not had any update (including commits, comments, labels, milestones, etc) for 30 days. It will be closed automatically if no further update occurs in 7 day. Thank you for your contributions!

This PR is blocked on #25794, please don't close.

I've rebased this onto latest master and removed the "draft" status. Would be very thankful for your feedback.

I've kicked off a package build in https://fusion2.corp.google.com/invocations/06b5347f-0977-4949-a26f-aaaefd0650f3/targets. Using those packages, I'll need to find a way to test these

@flavorjones we'd also be interested in having an arm64 ruby gem for linux (see tracking issue - #26391). AFAIK this should now be possible with rake-compiler-dock.
Would you mind looking into that? (in a separate PR).

@jtattermusch Yes, it's possible, but there hasn't been an official release of rake-compiler-dock with rake-compiler/rake-compiler-dock#54 yet. I'll take the responsibility of submitting a PR once that's happened.

Looks like CI failed with unrelated timeouts. Can someone kick it again?

What's holding this?

CI failure now is the same issue previously seen at #25794 (comment)

I'll try to find some time to git-bisect this and figure out why it's popped up for this changeset.

@sobrinho If you want to help figure this out, I'd be happy to have your help. Thanks for your patience.

@jtattermusch I'm not able to reproduce this in my dev environment, so I rebased onto master. Would you mind kicking CI once more?

The Python Windows test failure is not related to this PR, and reads:

rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1677) [generator=3.1.3]

Otherwise, the previously-failing "Artifact Build" jobs seem to be green, so 🤞

https://fusion2.corp.google.com/invocations/06b5347f-0977-4949-a26f-aaaefd0650f3/targets

The arm64 gem from the adhoc build is here:
https://storage.googleapis.com/grpc-testing-kokoro-prod/test_result_public/prod/grpc/core/master/linux/grpc_build_packages/17740/20211019-233857/github/grpc/artifacts/grpc-1.42.0.dev-arm64-darwin.gem

Can someone try that gem on an arm64 mac (I don't have one) and report back whether it works fine?

Also, the distribtests have passed:
https://fusion2.corp.google.com/invocations/563f2c07-1752-42b6-b56a-e0c99cf286f5/targets/grpc%2Fcore%2Fexperimental%2Fgrpc_distribtests_multiplatform/log

Marvelous! Worked perfectly!

Marvelous! Worked perfectly!

More details on what exactly you tried and what was the result is welcome.

@jtattermusch I created one ruby app connected to the pub sub emulator pushing messages to a few topics and another ruby application listening to those topics.

I ran the apps I have in production and checked if they worked locally on M1 and they did 👍

@jtattermusch @flavorjones thanks for this fix! This will solve a major pain point - new hires with new laptops can't install grpc without this fix.

When do you think this will be released in a new gem as 1.42.0?

@jtattermusch @apolcyn I've gotten a few reports from folks within Shopify that the 1.42.0.pre1 arm64-darwin gem isn't working correctly. (The x86_64-darwin gem seems OK.) You may want to hold off on releasing the arm64-darwin gem in the 1.42.0 final release.

I don't have a good repro yet but the evidence is that, within a running Rails server, we see:

dyld[58650]: missing symbol called

and the crash report includes this stack walkback:

Thread 22 Crashed:
0   dyld                          	       0x104da34dc __abort_with_payload + 8
1   dyld                          	       0x104daa4dc abort_with_payload_wrapper_internal + 104
2   dyld                          	       0x104daa510 abort_with_payload + 16
3   dyld                          	       0x104d628e4 dyld4::halt(char const*) + 328
4   dyld                          	       0x104d80f7c dyld4::APIs::_dyld_missing_symbol_abort() + 44
5   grpc_c.bundle                 	       0x114bcab8c 0x114954000 + 2583436
6   libsystem_pthread.dylib       	       0x194c3929c __pthread_once_handler + 76
7   libsystem_platform.dylib      	       0x194c51968 _os_once_callout + 32
8   libsystem_pthread.dylib       	       0x194c39234 pthread_once + 100
9   grpc_c.bundle                 	       0x114bf9c1c 0x114954000 + 2776092
10  grpc_c.bundle                 	       0x114bcac8c 0x114954000 + 2583692
[...]

We'll keep working on finding an easier repro, and if I get one we'll open a new issue.

@jtattermusch @apolcyn I've gotten a few reports from folks within Shopify that the 1.42.0.pre1 arm64-darwin gem isn't working correctly. (The x86_64-darwin gem seems OK.) You may want to hold off on releasing the arm64-darwin gem in the 1.42.0 final release.

I don't have a good repro yet but the evidence is that, within a running Rails server, we see:

dyld[58650]: missing symbol called

and the crash report includes this stack walkback:

Thread 22 Crashed:
0   dyld                          	       0x104da34dc __abort_with_payload + 8
1   dyld                          	       0x104daa4dc abort_with_payload_wrapper_internal + 104
2   dyld                          	       0x104daa510 abort_with_payload + 16
3   dyld                          	       0x104d628e4 dyld4::halt(char const*) + 328
4   dyld                          	       0x104d80f7c dyld4::APIs::_dyld_missing_symbol_abort() + 44
5   grpc_c.bundle                 	       0x114bcab8c 0x114954000 + 2583436
6   libsystem_pthread.dylib       	       0x194c3929c __pthread_once_handler + 76
7   libsystem_platform.dylib      	       0x194c51968 _os_once_callout + 32
8   libsystem_pthread.dylib       	       0x194c39234 pthread_once + 100
9   grpc_c.bundle                 	       0x114bf9c1c 0x114954000 + 2776092
10  grpc_c.bundle                 	       0x114bcac8c 0x114954000 + 2583692
[...]

We'll keep working on finding an easier repro, and if I get one we'll open a new issue.

Do we know what was the missing symbol?

I don't have more information yet, My employer is shipping me an M1 to reproduce and diagnose, but it hasn't arrived yet. (Hopefully it will arrive this week.)

+1 for this! I can't figure a workaround for ruby 2.6.6 on M1. 2.6.6 won't install with x64 arch.

See #28271 (comment) please

Does anyone know what happened with this? Seems the arm64-darwin gems were never shipped/published after the challenge in 1.42.0-pre so just wondering if the effort was abandoned.

I note #28271 (comment) but it looks like all the problems were resolved. Still, no precompiled arm64-darwin gems on rubygems with 1.57.0 :(

@chadlwilson Yes, I have an update -- was working on this last month in my spare time.

The current issue with arm64-darwin gem support is that stripping the shared object also invalidates the ad-hoc code signature in the file.

I've created a PR to rake-compiler dock at rake-compiler/rake-compiler-dock#104 that fixes this. That PR is nearly ready to merge, I just need to spend another hour or so adding test coverage.

So the order of operations should be, I think:

1. finish strip does ad-hoc code signatures on darwin rake-compiler/rake-compiler-dock#104 and merge it
2. release a patch update for rake-compiler-dock and publish new images
3. this project, grpc, needs to update its build containers to use the new rake-compiler-dock base
4. we should test that the precompiled gems work as expected for arm64-darwin
5. grpc needs to cut a new release or patch release

Make sense?

Interesting - good to hear @flavorjones - thanks for the clarification!

Curiously, in the context of a similar endeavor for aarch64-linux there was a comment at #33079 (comment) that the latest binaries from here for 1.57.0 were working OK on arm64 MacOS (in addition to aarch64-linux, the main topic of that discussion).

But perhaps required ignoring the signature checks if I understand the issue above. I haven't yet tried myself.

Looks like the strip was removed in #33641 according to #33079 (comment) so the extra work here may no longer be required for grpc @flavorjones ?

@chadlwilson 🤷 Yup, looks like that's the case.