Repository navigation
Unable to run test containers with Apple Silicon Mac #3834
Description
% docker version
Client: Docker Engine - Community
Cloud integration: 1.0.8
Version: 20.10.3
API version: 1.41
Go version: go1.13.15
Git commit: 48d30b5
Built: Fri Jan 29 14:28:27 2021
OS/Arch: darwin/amd64 (rosetta)
Context: default
Experimental: true
Server: Docker Engine - Community
Engine:
Version: 20.10.3
API version: 1.41 (minimum version 1.12)
Go version: go1.13.15
Git commit: 46229ca
Built: Fri Jan 29 14:32:07 2021
OS/Arch: linux/arm64
Experimental: false
containerd:
Version: 1.4.3
GitCommit: 269548fa27e0089a8b8278fc4fc781d7f65a939b
runc:
Version: 1.0.0-rc92
GitCommit: ff819c7e9184c13b7c2607fe6c30ae19403a7aff
docker-init:
Version: 0.19.0
GitCommit: de40ad0
Running docker via Docker Desktop Preview Version 3.1.0(60984)
Disabled: Use gRPC FUSE for file sharing
Error:
Caused by: java.lang.IllegalStateException: Could not find a valid Docker environment. Please see logs and check configuration
at org.testcontainers.dockerclient.DockerClientProviderStrategy.lambda$getFirstValidStrategy$7(DockerClientProviderStrategy.java:215)
at java.base/java.util.Optional.orElseThrow(Optional.java:408)
at org.testcontainers.dockerclient.DockerClientProviderStrategy.getFirstValidStrategy(DockerClientProviderStrategy.java:207)
at org.testcontainers.DockerClientFactory.getOrInitializeStrategy(DockerClientFactory.java:136)
at org.testcontainers.DockerClientFactory.client(DockerClientFactory.java:178)
at org.testcontainers.LazyDockerClient.getDockerClient(LazyDockerClient.java:14)
at org.testcontainers.LazyDockerClient.authConfig(LazyDockerClient.java:12)
at org.testcontainers.containers.GenericContainer.start(GenericContainer.java:310)
at example.mycompany.MySql.init(MySql.groovy:16)
at example.mycompany.MySql.<clinit>(MySql.groovy:10)
... 5 more
Using test containers in a Gradle build
testImplementation(platform("org.testcontainers:testcontainers-bom:1.15.2"))
testImplementation("org.testcontainers:mysql")
testImplementation "org.testcontainers:selenium"
Reacted by Łukasz Myśliński, Etibar Hasanov, jerryleooo, iakov-n26, Rafael Lima Joia, Maciek Opala, aosorio1, Sebastian, Gaurav Deshpande, Dušan Živković and 5 more
Activity
bsideup commentedon Feb 25, 2021
@sdelamo please share the full log
ansotolab commentedon Feb 25, 2021
@bsideup I have the same issue and I wrote it here #3810
bsideup commentedon Feb 25, 2021
@ansotolab I do not think you have the same issue. In your case, the environment detection works fine.
ansotolab commentedon Feb 25, 2021
@bsideup sorry it could be my fault because I wrote two comments on the same issue, I'm talking about the first one #3810 (comment) you can see there is an error log dropdown. In it appears:
09:54:24.871 [main] ERROR org.testcontainers.dockerclient.DockerClientProviderStrategy - Could not find a valid Docker environment. Please check configuration. Attempted configurations were:java.lang.IllegalStateException: Could not find a valid Docker environment. Please see logs and check configurationThe second comment #3810 (comment) is a temporary solution. If I run the testcontainers inside Docker I didn't receive any error but it's not the best option for development.
sdelamo commentedon Mar 2, 2021
@bsideup Not sure how if this is the log you mentioned:
bsideup commentedon Mar 2, 2021
@sdelamo it is, thanks :)
Your problem is related to JNA:
Consider either using JNA 5.7.0 that is compatible with M1 or use x86 build of JDK.
ikattey commentedon Mar 2, 2021
I can confirm that adding the following Maven dependency fixed this issue for me:
sdelamo commentedon Mar 3, 2021
Thanks @bsideup and @ikattey
Now I get a different error:
I am able to download this docker image in an Intel Mac.
dizney commentedon Mar 11, 2021
@sdelamo mysql does not provide arm images. For example https://hub.docker.com/_/mariadb does provide them, so you might try that.
rdehuyss commentedon Apr 21, 2021
For those working with Gradle, adding the following solved the issue for me:
testImplementation 'net.java.dev.jna:jna-platform:5.8.0'bsideup commentedon Apr 21, 2021
FYI Testcontainers 1.15.3 is released now and depends on 5.8.0 by default.
rdehuyss commentedon Apr 22, 2021
@bsideup indeed, I see it - however, for me it only works if I add the
testImplementation 'net.java.dev.jna:jna-platform:5.8.0'. Can't explain why yet - maybe because I'm using gradle platform to manage my dependencies?To reproduce, see:
https://github.com/jobrunr/jobrunr/blob/silicon-mac/core/build.gradle#L52-L53
Any test that uses TestContainers will fail like
LettuceRedisStorageProviderbsideup commentedon Apr 23, 2021
@rdehuyss it is really a question to Gradle and not Testcontainers :)
Someone pointed out that
visible-assertionsalso depends on JNA (5.2.0) and that some build systems for some reason prioritize it overdocker-java-transport:#4008 (comment)
To help with that, since
visible-assertionsisn't used by our main classpath anymore, we plan to remove it as a dependency in the next release. You can also try excludingorg.rnorth.visible-assertions:visible-assertionsalready today, just to check whether that's the casewimdeblauwe commentedon Jul 20, 2021
When using Maven with Testcontainers 1.15.3, you can exclude
visible-assertionslike this:11 remaining items
dicolasi commentedon Sep 17, 2021
tried in multiple ways, seems impossible to get those logs.
TranquilMarmot commentedon Sep 20, 2021
I'm on
testcontainersversion1.16.0and it still does not work on my M1 Mac.Here are the logs:
I tried explicitly adding
jnadependencies but they didn't help:Update: After pulling from our main branch, this started working again. I haven't narrowed it down but I think that
org.springframework.data.spring-data-elasticsearchwas pulling in an old/bad version of JNA and we upgraded that dependency (it's now on7.13.4)jonydog commentedon Dec 20, 2021
Also had this issue with M1 Mac, adding
fixed the issue.
add jna-platform library for unit test
kingboy2903 commentedon Mar 10, 2022
I have same issue with M1 Pro. And this will be solved with upgrade test container to newest version and add dependency
jnaAnatoly4444 commentedon Apr 16, 2022
same problem with mac 2021
bibinss commentedon Jun 18, 2022
In my case I was seeing the
java.lang.UnsatisfiedLinkError: Can't load library:even after upgrading jna lib version to 5.7.0.Further debugging showed that I had another library
net.snowflake:snowflake-jdbc.3.12.10in the class path which had the same jna (com.sun.jna...) code in it. So at run time, my JVM was loading the one from snowflake jar causing the failure.I upgraded to latest
net.snowflake:snowflake-jdbc.3.13.19to resolve the issue. It took me a while to figure out the issue, but was relieved finally :)shubhimohta commentedon Jul 13, 2022
For me even after explicitly mentioning the dependency
testImplementation 'net.java.dev.jna:jna-platform:5.8.0', it did not work for me as one of the transitive dependency was using jna version 4.5.1. (transitive dependencies are dependencies that your component needs, but only because another dependency needs them)So I had to force gradle to pick the newer version of jna which worked for me.
Here is the reference to how to declare versions like I did above:
https://docs.gradle.org/current/userguide/rich_versions.html#sec:strict-version
One can select what works for them, but since any version above 5.7 is compatible I have used this. (edited)
JohnnyMarnell commentedon Aug 10, 2022
This thread saved me 🙏 , in our case I needed to exclude JNA jars from very old elasticsearch version, e.g.:
Ended up adding maven enforcer rule like:
JohnnyMarnell commentedon Sep 14, 2022
happening again for me, this time no other JNA though
AndyLau223 commentedon Sep 22, 2022
Thanks for all the active comments. This issue happens on my machine(Mac M1), and the solution of adding
testImplementation 'net.java.dev.jna:jna-platform:5.8.0'into build.gradle file works fine for me, now the tests are running smoothly.harihara6 commentedon Jun 15, 2023
For those running test containers on Mac M2, ( if withPlatform override is not supported on your versions), I did this
Just removed existing images
docker system prune -afI pulled the latest myself
docker pull arm64v8/mysqland tagged it the way I want
docker tag arm64v8/mysql:latest mysql:5.7.22Now the tests are running as expected.
fatso83 commentedon Sep 21, 2023
Just a little note for you others that have tried all the tips above, updating dependencies and exluding jna's to no avail: There's one other thing that will also make the tests fail with
docker-machine executable was not found on PATH...If you try running any Docker command, like
docker psand getcannot connect to the Docker daemon, you have forgotten to start the Docker daemon. Don't ask me how I know 🙈 Start it up and then re-rerun your tests 👍swar8080 commentedon Oct 11, 2023
For anyone using docker desktop that's hitting issues starting the testcontainer/ryuk image, I had to enable the Allow the default Docker socket to be used (requires password) setting to fix it:
eddumelendez commentedon Oct 11, 2023
You don't need that option with latest versions.