Thank you for the report, @slashrsm. There are known issues with disk speed on write-heavy workloads, which we are already talking to Apple about.

There are two more things that would be interesting to test if you are so inclined.

1. Can you reproduce the slowness inside a VM created using https://github.com/evansm7/vftool, without any Docker code?
2. We posted an experimental Intel build to the dev preview channel last week that supported switching between the new virtualization.framework and the old hypervisor.framework. Do you see much worse performance in the former than the latter?

Both of these would suggest that it is a MacOS problem rather than a Docker problem.

@stephen-turner Thank you.

1. Can you reproduce the slowness inside a VM created using https://github.com/evansm7/vftool, without any Docker code?

I will try and report back.

2. We posted an experimental Intel build to the dev preview channel last week that supported switching between the new virtualization.framework and the old hypervisor.framework. Do you see much worse performance in the former than the latter?

I did try it for some of the benchmarks (Gzip and database import) and the performance was comparable to hypervisor.framework build.

@rfay Suggested to add benchmark for Drupal 9 Umami database dump (also publicly available), so I added that.

@stephen-turner Excuse me, but it turns out that I didn't enable the experimental virtualization.framework feature when I was testing last time. Tried it again with Drupal 9 Umami import (benchmark 3.) and I was able to see similar regression on Intel as I am seeing on M1:

• Intel hypervisor.framework: 3.129s
• Intel virtualization.framework: 1m 41.14s
• M1: 1m 19.77s

I managed to test the database import in a bare virtualization.framework VM (followed these steps). I imported Drupal D9 Umami database, which took 3.129s on Intel Docker and 1m 19.77s on M1 Docker. I got the following results:

Import into MariaDB: 2m15.416s
Import into MySQL: 10m 19.303s

Ubuntu 20.04 comes with MariaDB 10.3, while I used 10.5 in my previous benchmarks. I am not sure why it is even slower, but it could be to the older version.

It does seem that the problem is in the virtualization.framework itself though.

Thanks for that, @slashrsm, that's really helpful. I think that confirms our suspicions. As I said above, we've talked to Apple about this too. We're also wondering if we can speed it up by flushing less often: there would be a slight risk of lost data if there was a power cut or something, but in a development context that might be an acceptable trade-off.

cc @djs55 for visibility

This is really quite an amazing slowness on a system that in most ways is way faster than anything else. Such a surprise every time I import a database to see it run about 10x slower than amd64.

I don't think this is specific to database performance, Just experienced this myself where it took 484 seconds to chmod a NodeJS + PHP project consisting of 450 MB in extracted packages.

Pinging a static webpage running inside of the docker container using wrk resulted in the following benchmarks on my MBP M1.

elycin@Elys-MacBook-Pro wrk % ./wrk -t8 -c1k -d30s http://localhost:80/
Running 30s test @ http://localhost:80/
  8 threads and 1000 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     1.33s   331.76ms   2.00s    73.96%
    Req/Sec    13.99     10.17    70.00     80.65%
  2820 requests in 30.11s, 578.32KB read
  Socket errors: connect 0, read 3241, write 0, timeout 2313
  Non-2xx or 3xx responses: 2820
Requests/sec:     93.67
Transfer/sec:     19.21KB

It seems to be related to slow I/O in some cases. It is listed under known issues for the latest preview.

Using tmpfs helps a lot, which may be useful for development purposes at least.

Here is a comparison of running a pgBackRest integration test (which runs Postgres instances) on Intel (2020 iMac) and M1 (2020 Air):

test/test.pl --vm=co7 --module=real --clean --log-level-test-file=off

normal:

Intel VirtualBox: 224s
Intel Docker: 136s
M1 Docker: 980s-1600s (sometimes hangs)

tmpfs:

Intel VirtualBox: 120s
Intel Docker: 124s
M1 Docker: 130s

Just a happy note that @djs55 's current experimental build seems to have a huge effect on this issue as well. Massively faster to do a db import. See #5476 (comment)

Thanks, @rfay, that's very interesting.

Everyone watching this thread, please try the new RC2 build at https://docs.docker.com/docker-for-mac/apple-m1/ and give us feedback whether it solves this problem (and whether it causes any new problems!). Thank you.

Just tried the RC2 and first impressions are very good – test suite now runs faster than on Intel, previously it was 10x slower or more. Will report back if any other problems become apparent!

Yes, that helps a lot.

Here is the updated comparison of running a pgBackRest integration test (which runs Postgres instances) on Intel (2020 iMac) and M1 (2020 Air):

test/test.pl --vm=co7 --module=real --clean --log-level-test-file=off

normal:

Intel VirtualBox: 224s
Intel Docker: 136s
M1 Docker: 135s

tmpfs:

Intel VirtualBox: 120s
Intel Docker: 124s
M1 Docker: 125s

There is one problem I noticed however. /dev/null is being created as read/write only for root:

docker@pgbackrest-test:~$ ls -lah /dev/null
crw-rw---- 1 root root 1, 3 Mar 26 17:59 /dev/null

I had to fix this before I could run our tests with:

rm /dev/null
mknod /dev/null c 1 3
chmod 666 /dev/null

Presumably this has something to do with qemu but it certainly represents a behavioral change from Intel Docker or prior M1 versions. Not a Docker expert so perhaps there is something I am missing here.

Thanks @dwsteele, we'll look into the /dev/null issue.

RC2 is way faster for me.

Apple M1 MacBook Pro 16GB RAM

Resources

CPU: 2
RAM: 6 GB

Import of an 2GB Database:

RC1: ~120 Min
RC2: ~53 Min

This is a huge improvement for me.
I'm still stuck with MySQL 5.7, so Rosetta 2 is probably the reason why its still 50 min. On lInux its about 10 Min.

Edit:
MariaDB 10.5 takes only 5 min 😁

PHPUnit tests with a lot I/O (sqlite Fixtures) in an not mounted folder

RC1: ~30 Min
RC2: ~3 Min

Again big improvement.

thx alot

Thanks for the new release, it's a massive improvement. The time taken for a Python test suite to run is equivalent between using tmpfs and a volume with RC2, whereas using a volume on prior versions would take perhaps 10 times as long to run.

I can also confirm that I saw a massive improvement with RC2 on qemu.

@dwsteele thanks for your report. I'm perplexed by the /dev/null problem -- could you share a small repro example to demonstrate the issue? For me whenever I run a container I see normal permissions e.g.:

% docker run -it alpine sh
/ # ls -l /dev/null
crw-rw-rw-    1 root     root        1,   3 Mar 29 13:21 /dev/null

The strange thing is that the choice of hypervisor (qemu vs virtualization.framework vs hyperkit) should not affect the filesystem presented to containers. Perhaps different timing has exposed a race condition somewhere?

@djs55 I have attached the Dockerfile. We are testing this for dev purposes on Intel/M1 Macs (currently we use Vagrant/VirtualBox on Intel). It's pretty unconventional but worked OK until the issue with the qemu-based M1 Docker.

Dockerfile.txt

Everyone watching this thread, please try the new RC2 build at https://docs.docker.com/docker-for-mac/apple-m1/ and give us feedback whether it solves this problem (and whether it causes any new problems!). Thank you.

Is this considered a hybrid emulation approach? Is there a potential future release that would try to utilize BigSur's virtualization.framework?

@rovo79 The previous builds all used virtualization.framework: this build allows you to swap between qemu and virtualization.framework.

Everyone watching this thread, please try the new RC2 build at https://docs.docker.com/docker-for-mac/apple-m1/ and give us feedback whether it solves this problem (and whether it causes any new problems!). Thank you.

I've updated my Docker installation to RC2 and database performance has improved drastically.
With RC1 restoring a Postgres backup clocked about 1 hour, with RC2 it clocks under 3 minutes 🎉

Overall this feels snappier, but applying Postgres migrations is still really slow to the point that my project doesn't even boot (it boots after all migrations are applied, but I manually stop it at 30m since there is no point waiting more than a couple of minutes). Migrations create more than 140 tables, they also add and remove some columns.

I have tried both with "Use new virtualization framework" option enabled and disabled. I also completely uninstalled Docker (using the Uninstall option from the menu, then dragging the app to the trash can) before installing RC2.

Overall this feels snappier, but applying Postgres migrations is still really slow to the point that my project doesn't even boot (it boots after all migrations are applied, but I manually stop it at 30m since there is no point waiting more than a couple of minutes). Migrations create more than 140 tables, they also add and remove some columns.

I have to say that my experience has been quite the opposite with regards to Postgres, although I don't have anywhere near as many tables as that (only 35). Prior to RC2, the migrations would take 1-2 minutes, but now it's 1-2 seconds.

One thing I have noticed though, is that RC2 has a much more significant impact on battery life. I assume this is because virtualization.framework is well optimized in this regard vs QEMU?

Probably, yes. We have noticed slightly higher CPU with qemu.

I just downloaded Preview RC3 and I have still issues running migrations in apple silicon.

@stephen-turner GA 3.3.1 fixed the issue I had with /dev/null permissions. I see it in the release notes but just wanted to report that it worked in my case as well. Thanks!

I'm going to close this ticket now. The original issue is fixed by making virtualization framework no longer the default.

Closed issues are locked after 30 days of inactivity.
This helps our team focus on active issues.

If you have found a problem that seems similar to this, please open a new issue.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows.
/lifecycle locked