Disk Performance

[d-shehu]

Neat project, glad to see Apple is providing greater support for container dev.

One of the major hurdles with running containers (Docker, podman) or VMs on MacOS is relatively slow disk I/O, primarily with random read/write.

How's disk performance and if it's still slow is there a plan for robust passthrough of NVME disks especially over Thunderbolt? Thanks.

[jglogan]

Can you be more specific? Are you referring to disk I/O to the root disk or named volumes (virtioblk mounts)? Or to "bind mounts" (virtiofs mounts)?

What sort of "random read/write" workload are you referring to, and what are you comparing the container tools' performance to?

Knowing that will help us understand where the gaps might lie.

See also this issue recently filed on containerization that shows a more specific comparison of I/O performance (appears to be all for virtioblk devices and not for virtiofs). apple/containerization#729

[d-shehu]

It started with running my Docker stack with bind mounts 5-6 ago on my M1 laptop and continues to this day even with VirtioFS on my M3 Pro. I've tested running the VM on root disk or NVME over Thunderbolt 4. So I gave up on Docker or containers on MacOS. This summarizes things nicely.

https://www.paolomainardi.com/posts/docker-performance-macos-2025/

For the last 3-4 years, I've always run UTM VMs on my Mac so I can leverage my toolchain and keep things consistent with my Linux boxes. Performance was better as was compatibility. And I always used Apple's virtualization since QEMU virtualization with UTM was noticeably slower.

It's been tolerable for my little apps but recently I wanted to do more with (1) nested virtualization for a few special cases. And I've been testing apps that use (2) elastic search, (3) running K3s and minio nodes and other I/O bound use cases.

I especially noticed the sluggish behavior with container overlays in the nested VM scenario when pulling images for podman. It took upwards of 5+ minutes vs 20-30 secs on bare metal system with comparable SSD.

Basic tests revealed random read and writes were ~7-14x slower in the VM on M3 Pro Macbook again with virtualization and Arm Linux. Even with just 1 VM running a minimized headless Linux server!

I saw comparable slowdown with Intel Macbook 2019. So it doesn't seem specific to M series.

utmapp/UTM#7703

With AI bubble, I wanted to leverage spare Macs as Linux server. But aside from doing something crazy like trying to run Proxmox and then MacOS in a VM, I'm hitting a wall. Hope that helps clarify things.

TLDR
What is the best way to run with Apple virtualization? Perhaps I'm doing something wrong when configuring my VM?

What is the expected performance for random reads/writes in a VM vs MacOS host?

Thanks.

[jglogan]

TLDR
What is the best way to run with Apple virtualization? Perhaps I'm doing something wrong when configuring my VM?

"bind mounting" directories from the host uses virtiofs which is pretty slow for heavy I/O workloads like builds where writes are relatively small and there's a fair number of metadata writes.

Using a named volume will be significantly faster - it's using a virtioblk mount of an ext4 disk image file that resides on your host.

It's been a while since I've benchmarked but for builds, I'd typically see native clean build times be about 2-3x faster than virtioblk, which is in turn about 3x faster than virtiofs.

Random vs sequential isn't so much of a factor with SSDs. Very small writes are bad for performance and SSD wear but generally the OS buffer cache should be avoiding that.

[d-shehu]

Thanks, I've been using ext4 disk image as my roaming personal volume to share among machines (Linux VM on Mac/Win vs Linux). But based on limited benchmarking it's actually slower. Since I use it to quickly rsync large amounts of data between machines it's fine for that use case.

I'd hoped that Apple containers would be implemented more like native Linux containers with compatibility for OCI image or the ability to reuse Dockerfiles to build the "same" image on a Mac. Alternatively if we had options with Apple virtualization and could passthrough disks/NICs to the underlying VM that would be great since we could stick with native Linux tooling.

I hope this gets more publicity with 1.0 and we can re-evaluate performance.

[jglogan]

I'd hoped that Apple containers would be implemented more like native Linux containers with compatibility for OCI image or the ability to reuse Dockerfiles to build the "same" image on a Mac.

Apple containers are Linux containers that use a 1:1 container:VM model, as opposed to n:1.

See: https://github.com/apple/container/blob/main/docs/technical-overview.md#how-does-container-run-my-container

You use container build to build an OCI image, and container run to run an OCI image.

It's not clear what this has a to do with the I/O performance. Whether you run a traditional Linux VM using Parallels, UTM, or some other tool, or you run a Linux container in a Linux VM like container, if you're using Virtualization.framework for the VM, the performance is determined by the maximum IOPS and throughput from your workload, through the Linux storage stack and virtioblk driver, and down through the macOS kernel to the storage device on the host.

[d-shehu]

Thank you for clarifying the architecture. I prefer bind mount for portability since persistent data can easily be migrated from machine to machine which is handy for dev and ops.

I was comparing running Docker (or LXD) containers on

1. Bare metal Linux
2. Linux VM on Linux bare metal
3. Linux VM on Mac over UTM
4. Docker containers on MacOS ...

In my limited experience, there is a large performance reduction with disk I/O with Mac hardware due to the virtualization. Devs are often given Mac laptops but need to run Linux toolchains. And Brew is not always the answer.

Anyways, I'd be happy to revisit the performance of Apple containers and compare those to Linux with native volumes and bind mount when it gets to 1.0. But it sounds like I shouldn't expect any miracles.

I'll mark this as answered. Thanks again.

[jonct]

It seems to me that you may have missed a key point that could be valuable to you right away?

Using Linux-native block storage volumes inside Linux VMs is fast — on Mac hardware, with virtualization… all of it. (And using Mac-native block storage within macOS is pretty fast, too.)

Using bind mounts to shuttle data back and forth between any two systems' independent kernels will add performance costs. Even between a Linux guest VM and its Linux host. They burn effort serializing/deserializing messages, switching context, conservatively managing cache coherency on each side… your mental model should be of a network file share.

The choice, then, is yours to make. Don't wait for a miracle — no implementation can or will ever fully close that gap for you.

Fortunately, you can mix and match where appropriate! For instance, I mount node_modules directories as volumes (or as tmpfs) inside bind-mounted project directories. This also helps mitigate significant issues with Time Machine and cloud sync services.

💡 Side note: chflags hidden node_modules and tmutil addexclusion node_modules may be of interest.

Lastly, consider taking a closer look at the tools to export/import the contents of volumes. That's not as difficult as it may seem, and the need is comparatively rare. So in many cases (as with node_modules) trading for speed is an easy win.

[jglogan]

Using Linux-native block storage volumes inside Linux VMs is fast — on Mac hardware, with virtualization… all of it. (And using Mac-native block storage within macOS is pretty fast, too.)

[snip]

The choice, then, is yours to make. Don't wait for a miracle — no implementation can or will ever fully close that gap for you.

This is the best answer.