k3sm licensed apache-2.0 (DCO)

Why k3sm

Every other way to run Kubernetes on a Mac starts by hiding the Mac. k3sm keeps the Kubernetes API and replaces the substrate underneath it.


The established ways to run Kubernetes on a Mac all work the same way: boot a Linux virtual machine, run Linux Kubernetes inside it, and forward some ports back to the host. That works well for developing Linux workloads on a laptop. It does not help when the work itself has to run on macOS.

How Do You Orchestrate Work That Has to Run on macOS?#

Some work cannot leave the Mac. An iOS or macOS build has to happen on Darwin, with Xcode and a real toolchain. Code signing and notarization need Apple’s own tooling and keychain. Metal shader compilation, CoreML, and MLX inference need the GPU and the unified memory that a hypervisor boundary either hides or taxes. Test suites for Apple platforms need the platform.

Today that work is scheduled by whatever grew up around it: a rack of Mac minis, shell scripts, one bespoke CI runner per machine. Kubernetes already handles this shape of problem (declarative workloads, health checking, rolling updates, service discovery, secrets), but it has not been available on the platform where this work has to run.

k3sm keeps the Kubernetes API and replaces the Linux substrate underneath it.

What a Linux VM Costs You#

A VM-based Kubernetes on a Mac costs you four things.

  • The Mac is invisible to your workload. Inside the VM there is no Xcode, no codesign, no keychain, no Metal, no ANE, so anything Darwin-specific has to be shelled back out to the host, which reintroduces the ad-hoc scripting the scheduler was supposed to remove.
  • The GPU and unified memory sit behind a boundary. Apple Silicon’s headline advantage is that the CPU and GPU share memory with no copy, and a guest OS re-introduces the copy, or loses the GPU entirely.
  • You pay twice for the filesystem, because bind-mounting host directories into a Linux guest is the slowest, most-complained-about part of every Mac container stack.
  • A desktop VM’s Kubernetes is one machine, a single-node cluster on your laptop, and it cannot schedule a room full of Macs.

What k3sm Does Instead#

k3sm keeps every piece of Kubernetes that is already portable Go, and rebuilds only the pieces that are Linux-shaped.

  • The control plane is upstream Kubernetes. kube-apiserver, scheduler, and controller-manager build from source for darwin/arm64 and run as supervised children of one binary, with kine over SQLite standing in for etcd. RBAC, admission, CRDs, and workload controllers behave the way they behave everywhere, because they are the same code.
  • The node is a Virtual Kubelet with a Darwin provider. The upstream kubelet cannot even compile for darwin/arm64, so the provider rebuilds the API-facing half (Pod phases and conditions, probes, graceful stop, memory-pressure eviction, logs, exec, top) on top of native processes.
  • The runtime is macOS. A Pod’s containers are posix_spawn’d arm64 Mach-O binaries running at real host paths, each confined by a generated default-deny Seatbelt profile, with an APFS clonefile copy-on-write root. They see the real /System, so dyld and every Apple framework resolve with nothing to plumb.
  • The network is macOS. Pod IPs are lo0 aliases; Services are a userspace proxy that owns each VIP socket; cross-Mac traffic is a wireguard-go mesh over a root-created utun. There is no iptables, no CNI, and no NAT between nodes.

architecture walks the whole path from kubectl apply to a running process.

What You Get#

  • Your workload is a first-class macOS process, reaching full-speed local disk, the Apple frameworks, the GPU, and unified memory directly rather than through a guest.
  • One scheduler covers several Macs, and workers join a wireguard mesh with a single token. Multi-node runs on two Macs so far and is experimental until v0.3.
  • The manifests you already have keep working. Deployments, StatefulSets, Jobs, CronJobs, Services, Ingress, ConfigMaps, Secrets, PVCs, and RBAC run on the same upstream controllers.
  • The default path runs no hypervisor, so it carries no hypervisor overhead.

What It Costs#

  • Your existing Linux images do not start on the default path. A k3sm Pod is a native darwin/arm64 executable, so a Linux image is refused at pull there. Images themselves are fine. Build a Mac binary into one, push or load it, and it runs with the tags, digests and pull secrets you expect (what runs). The payload changes and the workflow does not. A Pod can also name the vm RuntimeClass, which boots a linux/arm64 image in a per-pod micro-VM and reaches Running. linux/amd64 is a later release.
  • k3sm cannot pass CNCF [Conformance]. The suite assumes Linux containers, cgroups, CNI, and network namespaces, and k3sm has none of them. The conformance profile sets out what is claimed instead.
  • A node is one trust domain, and all Pods on it share the unprivileged _k3sm user. Seatbelt bounds reach; it does not give you per-Pod uid isolation or a network namespace. NetworkPolicy is a hint enforced for Service ingress on the server node, and workers enforce none.
  • The resource model is best-effort. Memory is sampled and can drive an OOMKill, while CPU is QoS, not CFS millicores, so CPU limits are not enforced. Under memory pressure the node evicts one Pod at a time, ranked as the kubelet ranks them; there is no disk or PID eviction.
  • A native Pod cannot reach the login keychain, so Developer ID signing and notarization run on the host.
  • The current release is v0.1.6, a pre-release build, ad-hoc signed and not notarized, and the install script installs the newest release. The Homebrew tap and notarized package come with the first stable release.

The full list is limitations; the fair side-by-side against the VM-based options is compare.

Who This Is For#

For an amd64 Linux container, or for mature Linux support, use a Linux VM. An arm64 Linux image can run here too, via the vm RuntimeClass. k3sm is for work that needs the Mac itself: a build farm, a CI fleet of Mac minis, on-device model serving, or any service that has to be a Darwin process and should be scheduled, health-checked, and rolled out like everything else you run.