Boris got a new laptop.
An M5 Max, an engine that had never seen one, and sixty-two kernels that had to earn their place on it again.
Boris got a new laptop. An M5 Max, at the end of August, and it became the main machine the same afternoon. dasLLAMA had never run on an M5. Anton lent us his Mac mini - an M4 Pro, the previous generation, sitting on the other side of the globe and reached over ssh - so we could watch the two chips side by side. And we got to work.
What a new chip is, to this engine
dasLLAMA does not ship kernels tuned for a chip. For every heavy operation - a matrix multiply, an attention step, a normalization - it ships several ways of computing the same thing. The first time it runs on a machine it tries them all, keeps the fastest of each, and writes the picks into a small file called a sidecar. Sixty-two such choices for the M5 Max. So "supporting the M5" was not a port. It was letting the tuner run and then looking hard at every place where the pick had changed.
Three places did.
The M5's GPU has tensor units - dedicated matrix hardware, reached through Apple's Metal 4 tensor API. For each matrix multiply in the engine we wrote a tensor-unit version next to the plain one and let the machine choose. On the M5 Max the tensor version came out ahead in 23 of them: every multiply over the compressed 4-, 5-, 6- and 8-bit weights, the 16-bit ones, the expert layers of the mixture-of-experts models, and the attention scores. The choice lives in the sidecar, not in the code: a machine that never tried keeps the plain version, and one that tried and found it slower keeps it too.
On the CPU side, the M5 Max has no Efficiency cores at all. It has 6 cores Apple calls Super and 12 it calls Performance, and all of them are the real thing. On Anton's M4 we had taught the CPU lane to leave the Efficiency cores out of the worker pool, because they only slowed everyone down. On the M5 that habit would have quietly thrown away a third of the chip. Now all 18 work on a prompt, and token generation keeps the six Super cores and parks the twelve Performance ones, which would only get in the way of the fast six. We could not believe our eyes, so we checked it twice, on both machines.
The vision towers and the speech-to-text encoders run on the GPU on the M5 the way the text path does. Nothing new in the design - the same GPU plumbing - but each one had to be tried again and its picks re-made.
What we measured
So we measured it on the M5 and - look at that. What a beast of a machine.
| model | Anton's M4 Pro (pp512 / tg128) | Boris's M5 Max (pp512 / tg128) | M5 / M4 |
|---|---|---|---|
| gemma-4-E4B Q8_0 | 769.7 / 45.4 | 5303.2 / 90.1 | 6.9x / 2.0x |
| gemma-4-12B Q4_K_M | 290.8 / 29.9 | 2077.0 / 63.7 | 7.1x / 2.1x |
| gemma-4-26B-A4B Q4_K_M | 778.2 / 56.4 | 3870.0 / 118.1 | 5.0x / 2.1x |
| gpt-oss-20b mxfp4 | 853.4 / 78.7 | 3995.1 / 162.7 | 4.7x / 2.1x |
| Mistral-Small-24B Q4_K_M | 141.2 / 17.3 | 1163.7 / 38.3 | 8.2x / 2.2x |
| Qwen3-30B-A3B Q4_K_M | 771.1 / 86.1 | 3752.9 / 165.7 | 4.9x / 1.9x |
| Qwen3.8-27B Q4_K_M | 126.4 / 12.9 | 913.9 / 27.8 | 7.2x / 2.2x |
One chip generation apart, the same models, the same engine. Prompt processing moved five to eight times. Token generation, which is bound by memory bandwidth, about twice.
The models that do more than text ride the same machine, the same day. gemma-4-12B's vision tower takes 7.1 ms where llama.cpp's multimodal path takes 95 ms; Qwen2.5-Omni's window vision transformer 106 ms against 118 ms.
Speech, too. whisper large-v3-turbo transcribes the short JFK clip that whisper.cpp ships as its sample in 140 ms, against whisper-cli's 153 ms on the same GPU. Canary-Qwen, NVIDIA's speech-to-text model, in its new 8-bit form does the same clip in 307.6 ms on the M5 and 512.9 ms on the M4, where NVIDIA's own NeMo on Torch MPS takes 704.7 ms and 1076.5 ms. On the long gb1 clip we read 6484 ms and NeMo 4383 ms, which gives us something to do during the long winter evenings.
Glad we have llama.cpp, whisper.cpp and NeMo numbers next to ours. Those are the industry standard for this kind of thing, and they are the only way we know whether we are fast, and the only way we know when to stop. Without a yardstick a new chip is just a machine that runs, and every one of these numbers would be a little worse.
Sixty-two kernels walked into a new chip.
Most walked out the same. Twenty-three swapped to the tensor units. A few found out, in the first afternoon, that the form that had carried them for two generations was now the slow one, and had to be rewritten before the tuner had anything worth picking. None of this is visible from a spec sheet. It all happily went into the tuner log, and so did our last week - which probably says something about us.