m4-sme-exploration README

The project's README from github.com/tzakharko/m4-sme-exploration. As one markdown file: README.md.

Exploring SME performance of Apple M4

Introduction

The 2024 iPad Pro, featuring the Apple M4 chip, is the first device available to the public that supports ARM's scalable matrix extension (SME). Although Apple has included a matrix accelerator in its devices since 2019, it used a proprietary instruction set called AMX[^1]. The developers had to use Apple-provided numerical frameworks to take advantage of the hardware. The M4 chip with SME changes this by allowing programmers to write low-level code that directly targets the matrix hardware, bringing potential performance improvements to scientific and machine learning algorithms.

My investigation aims to explore the M4 SME hardware and study the SME instruction set. I am particularly interested in how these features can be used to accelerate vector operations. The initial experiments study the peak compute throughput for common matrix and vector operations and the memory transfer rates using various methods. My methodology relies on generating multiple variants of microbenchmark code over a wide range of parameters and studying the results (see the Python code in tools/ for how the benchmarks are generated). I expect to extend this project as new aspects of the hardware are tested. Pull requests and commentary are welcome!

Another in-depth investigation of M4 SME is provided by a team at the University of Jena, who are building a high-performance GEMM implementation [^2]. It is an excellent read if you are interested in the topic.

[^1]: https://github.com/corsix/amx - description of Apple proprietary AMX extensions [^2]: https://scalable.uni-jena.de/opt/sme/

Contents

Running the tests

Requirements

Note: this project currently only runs on an external M4 iPad Pro. As M4 Macs are expected to be available soon, it would be desirable to run these tests locally. Pull requests are welcome!

Building and running

After downloading or cloning the project, perform the following steps

# setup a paired M4 iPad and the code signing information
./setup
# compile the project and install it on the iPad
make build
# run the tests — this will take a while!
make run
# copy the benchmark results
make copy_results

After the testing is done, the generated JSON reports are copied from the iPad and placed in the results/ folder. If you have R and Quarto installed, you can also build the R-Markdown reports using make reports.

By default, both single-core and multi-core tests are executed. This can take a long time. If you are only interested in peak single-core rates, you can change the second line in src/tests.swift from let multiCoreTests = true to false and rebuild.

m4-sme-exploration (tzakharko/m4-sme-exploration)

Searching Database…

245 ·GitHub
M4-sme-exploration
What is this?A research project that microbenchmarks the Scalable Matrix Extension (SME) of Apple's M4 chip, measuring peak compute throughput for common matrix and vector operations and memory transfer rates.
So is it a finished app, or more of a lab notebook with code?It is an ongoing investigation: Swift microbenchmarks plus Python scripts that generate test variants, with collected JSON results and R-Markdown reports rendered via Quarto.
How do I run these benchmarks myself?Pair an M4 iPad Pro with your Mac, install XcodeGen (brew install xcodegen) and PyYAML, generate and build the tests in Xcode, then run them — single-core and multi-core by default, which can take a long time.
What do I need to have installed?An M4 iPad Pro in Developer Mode paired with your Mac, Apple Xcode with Command Line Tools, XcodeGen, Python3 with PyYAML, and Quarto with R/tidyverse if you also want the rendered reports.
Can I run it on an M4 Mac instead of an iPad?Not yet — the project currently only runs on an external M4 iPad Pro. The author plans to run the tests locally once M4 Macs are available.
What exactly do the benchmarks measure?Peak compute throughput for common matrix and vector operations, plus memory transfer rates using various methods — the initial experiments cover outer-product microbenchmarks.
Do the tests use all CPU cores by default?Yes, both single-core and multi-core tests run and this can take a long time. If you only want peak single-core rates, set let multiCoreTests = false in src/tests.swift and rebuild.
Is the project finished?No — the author expects to extend it as new aspects of the hardware are tested, and pull requests and commentary are welcome.
Where can I read a deeper dive into M4 SME?The README recommends an in-depth investigation by a University of Jena team that is building a high-performance GEMM implementation on M4 SME, calling it an excellent read on the topic.
What are similar projects?cpu-performance-engineering, simdjson, ANE, backburner, ggml, try-omarchy, darwin-xnu, Singeli
Which topics is it in?aarch64, simd, benchmarking, science, cpu, arm, scientific-computing, arm64, low-level, xcode, systems-programming, apple
Creator
tzakharko
Size
13.6 MB
Stars
245
Updated
CommitSep 02
Age
2y
Commits · 90 Days