WHY TF AM I DOING THIS
It’s a long story. Right after finishing and submitting my first research paper 1, I did not opt to take a break. My boss asked me to, but I felt like doing research is some sort of resting for me. Plus, I would give myself a day off when I love to.
So, what’s next. My labmates and I discussed a little about the unified memory architecture of Apple Silicon. It’s intriguing, and yet we have no idea where to start because little is known about it. Well, thanks to aPPLE’s isolated ecosystem. Although we’ve done some attacks on the UM of Nvidia DGX, the ideas still need polishing. So, we’d start small, by collecting as much information as we can, to hack the Apple Silicon.
Several months of investigating and brainstorming, we found that Apple offers NO documentations nor performance profiling primitives 2. It’s been really hard to reverse engineer the Apple GPU’s architecture, because profiling it at a coarse-grained (eg amplifying or self-loop timer) with (previously practical) microbenchmarks gives unreliable outcomes. The only way is turn to Asahi Linux for insights. Thanks the developers!
Obsolete Docs and Device Support
Due to war of Rust for Linux project, marcan quitted the development, along with the brokeup of Asahi Linux’s core dev, like Lina, Alyssa, etc. They’re facing a really crisis and at this time, nobody’s working on the docs and development. The M3/M4 has been announced years but is not supported. So in cases the docs does not work, we have to trouble-shoot on our own.
Getting things work
So you’ll have to obtain a M1/M2 device (machine to play with) and a M-series Macbook (the terminal) to work on before you can work on it. Mine is a Mac mini (2020) base model with M1, bought at the beginning of my college life, and my daily laptop, a Macbook Air (M3). Mac mini is a perfect toy as I don’t rely on it. Thus, make sure nothing important is stored on the toy machine and then you can play with it whatever you like. 3
Install m1n1
m1n1, as you know it, is a hypervisor, aka, a bare metal program that runs with any access to the chip. It’s readme tells how to compile it, but we don’t have to do this.
Get a playful m1n1 on target machine by installing a Asahi Linux without Asahi Linux
export EXPERT=1
curl https://alx.sh | sh
then follow the instructions, i.e., freeing up disk space for Asahi Linux (20GB is enough), and choose 6. Tethered Boot. This option gets you a m1n1 in proxy mode. The proxy mode gives you a play ground to mess around without those disrupting things, and let you connect to it via a serial port, just like a Raspberry Pi!
Remember the notes before it reboots. If you forgot it, just waits the machine completely shuts down and press the button for ten seconds, release it and wait for the boot option shows up. Choose Asahi Linux and follow the steps. It shall be quite straightforward.
Prepare the terminal
Next, setup the console and terminal machine, the Macbook. You’ll need two things
- m1n1 contains the python scripts. So don’t need to compile it.
- [macvdmtool] is the console to remotely reboot and enable serial debug port on M1/M2.
Get a USB3 dual Type-C cable (the ones from iPhone is good), and plug in the DFU port. The DFU can be identified here. Intuitively, it’s the nearest Type-C port to the power port. Compiles macvdmtool to see if it works
git clone https://github.com/AsahiLinux/macvdmtool
cd macvdmtool && make
sudo ./maccdvtool reboot serial
output
Mac type: J613AP
Looking for HPM devices...
Found: IOService:/AppleARMPE/arm-io@10F00000/AppleH15IO/nub-spmi-a0@D4908000/AppleSPMIController/hpm0@C/AppleHPMARMSPMI
Connection: Sink
Status: APP
Unlocking... OK
Entering DBMa mode... Status: DBMa
Putting target into serial mode... OK
Putting local end into serial mode... OK
Exiting DBMa mode... OK
If you don’t see this, check if you get the cable right and plugged.
Handshake with m1n1
If Running proxy... shows up after you reboot with macvdmtool, you nailed it. The final step,
is to connect. The device appears like /dev/cu.usbmodem*, check if it exists
ll /dev/cu.usbmodemC07JK1Q4Q6NV1
setup m1n1’s playground
git clone https://github.com/AsahiLinux/m1n1
cd m1n1
pip3 install --break-system-packages pyserial construct serial.tool
then use it to get a shell
export M1N1DEVICE=<path-to-device>
python3 proxyclient/tools/shell.py
you’ll see
Fetching ADT (0x00058000 bytes)...
m1n1 base: 0x803708000
Have fun!
>>>
Play with m1n1
I’m still digging into this part. Spell your magic!
We’ve seen some researches with this, like reverse engineering on PAC (demystify Apple PAC, PACMAN attack) , Augury/GoFetch and m1racle (accidentally discovered by marcan during m1n1 development). So chances are you can discover something new with m1n1!
Yes, it got accepted! I’m still writing up for it, for those who are not expert in this area. ↩︎
There’s still one, the timestamp counter, yet it can only profile kernel executing other than CUDA’s
clock(). Ugh, the counter is useless. I loveclock(), but Apple not. ↩︎I’ve reinstalled the macOS 3 times before m1n1 works. Be careful! ↩︎