Getting Dual MI50 32GB Cards Working with llama.cpp ROCm on Ubuntu 22.04

r/LocalLLaMA
Generative AI AI Hardware Open Source AI

I've been banging my head against this for a while now, so I figured I'd write up what actually worked before I forgot half of it. This is for anyone running dual AMD Instinct MI50 32GB cards (gfx906) and trying to get ROCm inference working in llama.cpp. Spoiler: the official docs won't get you there. There are several layers of problems stacked on top of each other, and you need to fix all of them. It took way longer than it should have, and at multiple points I genuinely considered throwing the cards out a window.