llama.cpp fixes to run Bonsai 1-bit models on CPU (incl AVX512) and AMD GPUs
r/LocalLLaMA
•
Generative AI
Open Source AI
PrismAI's fork of llama.cpp is broken if you try to run on CPU. This also includes instructions for running on AMD GPUs via ROCm. submitted by /u/UncleOxidant [link] [comments]