Open source LLM compiler for models on Huggingface. 152 tok/s. 11.3W. 5.3B CPU instructions. mlx-lm: 113 tok/s. 14.1W. 31.4B CPU instructions on macbook M1 Pro.
r/LocalLLaMA
•
Generative AI
Open Source AI
AI Tools
Submitted by /u/pacifio [link] [comments]