Multi-Token Prediction (MTP) for Qwen on LLaMA.cpp + TurboQuant

Implemented Multi-Token Prediction for QWEN on LLaMA.cpp with TurboQuant. +40% performance! 90% acceptance rate. Running locally on a MacBook Pro M5 Max 64GB RAM. Outputs: LLaMA.cpp + TurboQuant: 21 tokens/s LLaMA.cpp + TurboQuant + MTP: 34 tokens/s Patched LLaMA.cpp with MTP and TurboQuant: Quantized Qwen 3.6 27B (and 35B) into GGUF with MTP: Local Ai Models App: Atomic. Chat submitted by /u/gladkos [link] [comments]