Testing llama.cpp MTP support on Qwen3.6 - RTX 5090
r/LocalLLaMA
•
Generative AI
AI Hardware
Open Source AI
Setup: - RTX 5090, 32 GB, Linux - Built llama.cpp from 4f13cb7 (the official ghcr.io/ggml-org/llama.cpp:server-cuda image hasn't picked up the merge yet as of writing - had to docker build from source with CUDA_DOCKER_ARCH=120) - Unsloth's Qwen3.6-27B-MTP-GGUF Q5_K_M and Qwen3.6-35B-A3B-MTP-GGUF UD-Q4_K_M - 128k context, flash-attn, q8_0 KV cache, temp 0.8, --parallel 1 (required for MTP) - Same GGUF for "MTP on" and "MTP off" - only the --spec-type draft-mtp --spec-draft-n-max 3 flag toggled. This isolates MTP from quant differences.