AI RESEARCH
Fast NF4 Dequantization Kernels for Large Language Model Inference
arXiv CS.LG
•
ArXi:2604.02556v1 Announce Type: new Large language models (LLMs) have grown beyond the memory capacity of single GPU devices, necessitating quantization techniques for practical deployment. While NF4 (4-bit NormalFloat) quantization enables 4$\times$ memory reduction, inference on current NVIDIA GPUs (e.g., Ampere A100) requires expensive dequantization back to FP16 format, creating a critical performance bottleneck.