AI RESEARCH
Multi-Scale Dequant: Eliminating Dequantization Bottleneck via Activation Decomposition for Efficient LLM Inference
arXiv CS.LG
•
ArXi:2605.13915v1 Announce Type: cross Quantization is essential for efficient large language model (LLM) inference, yet the dequantization step-converting low-bit weights back to high-precision for matrix multiplication has become a critical bottleneck on modern AI accelerators. On architectures with decoupled compute units (e.g., Ascend NPUs), dequantization operations can consume cycles than the matrix multiplication itself, leaving the high-throughput tensor cores underutilized.