AI RESEARCH
ArcLight: A Lightweight LLM Inference Architecture for Many-Core CPUs
arXiv CS.CL
•
ArXi:2603.07770v1 Announce Type: cross Although existing frameworks for large language model (LLM) inference on CPUs are mature, they fail to fully exploit the computation potential of many-core CPU platforms. Many-core CPUs are widely deployed in web servers and high-end networking devices, and are typically organized into multiple NUMA nodes that group cores and memory. Current frameworks largely overlook the substantial overhead of cross-NUMA memory access, limiting inference scalability and intelligence enabling on such platforms.