If Memory Could Compute, Would We Still Need GPUs?

If Memory Could Compute, Would We Still Need GPUs? The bottleneck for LLM inference isn't GPU compute. A February 2026 ArXi paper (arXi:2601.05047) states it plainly: the primary challenges for LLM inference are memory and interconnect, not computation. GPU arithmetic units spend than half their time idle, waiting for data to arrive. So flip the paradigm. Compute where the data lives, and data movement disappears. This is the core idea behind Processing-in-Memory (PIM). SK Hynix's AiM is shipping as a commercial product. Samsung announced LPDDR5X-PIM in February 2026.