My experience with the Intel Arc Pro B70 for local LLMs: Fast, but a complete mess (for now)
r/LocalLLaMA
•
Generative AI
Open Source AI
AI Tools
Full disclaimer using ai to help clean up my mess of thoughts. i have a tendency of not being coherent once i get many words out. TL;DR: Bought a B70 on launch day. Achieved an impressive 235 t/s with Gemma 3 27B on vLLM(100 requests), but the software stack is a nightmare. MoE is barely ed, quantifying new architectures is incredibly fragile, and you will fight the environment every step of the way. Definitely not for the faint of heart. Hey everyone, I ordered the Intel Arc Pro B70 on the 27th right when it released.