Intel B70: LLama.ccp SYCL vs LLama.cpp OpenVino vs LLM-Scaler

r/LocalLLaMA
Generative AI Open Source AI AI Tools

In case anyone is interested, I decided to test out LLama.cpp's new OpenVino backend to see how it compares on Intel GPUs. At first glance, it stomps all over the previous best-case, SYCL, but lags behind LLM-Scaler (Intel's VLLM fork), likely just due to the hardware optimizations against GPTQ/Int4. Interestingly tg512 was fastest on SYCL, but in real world, the prompt processing always seems the be the indicator on this card. As usual with Intel, model selection is. poor.