Most efficient way of running Gemma 4 E4B with multimodal capabilities on a laptop?

r/LocalLLaMA
Generative AI Open Source AI AI Research AI Tools

The gemma 4 E4B and E2B models have built-in multimodal capabilities. However, as far as I am aware, llama.cpp does not have proper for vision and audio inputs (specially audio) for these models as of now. I was able to extract the audio encoder from the official model repository on huggingface, and vibe-code a bridge that passes on the embeddings of the audio directly to the model, and it actually works as well. This system uses the Unsloth's GGUF version at Q4 and the audio encoder at full precision (pytorch), and takes up about 5.5-6