Small model (8B parameters or lower)

r/LocalLLaMA
Open Source AI

Folks, Those who are using these small models, what exactly are you using it for and how have they been performing so far? I have experimented a bit with phi3.5, llama3.2 and moondream for analyzing 1-2 pagers documents or images and the performance seems - not bad. However, I dont know how good they are at handling context windows or complexities within a small document over a period of time or if they are consistent. Can someone who is using these small models talk about their experience in details? I am limited by hardware atm and am saving up to buy a better machine.