Any good MOE ~60B models? I have 64GB vram
r/LocalLLaMA
•
Open Source AI
I have a build with 2 x MI50 32GBs and 64 gigs of DDR4 (bought before rampocolypse for ~630 USD total, I’m not rich) and I’m not gonna upgrade it for a long while. Are there any good MOE models that are around 60B in parameters so I can make use of all the VRAM? I feel like I’m stuck in a weird spot where using small models fees like a waste but I can’t really use larger models. I’ve been liking Gemma 4 31B at q4 quantisation but it’s a bit slow at both prompt processing and tps. I use it almost just for creative writing. Any suggestions? Thanks submitted by /u/opoot_ [link] [comments.