Hardware Choice for 27b to 31b models.
r/LocalLLaMA
•
AI Hardware
I've come to a point where I find the 27b and 31b models quite impressive. I have a 16 GB AMD Radeon 7800xt. It performs quite well. It was $700. Here is my question: Is the dual GPU approach performance hit worth it if I save around $400 over a single larger card? Is 32gb even a meaningful step up and is running 9700xt pro with a second 7800xt for total of 48gb a realistic requirement for these size models? I would like to have vram for running these models and I could go with dual 16 GB cards or a single larger card, but here's the cost difference: A) Sell 7800xt for $550.