My company just handed me a 2x H200 (282GB VRAM) rig. Help me pick the "Intelligence" ceiling.
r/LocalLLaMA
•
Generative AI
AI Hardware
My workplace just got a server equipped with 2x Nvidia H200 GPUs (141GB HBM3e each). I've been asked to test LLMs on it since they know "I do that at home". While I have experience with smaller local setups, 282GB of VRAM is a different beast entirely. I want to suggest something "interesting" and powerful than just the standard gpt oss or something. Im interested in raw "intelligence" over ultra high speeds. So what models / quants would you suggest for them to put on it? EDIT: They were actually a bit specific about the use case.