We all had p2p wrong with vllm so I rtfm

r/LocalLLaMA
AI Hardware AI Tools

So either way you have pro gpu (non geforce) or p2p enabled driver, but no nvlink bridge and you try vllm and it hangs. In fact vllm relies on NCCL under the hood will try to p2p assuming it has nvlink. But if your gpu can p2p over pcie but still nvlink fails. Thats why everywhere you see NCCL_P2P_DISABLE=0 So how can you use p2p over pcie? By telling nccl which level of p2p is ok. By adding VLLM_SKIP_P2P_CHECK=1 NCCL_P2P_LEVEL=SYS (of