Is hosting an SLM cheaper than using APIs with 20$ cost per user?
r/LocalLLaMA
•
Generative AI
Hello, A non technical client discussed the possibility of having an SLM to reduce inference costs per user for the product we're using that has some AI layer to some tasks. I mentioned that using an SLM to host and scale might be costlier and requires consistent maintenance as in updating or