Why run local? Count the money
r/LocalLLaMA
•
Open Source AI
I’m not a coder, but I run local models. I gave in to agent hype (I was building my own, but there is so much to do) and installed Hermes. Running with Qwen-397b out of a 2 spark cluster. So…I asked Hermes today to tally the token count, and the result…200M tokens. In 5 days. At this rate, using an agent for tasks like installing software and debugging things I want to try out, what is the cost I am saving? Artificial Analysis says the price is about 1.25 dollars per million tokens on average from providers.