Xoul - Building a Local AI Agent Platform with Small LLMs: The Walls of Tool Calling and Practical Solutions

This post is a real-world account of developing Xoul, an on-premise Local AI agent platform, where we hit the walls of small LLM Tool Calling limitations and overcame them one by one at the application layer. Background: "Let's Build a Local Agent" With large models like GPT or Claude, Tool Calling is near-perfect. But the moment you need to run small local LLMs (Ollama + Llama3/Qwen/Oss under 20B) for on-premise environments or cost reasons, reality hits hard.