Building a local automation agent for iPhones: Need help

r/LocalLLaMA
Generative AI Open Source AI

Hey LocalLLaMA My co-founder and I are building PocketBot, basically an on-device AI agent for ithat turns plain English into automations. It runs a quantized 3B model via llama.cpp on Metal, fully local with no cloud. The core system works, but we’re hitting a few walls and would love to tap into the community’s experience: Model recommendations for tool calling at ~3B scale We’re currently using Qwen3, and overall it’s decent. However, structured output (JSON tool calls) is where it struggles the most.