LAI #125: Karpathy’s Agent Ran 700 Experiments Without Him

The Context Rut, plus vectorless RAG, why attention is kernel evaluation, and the end of XGBoost’s decade Good morning, AI enthusiasts! An AI agent just ran 700 experiments on its own, found patterns, and optimized its own performance, no human in the loop. This week, I break down Karpathy’s Auto Research project and the technical bottleneck it exposes: the Context Rut. We also cover why putting business logic inside a prompt is one of the most common mistakes in production LLM systems and what to do instead.