KramaBench: A Benchmark for AI Systems on Data-to-Insight Pipelines over Data Lakes

ArXi:2506.06541v3 Announce Type: replace-cross Discovering insights from a real-world data lake potentially containing unclean, semi-structured, and unstructured data requires a variety of data processing tasks, ranging from extraction and cleaning to integration, analysis, and modeling. This process often also demands domain knowledge and project-specific insight. While AI models have shown remarkable results in reasoning and code generation, their abilities to design and execute complex pipelines that solve these data-lake-to-insight challenges remain unclear. We.