AI Reproducibility Crisis: Why Claims Fail to Verify

Dev.to AI
AI Research

A paper reports a new state-of-the-art result. The repo is public. The figures look clean. The conference is top-tier. In the AI reproducibility crisis, that still does not mean a non-author can verify the claim. That is the real shift. The problem is not just missing code. It is that the decisive details often live outside the polished artifact: preprocessing scripts, random seeds, undocumented defaults, evaluation quirks, dataset filtering, or a half-finished repo that reproduces the table except for the number the paper is selling. A claim can be persuasive without being checkable.