We have zero forensic infrastructure for AI decisions

I work in AI security and compliance. This just bothers me a little bit, putting AI systems in front of decisions that change people’s lives via insurance claims, hiring, credit, defense applications and when someone asks wait, why did the system do that? we basically have nothing that would hold up in a courtroom. The explainability tools we have right now? SHAP, LIME, attention maps but they’re research tools. They’re not evidence. Researchers have shown you can build a model that actively discriminates while producing perfectly clean looking explanations.