How do I explain "Black Box" model decisions to non-technical stakeholders?

All answers to this question.

This is a huge topic in "Responsible AI" right now. SHAP (SHapley Additive exPlanations) is generally considered the gold standard because it’s based on solid game theory and provides consistent results. You can show stakeholders a "force plot" which literally breaks down a single prediction: for example, "This person was rejected because their debt-to-income ratio added +0.2 to the risk score, while their credit age only subtracted -0.05." It turns the black box into a series of additive factors. LIME is also good but can be a bit "noisy" since it approximates the model locally. For legal compliance, SHAP is usually preferred.

   Answered 2025-01-07 by Deborah Lewis

Have you looked into Global vs Local interpretability? Sometimes stakeholders want to know how the model works overall, not just for one person. Do you need to provide a summary of the most important features across the whole population?

   Answered 2025-01-10 by Thomas Clark

• Thomas, the legal team specifically needs to provide "adverse action" notices, so they need local interpretability for every individual decision. However, the executive team wants the global view to ensure we aren't biased against certain demographics. I think I need a solution that covers both. If I use SHAP summary plots, will that satisfy the global requirement while still allowing me to drill down into the specific "force plots" for those individual legal notices?

     Commented 2025-01-12 by Jennifer Reed

You could also try using a "Surrogate Model." Train a simple Decision Tree to mimic the Random Forest. It won't be as accurate, but it’s very easy for a human to read

   Answered 2025-01-15 by Jason Miller

• Surrogate models are a clever workaround, Jason. I’ve used them in healthcare presentations before. It helps bridge the gap between complex math and intuitive business logic!

     Commented 2025-01-17 by Deborah Lewis