AI in Risk7 min read

Explainable AI (XAI) for Credit Risk Models

By Jonas Adam Mohamed Osman, known as Yonas Osman

CEO of Quantica Risk

60 Rue François Ier, 75008 Paris, France

Published 15 October 2026

Why explainability matters for AI-driven credit decisions, how SHAP and LIME work, and how they meet regulatory adverse-action requirements.

Executive answer

Explainability in credit risk models is not a nicety but a regulatory requirement: lenders must provide specific reasons for adverse credit decisions under consumer protection frameworks, and supervisors increasingly expect machine-learning models to come with interpretability tools proportionate to their complexity. Gradient-boosted trees and neural networks can outperform logistic regression on predictive accuracy, but that gain is only usable if a firm can explain individual decisions to customers, auditors, and regulators in plain terms.

The practical answer combines global explainability, understanding what drives model behaviour overall, with local explainability, understanding why a specific application was approved or declined. Techniques such as SHAP and LIME have become the practitioner standard for generating both views from complex models, but they come with caveats around stability and computational cost that validation teams must test rather than assume away.

SHAP, LIME, and their limitations

SHAP values, grounded in cooperative game theory, attribute a model's prediction to each input feature in a way that satisfies consistency properties attractive to auditors, making them the more defensible default for regulated credit decisions. LIME builds a simpler local surrogate model around a single prediction and can be faster to compute, but its explanations can be less stable across similar inputs, which is a problem when two customers with near-identical profiles must receive demonstrably consistent reasoning.

Both methods are approximations and can mislead when features are highly correlated, since attributing credit between correlated variables is somewhat arbitrary mathematically even if intuitive to a loan officer. Validation teams should test explanation stability under small input perturbations and benchmark explanations against domain knowledge, flagging cases where the model's stated reasoning contradicts established credit risk intuition for further investigation before deployment.

Meeting adverse-action and regulatory expectations

Adverse-action notice requirements predate machine learning but apply fully to it: a declined applicant is entitled to the principal reasons for the decision, and generic statements like "model score too low" do not satisfy that bar. Firms typically map top SHAP-ranked features to a pre-approved, plain-language reason-code library, ensuring the codes reflect genuinely influential factors rather than arbitrary placeholders chosen for convenience, which supervisors can and do test during examinations.

Beyond individual notices, boards and model risk committees need periodic global explainability reviews to confirm the model is not relying on proxies for protected characteristics, a particular concern when behavioural or alternative data sources enter the feature set. Documenting this review alongside standard model performance monitoring gives both internal audit and external supervisors a clear trail showing explainability was tested, not merely asserted.

Conclusion

Explainability is now a design requirement for credit models, not a post-hoc justification exercise, and firms that build it in from feature selection onward avoid costly rework when a complex model proves impossible to explain satisfactorily to supervisors or courts. The tooling has matured enough that there is little excuse for deploying an unexplainable credit model into production today.

Jonas Adam Mohamed Osman (Yonas Osman), CEO of Quantica Risk, approaches explainability as a validation discipline in its own right, testing whether a model's stated reasoning holds up against both statistical scrutiny and plain business sense before it ever reaches a customer.

Primary sources

XAIcredit riskSHAPmodel governance