XAI (Explainable AI)
Last Updated: July 29, 2026 | By Mihail Sebastian | AI Dictionary
The research field that builds techniques like SHAP, LIME, and saliency maps to explain the outputs of AI models whose internals are opaque.
What is XAI (Explainable AI)?
XAI (explainable AI) is the research field that builds techniques for explaining the outputs of AI models whose inner workings are opaque. The property those techniques aim to produce is explainability; XAI is the field and the toolbox.
The field grew as black-box models moved into lending, hiring, and medicine, and DARPA’s Explainable AI research program gave the acronym wide currency. Where the law or a clinician demands a reason for a decision, XAI methods are what supply it.
How XAI Works
XAI methods treat a trained model as a fixed object and interrogate it from outside.
- Feature attribution scores how much each input pushed a prediction. SHAP distributes credit using Shapley values from game theory; LIME fits a simple surrogate model around one prediction to see which features drive it locally.
- Counterfactual explanations find the smallest input change that flips the decision: “had your income been €4,000 higher, the loan would have been approved.”
- Saliency maps apply the same idea to vision: they mark the regions of an image the model weighted most.
Every method carries the same caveat: the explanation approximates the model rather than reading it out. A convincing explanation can still misstate what the model computed.
XAI vs Explainability
Explainability is a property a system has or lacks; XAI is the field that builds the methods to achieve it. You apply XAI techniques to a model, and the result, if it works, is an explainable system.
In practice the terms blur, and vendors use them interchangeably. The dictionary keeps them apart because the questions differ: “is this decision explainable?” is about the system, while “which XAI method did you use, and is it faithful?” is about the tooling.
Example of XAI
Dermatology research turned up a now-classic case. A skin-lesion classifier posted strong accuracy, but saliency maps showed it attending to surgical ink markings near lesions, not only to the lesions themselves.
The shortcut had a logic: clinicians tend to mark lesions they already suspect are malignant, so ink correlated with cancer in the training images. The model had partly learned to read the doctor’s suspicion instead of the skin.
No accuracy metric would have caught this; the model scored well for the wrong reason. An XAI technique exposed the shortcut before it could fail on unmarked patients.
Related AI terms: Explainability · Interpretability · Black Box Model · Transparency
Did you like the XAI (Explainable AI) gist?
Learn about 250+ need-to-know artificial intelligence terms in the AI Dictionary.
Mihail Sebastian — Writes about AI governance, regulation, and the technology behind them. Placeholder bio — replace with a real credential line. About