Interpretability
Last Updated: July 29, 2026 | By Mihail Sebastian | AI Dictionary
The degree to which a model's inner workings are understandable by design, so a person can trace how inputs become outputs without post-hoc explanation tools.
What is Interpretability?
Interpretability is the degree to which a person can understand how a model turns inputs into outputs by inspecting the model itself, because its structure is simple enough to read directly.
Decision trees, linear and logistic regression, and small rule sets are interpretable in this sense: their logic is the model. The property matters most where decisions must be checked and contested, in credit, medicine, and criminal justice.
How Interpretability Works
An interpretable model earns the label through structure. A linear model states each feature’s effect as a weight you can read off: one more year of credit history adds this many points to the score. A decision tree is a chain of if-then splits you can trace by hand.
Keeping models this readable imposes discipline on the builders: few features, no tangled interactions, relationships a person can hold in their head.
Interpretability comes in two scopes. Global interpretability means you understand the model’s entire logic, everything it will do on any input. Local interpretability means you understand one decision: why this applicant, this diagnosis, this flag.
Interpretability vs Explainability
An interpretable model is understandable by design; explainability reconstructs an understanding after the fact, using post-hoc tools such as SHAP or LIME on a model that is not readable itself.
The distinction has teeth in high-stakes settings. Cynthia Rudin’s widely cited 2019 argument holds that when decisions seriously affect people, institutions should build interpretable models rather than explain black boxes, because a post-hoc explanation can be a plausible story that fails to match what the model computed.
| Interpretability | Explainability | |
|---|---|---|
| Where understanding comes from | The model’s own structure, by design | Reconstructed after the fact, from outside the model |
| Works on | Simple models: linear, trees, small rule sets | Any model, including black boxes |
| Typical tools | None needed; the logic is the model | SHAP, LIME, counterfactuals |
| Main risk | Simplicity can cost accuracy on complex tasks | Explanation may not match what the model computed |
Example of Interpretability
A bank screens loan applications with a decision tree. The first split asks whether the debt-to-income ratio exceeds 40 percent; the second, whether credit history is shorter than two years; the third, whether income clears a minimum.
An applicant is declined, and the loan officer traces the exact path: debt-to-income was fine, but the credit history was fourteen months, which routed the application to a leaf labeled decline. Nothing was approximated or reconstructed. The officer read the decision straight out of the model and can tell the applicant precisely what to change.
Related AI terms: Explainability · Transparency · Black-box Model · Decision Tree
Did you like the Interpretability gist?
Learn about 250+ need-to-know artificial intelligence terms in the AI Dictionary.
Mihail Sebastian — Writes about AI governance, regulation, and the technology behind them. Placeholder bio — replace with a real credential line. About