top of page

XWhy: Explaining Black Boxes with a SMILE

Model-agnostic explainability for machine learning and generative AI

​

Most machine learning models work as black boxes: they give an answer without showing which parts of the input led to it. For anyone who has to trust, audit or certify such a system, that is a problem.

XWhy is an open-source Python library that explains individual decisions of black-box models. It implements SMILE, Statistical Model-agnostic Interpretability with Local Explanations, a method developed in DEIS that uses statistical distance measures to make local explanations more reliable.

Because it only needs to query the model, XWhy works with classifiers, image models, large language models and other systems whose internals are hidden or too complex to inspect.

XWhy is led by Dr Koorosh Aslansefat. Its development was supported by the SESAME H2020 project.

How SMILE works

SMILE explains one decision at a time.

It takes the input to be explained and creates many variations of it, switching meaningful parts on or off: words in a prompt, regions of an image, features in a table.

It asks the black-box model for its output on each variation and measures how much the output changed, using a statistical distance suited to the kind of data.

It weights each variation by its relevance to the original input, fits a simple surrogate model to the weighted results, and reports how much each part of the input contributed to the decision, together with measures of how good the explanation is.

​​

Since the original paper in IEEE Software, SMILE has grown into a family of methods for different kinds of AI, most of them published as preprints so far.

Point clouds: explaining neural networks that classify 3D point-cloud data.

Image editing: explaining which words in an instruction shaped the output of an instruction-based image editing model.

Large language models: gSMILE explains which parts of a prompt drive a model’s response.

Knowledge-graph RAG: KG-SMILE explains retrieval-augmented generation built on knowledge graphs.

Concept-based models: ConceptSMILE audits whether concept-based explanations can be trusted.

Financial sentiment: local perturbation explanations of how large language models reason about financial sentiment, published at NLDB 2026.

Input → variations → model queries → statistical distances → weighted surrogate → explanation

Try it: explain an LLM prompt
​

XWhy installs with pip install xwhy. Explainers are available for tabular data, text, image classification, image generation and editing, and large-language-model prompts; point-cloud support is under construction.

This tutorial shows, step by step in a Jupyter notebook, how to find which parts of a prompt shaped an LLM’s answer. A second video explains image generation and editing models.

XWhy explanations are local approximations of a model’s behaviour. They do not reveal its internal reasoning or prove cause and effect, so for high-impact decisions they should be combined with validation, robustness testing and human review.

Using XWhy

XWhy is open source under the MIT licence. The code is on GitHub, and the documentation, tutorials and full publication list are on the XWhy website.

If you use it in research, please cite: K. Aslansefat, M. Hashemian, M. Walker, M. N. Akram, I. Sorokos and Y. Papadopoulos, “Explaining Black Boxes with a SMILE: Statistical Model-Agnostic Interpretability with Local Explanations”, IEEE Software 41(1), 2024.

Need explanations you can defend for an AI system? We work with partners on explainability and assurance for machine learning and generative AI.

Talk to us about XWhy →

← All technologies

bottom of page