
Safety of AI
Research theme 01 · Led by Dr Koorosh Aslansefat
In short
Safety of AI asks whether a machine learning component can still be trusted once it’s deployed, and how to tell automatically when it can’t. A model is validated on the data it was given. Then the world changes.
Why it matters
Classical safety engineering assumes a system you can specify. You state what it should do, analyse how it could fail, and argue that the remaining risk is acceptable. A learned component breaks that assumption: its behaviour comes from data, not from a specification an engineer wrote, so there’s no design model to analyse in the usual way.
In practice, accuracy measured at validation time is a statement about the past. When the input distribution drifts — new weather, a new sensor, a population the training set under-represented — the model can keep giving confident outputs that are wrong. For a system whose failure can hurt someone, that silence is the hazard.
This theme builds the monitoring and assurance that closes the gap: evidence generated while the system runs, not just at design time.
Theme lead: Dr Koorosh Aslansefat
Assistant Professor · Co-Lead, DEIS
Works on dependability, AI safety and explainable AI.
Also contributing: Mohadeseh Mollapour · Razieh Arshadizadeh · Dr Debarati Chakraborty
SafeML
Statistical distance measures that compare live inputs with the distribution a model was trained on, so a classifier can flag that it’s outside its validated envelope instead of answering anyway.
Out-of-distribution and dataset shift
Detecting when operating conditions have moved away from the training data, telling gradual drift from sudden change, and deciding what a safe response looks like in each case.
Interpretability
Explanation methods that help a safety engineer rather than a data scientist: tracing a failure to a cause that can be designed against, not just visualised.
SafeLLM
Extending assurance to large language models and the systems built on them, where outputs are open-ended and a “correct answer” is much harder to define than in classification.
AI assurance
Turning the above into an argument a regulator or safety case author can use: what evidence is needed, how it’s produced, how it’s kept current once the system is in service, and what it doesn’t cover.

Open questions
This is live research. These are the parts we consider unresolved:
• How much distribution shift is tolerable before a safety argument stops holding, and who decides.
• Whether runtime monitors can themselves be assured to the standard the system they protect requires.
• What an adequate safety case looks like for a generative model with an open-ended output.
Assuring an ML component? We work with industry partners on runtime monitoring and assurance evidence for deployed models.
Related themes:
03 · Security and Safety-Security
05 · Robots & Autonomous Systems
06 · Safety and Assurance Tools