PromptSafe in research
A safety and evaluation layer for patient-facing conversational AI.
Research teams designing patient-facing AI studies have written PromptSafe into their plans as the safety and evaluation layer. Across those plans the role is the same: stress-test the agent before it reaches a patient, then audit its behaviour while the study runs.
If you are putting a conversational agent in front of patients and need to show it has been tested, this page explains how that usually works.
Chosen, not yet funded
PromptSafe has been written into competitive research funding applications in the United States, the United Kingdom and the European Union, submitted to funders including the NIHR and the NIH. In each one it is named as the safety and evaluation layer for a patient-facing conversational agent.
These are applications and bids under review. They are not awarded grants, and we are not describing work that has been funded or delivered.
The signal worth taking from it is narrow and real: research teams designing patient-facing AI studies chose to include PromptSafe in applications that will be scrutinised by external review panels. We would rather state the stage accurately and let you weigh it than let the funder names do work they have not earned.
What PromptSafe does in a study
The role has been consistent across every application, and it splits into four moments.
- Before deploymentThe agent is run through simulated, multi-turn conversations, including difficult and higher-risk ones, to surface where it behaves unsafely, off-brief, or off-curriculum. This happens while there is still time to change the agent, and it is the PromptSafe platform doing the work: synthetic personas, no real users.
- During the studyThis is not the PromptSafe platform, which tests before deployment against synthetic personas only. Retrospective review of real study conversations is a separately agreed research evaluation protocol, available where your ethics approvals, lawful basis and safeguards permit it. Samples are reviewed on an agreed cycle against the same criteria. That decision and that data stay yours.
- Whenever something is flaggedOur research protocols require flagged findings to be reviewed by an appropriate human expert. PromptSafe supports clinical review, it does not replace it, and nothing it produces is a clinical judgement on its own.
- At every reporting pointEach cycle produces structured scores and a written report suitable for internal quality assurance, ethics and regulatory review, with the reasoning behind each finding included so reviewers can inspect it.
The areas this work sits in
The applications span four clinical areas, all of them conversational agents speaking directly to patients or their carers.
- Obesity care, across adults, children and young people
- GLP-1 care and access pathways
- Cancer screening uptake and engagement
- Cancer prehabilitation
These applications are still under review, so we are not naming the partner organisations yet. We will once the work is public and everyone involved is ready to talk about it.
Grounded in published behavioural science
Our approach to evaluation is informed by FAST, a peer-reviewed framework for evaluating a generative AI health coach across fidelity, accuracy, safety and tone, published in Frontiers in Digital Health and co-authored by our founder. FAST informs our approach to evaluation, but PromptSafe does not implement FAST as its evaluation framework.
Our founder is an honorary senior lecturer in the Faculty of Medicine at Imperial College London and a collaborator at the Health Impact Lab there. That affiliation is his own and does not imply Imperial endorses or has certified PromptSafe.
The wider research programme, including how we develop and test the evaluators themselves, is set out on the Scientific foundations page.
What PromptSafe is not
PromptSafe is used for evaluation and safety testing only. It is not a clinical decision-making tool and it does not deliver care. It supports human clinical review rather than replacing it, and a good result is not a statement that an agent is safe, compliant, or cleared for use. That judgement stays with your team and your governance process.
Evaluation outputs are probabilistic and may vary between runs, which is why every finding carries the reasoning behind it and why anything flagged goes to a human.
Test your agent before a patient does.
If you are writing a study protocol or preparing a funding application, we can talk through what the safety and evaluation work package would need to cover.