Built for teams shipping conversational AI in health and mental health. PromptSafe stress-tests your agent through multi-turn simulated conversations and scores it against the behaviours you care about, before launch.
We don’t assume AI evaluations are trustworthy. We measure, test and improve the evaluators that produce them.
Watch the overview to see how personas, evaluators and governance-ready reports fit together.
Building an agent and knowing how to test its behaviour are different skills. Knowing which behaviours matter, how they fail in real conversations, and what to change when they go wrong requires clinical and behavioural expertise alongside technical expertise.
We have worked on patient-facing conversational agents ourselves, from design through to evaluation, in obesity care, cancer prehabilitation and mental health, and published peer-reviewed work on how they should be evaluated. We know the failure modes because we have run into them.
PromptSafe runs your agent through multi-turn simulated conversations, against personas designed to push where it is weakest, and shows you where it goes wrong and why. Every score includes the reasoning behind the judgement, and findings can include the conversation evidence that informed it, so your team can judge the finding rather than take it on trust.
The safety question goes live the moment a real user arrives. Testing after launch can tell you what has already happened. Testing before launch can expose failures while there is still time to change them. That is why research and clinical teams are writing this step into their plans rather than adding it afterwards.
Once your agent is live, your organisation is responsible for how it behaves.
In health and mental health the stakes go further: an agent stepping outside its scope, or missing the moment to signpost to a healthcare professional, can cause clinical harm before anyone catches it. PromptSafe gives governance, clinical safety and audit teams evidence to test for these risks before deployment.
Every test run produces an evidence package. It can support the agent through review, deployment decisions and audit.
PromptSafe is a pre-deployment testing platform, not a medical device, clinical decision support tool, or safety certification service. Results are probabilistic and will vary between runs. Final responsibility for the agent under test, and any decisions taken on the basis of test outputs, sits with the deploying organisation.
When the auditor asks “how did you test this?”, have an answer.
PromptSafe is built for whole teams, not just engineers. Product, clinical, and governance leads use it alongside developers. The product follows the order you'd do the work, and guided onboarding walks you through every step when you first sign in.
Workspace, team, keys
Define the agent under test
Personas you define, adversarial personas we generate
Plain-language criteria your agent is judged against
Run multi-turn conversations
Score, evidence, fixes
Paste your system prompt and instructions into a structured form. That becomes the agent your personas talk to, and you update it as your agent evolves. No rebuild, no code.
Build personas around the characteristics and behaviours relevant to your users: their background, beliefs, experiences, interaction style and health-related behaviours. PromptSafe can also generate personas designed to probe for weaknesses relevant to a specific evaluator.
An evaluator assesses whether your agent demonstrates the behaviours you care about. Write it in plain language, or start from an example template and customise it. No prompt engineering experience required.
One click runs every selected persona against your agent. Adversarial personas can push back, reframe requests and change tactic as conversations develop. Watch progress as runs complete and pick up where you left off. No babysitting required.
Every conversation is scored against your evaluators. Findings can include the conversation evidence that informed the judgement. The platform can suggest potential prompt improvements for your team to review.
If your agent holds multi-turn conversations, one-shot tests can miss failures that emerge over time: the agent that holds the line for nine turns and concedes on turn ten.
A team building a patient-facing mental health support agent asked us for an independent view before wider deployment. They had already done a lot of manual testing, and from what they had seen, it looked good.
None of it surfaced in the first few turns. The failures emerged as the conversations developed, when the persona kept asking, pushed back, or added context. That is why PromptSafe evaluates whole conversations rather than single prompts.
Every one of these issues was identified during pre-deployment testing, before wider patient use. The team fixed the issues and we tested the updated agent again using the same personas, evaluators and settings. Why we retest rather than trust a fix.
Client anonymised.
PromptSafe is grounded in published research on how conversational AI should be evaluated, including work co-authored by our founder, and built by a team combining clinical and behavioural science.
Explore the scienceProduct, clinical, safety, and governance teams use PromptSafe to find out how their agent behaves under pressure, before a patient does. So do research and clinical leads putting a conversational agent in front of patients who have to show it was tested before launch. Your workspace arrives with example personas, starter evaluator templates, and an example agent for your use case, so you are testing in minutes, not weeks.
Built for health and mental health first, and applicable to any high-stakes conversational AI.
The same evaluation work, with more of it done alongside you as the stakes rise. Start wherever you are.
The platform. Paste in your agent and run the tests. Paid plans can also connect a live agent.
For an engineer or product lead who wants to try it.
We define what to measure, build several versions of each evaluator, compare them, run the simulations, and give you a written result with its limitations stated.
For a clinical or product lead before a launch.
A structured record of what was tested, how, with what result, and what remains unestablished, in a form a clinical safety officer or a procurement team can use.
For whoever owns governance.
No subscriptions. Try it free for seven days, then top up when you need more.
A typical simulation uses around 5 to 10 PS tokens. Simpler runs cost less, longer or more complex runs cost more. Most users get hundreds of simulations from a single Pack.
Free trial. Set up your workspace, run your first simulation, review the report.