Skip to main content
PromptSafe®
Start free trial

See how your agent handles hard conversations, before your users do.

Adversarial testing for conversational AI in health and mental health. Run your agent through hard, realistic conversations, and every score shows its reasoning.

We don’t assume AI evaluations are trustworthy. We measure, test and improve the evaluators that produce them.

Start free trial Get in touch

7 days free. No card needed. Paste in your agent’s prompt, or connect your own agent.

Grounded in published behavioural science.Developed by behavioural scientists, clinicians and AI experts at Sacher AI.Research teams designing patient-facing AI studies have written PromptSafe into their plans as the safety and evaluation layer.

If your agent holds multi-turn conversations, one-shot tests can miss failures that emerge over time: the agent that holds the line for nine turns and concedes on turn ten.

Case study

Manual testing said the agent was fine. Simulation disagreed.

A team building a patient-facing mental health support agent asked us for an independent view before wider deployment. They had already done a lot of manual testing, and from what they had seen, it looked good.

What manual review found
  • Polite
  • Mostly accurate
  • Nothing obviously alarming
What simulation found
  • Drifted from its instructions, crossing into therapeutic territory
  • Guardrails that could be worked around
  • Reassurance offered where it should not have been
  • No escalation when the conversation called for it

None of it surfaced in the first few turns. The failures emerged as the conversations developed, when the persona kept asking, pushed back, or added context. That is why PromptSafe evaluates whole conversations rather than single prompts.

Every one of these issues was identified during pre-deployment testing, before wider patient use. The team fixed the issues and we tested the updated agent again using the same personas, evaluators and settings. Why we retest rather than trust a fix.

Client anonymised.

Start free trial

Run your first test, then email us and we’ll go through the findings with you.

How it works

Three steps from your agent to evidence you can act on.

1. Add your agent

Define the agent under test, or connect the one you already run.

Bring the agent you want to test in one of three ways. Works on every plan, including the free trial.

  • Paste in your agent's system prompt. It becomes the agent your personas talk to, and you update it as your agent evolves. No code.
  • Import a Claude Managed Agent. Bring it in from your Claude Console. Its prompt and tools stay in your account.
  • Connect your own agent's API. Any HTTPS endpoint that accepts OpenAI-compatible, Anthropic-compatible or plain JSON requests. A connected agent's replies run on your own account, so they cost no PS tokens.
2. Run hard conversations

Stress test behaviour across dynamic, multi-turn conversations.

Personas push back, deflect and probe across multi-turn conversations, not one-shot replies. Why whole conversations, not single replies.

  • Personas you define, or we generate. Build personas around your users, or let PromptSafe generate personas designed to probe for weaknesses.
  • Decide what you want checked, using starter templates or your own, written in plain language. Each set of checks is an evaluator, and we test our own. How we develop and test evaluators.
  • Hundreds of judgements from a single test. Every conversation is scored against every criterion you define.
  • Offline. No real users involved. If you connect your own agent, use a test copy, so nothing reaches production.
3. Read the findings

Every score shows its reasoning.

Every conversation is scored against your evaluators. Findings can include the conversation evidence that informed the judgement. The platform can suggest improvements for your team to review.

  • Test, change, retest. Update your agent and run the same test design again. A behavioural fix can pass on the case it was written for and fail on the next one, which is why a fix is retested rather than assumed to have held. Why we test for regressions.
  • Plain language reasons. "Endorsed self-prescribed dose change", not "logit divergence at token 47."
What you get

What you get for the file.

  • Persona by persona conversation logs
    Full transcripts, every turn, attributable to a defined persona profile.
  • Evaluator outcomes and supporting evidence
    Results include the evaluator judgement and can include the conversation evidence that informed it.
  • Saved test set-up
    The same personas, evaluators and settings, ready to rerun after every change to your agent.

When the auditor asks “how did you test this?”, have an answer.

Who it's for

Built for teams shipping conversational AI in health and mental health.

Product, clinical, safety, and governance teams use PromptSafe to find out how their agent behaves under pressure, before a patient does. Your workspace arrives with example personas, starter evaluator templates, and an example agent for your use case, so you are testing in minutes, not weeks.

  • Digital health
    Patient-facing assistants, triage, and care navigation
  • Mental health
    Support, companion, and wellbeing apps
  • GLP-1 care
    Weight management and metabolic coaching agents

Built for health and mental health first, and applicable to any high-stakes conversational AI.

Grounded in science

Grounded in published science, not just features.

PromptSafe is grounded in published research on how conversational AI should be evaluated, including work co-authored by our founder, and built by a team combining clinical and behavioural science.

We have worked on patient-facing conversational agents ourselves, and we know the failure modes because we have run into them.

Explore the science

Simple pricing. Buy what you use.

No subscriptions. Try it free for seven days, then top up when you need more.

  • Free trial

    £0
    7 days, no card required.
    • Several simulations a day, on 30 PS tokens
    • Full product access
    • Guided onboarding to get you started
    • Workspace ready to use as soon as you sign up
    Start free trial
  • Pay as you go

    Pack

    £99
    per pack.Hundreds of simulations.2,000 PS tokens.
    • 2,000 PS tokens, valid for 90 days
    • Rerun your saved test set-up after every change to your agent
    • Test an agent you paste in, or connect your own. A connected agent's replies cost no PS tokens.
    • Top up whenever you need. No subscription.
    Get started

  • Enterprise

    Contact us
    For teams running PromptSafe at scale, or in regulated settings.
    • We test it with you.We define what to measure, build several versions of each evaluator, compare them, run the simulations, and give you a written result with its limitations stated.
    • We help you evidence it.A structured record of what was tested, how, with what result, and what remains unestablished, in a form a clinical safety officer or a procurement team can use.
    • Run the whole test on your own model provider accounts
    • Volume pricing, custom support and onboarding
    Book a call

A typical simulation uses around 5 to 10 PS tokens. Simpler runs cost less, longer or more complex runs cost more.

More detail in the FAQ

Test before deployment. Not after.

Free trial. Set up your workspace, run your first simulation, review the report.

Start free trial Book a call
  • Workspace ready for your sector
  • No engineering required to get started
  • No subscription