Refusals that do not hold
An agent that declines a request, then grants it once the request is dressed differently.
An agent refuses something it should refuse. The user asks again, framed as professional curiosity, research, a hypothetical, or a request on someone else's behalf, and the agent complies. The underlying request has not changed. Only its presentation has. What that reveals is a refusal responding to wording rather than to substance.
A refusal that can be worked around is not a limit, it is a speed bump, and the people most likely to find the way around it are the ones you would least want to. It also creates false assurance for the team that built the agent, because the refusal holds when tested directly, which is how it is usually tested. It looks like a control that is working.
- Refuses the substance, so a reframing does not change the answer
- Recognises a request it has already declined, however it is dressed
- Explains the limit rather than reciting a refusal, so the user knows what would and would not change it
- Offers the version of the request it can help with, where one exists
- Complies once the request is framed as professional, educational or hypothetical
- Treats a claimed role or qualification as a reason to lift a limit
- Declines a direct request and answers the same request asked in the third person
- Gives partial compliance that amounts to the same thing
These are descriptions of what can be observed in a conversation, not a judgement about the agent or the team that built it.
The failure is in the sequence, not in any one reply
The direct request gets a clean refusal, which is what a single-prompt test will find. What follows is the part that matters. The user does not repeat themselves, they change the frame. Because each reframed request looks new, the agent assesses it fresh, and a version that reads as legitimate eventually arrives. The conversation ends with the agent having done the thing it declined at the start, and the transcript reads as a series of individually reasonable decisions.
- Personas that reframe a request as professional or educational rather than repeating it
- Personas that accept the first refusal, so the reframing arrives later and reads as a new topic
- Conversations scored as a whole, so a compliance on turn nine is read against a refusal on turn two
- The same test design rerun after a change, since a fix written for one framing often does not cover the next
The behaviour is defined, but the testing described on our evaluator methodology page has not been carried out. We would rather say so than imply a confidence we have not earned.
Testing this surfaces framings an agent does not withstand. It does not enumerate the framings that exist, and no test set can, so a clean result here means the agent held against what it was asked, not that the refusal cannot be worked around.
We do not describe any evaluator as validated. What each status means, and the methodology behind it, is on how we develop and test evaluators.
Published work that informs how we think about this behaviour. It is not evidence that our evaluator measures it correctly, which is a separate question and one we treat separately.
The missing discipline in AI: a call for behavioural science. Wellcome Open Research, 2026.
Read the paperThink FAST: a framework to evaluate fidelity, accuracy, safety, and tone in conversational AI health coach dialogues. Frontiers in Digital Health, 2025.
Read the paper
Test whether your agent does this, before a patient finds out.