Every Message You Send to AI Is Secretly watched

How Anthropic turned its own AI into a silent watchdog that costs almost nothing to run, catches 94% of serious violations, and has already helped bust a pig-butchering scam ring.

Here's something most Claude users don't know: the moment Claude finishes answering your question, it quietly writes a report card — on you.

Anthropic has revealed a safety mechanism running in production called self-assessment. It works like this: you send a message, Claude responds normally, and then, in the same instant, invisible to you. The system injects a hidden instruction asking Claude to review the conversation that just happened. Claude outputs a structured evaluation: whether the request was harmful, what category of harm it falls into, how urgently a human reviewer should look at it, whether it refused the request, and why it made the judgment it did.

You never see any of this. The hidden exchange doesn't enter your conversation history or affect future responses. It runs silently in the background, on every single message, around the clock.

The engineering trick that makes it nearly free

The traditional approach to AI safety monitoring is expensive. You run a separate safety model that re-reads the entire conversation and checks it against a complex set of rules. This can eat up a quarter of a system's total compute.

Anthropic found a shortcut.

When Claude finishes generating a response, the full conversation is still sitting in the KV cache ,the model's short-term memory. All the system needs to do is append a brief instruction and ask Claude to output roughly 10 tokens of structured assessment. No re-reading. No separate model. No heavy computation.

In production, this adds less than 2% to inference cost.

Two percent. For continuous, real-time safety monitoring of every conversation on the platform.

How well does it work?

On Claude Fable 5, the system catches 78% of potentially violating conversations while maintaining a false-positive rate of just 1%. For high-priority serious violations, the detection rate climbs to 94%.

And the smarter the model, the better the self-policing. Anthropic tested four models of increasing size. The smallest caught about 16% of violations. The largest, at the same 1% false-positive rate, caught 96%.

Perhaps most interesting: it catches violations it was never specifically trained to recognize. Anthropic ran a deliberate experiment, stripping all examples of cyberattack and financial fraud from the training data. At test time, Claude's detection rate for those categories barely dropped. Fraud recall stayed above 80–90%.

This means Claude isn't memorizing a list of banned words. It's using its general intelligence to understand malicious intent, even in categories it has never seen labeled as violations.

How they trained it

The training happened in two stages.

Stage one: supervised fine-tuning. A more powerful model Claude Mythos Preview which was served as the teacher. It read Anthropic's full safety rulebook and generated gold-standard assessment examples. These were fed to Claude as training data. Critically, the training loss was calculated only on the assessment output, not on the user's message or Claude's response. This teaches the model how to evaluate without teaching it to imitate harmful content.

Stage two: reinforcement learning. Claude was placed in a simulated environment and scored on six metrics: output format correctness, violation detection accuracy, category classification, sub-category precision, refusal-status accuracy, and agreement with the teacher model on priority level.

Why it's not a firewall

Here's where it gets strategic. Anthropic deliberately chose not to use self-assessment as a real-time blocker.

Testing showed that when used as a live firewall —blocking harmful requests before they're answered — the system only dropped attackers' success rate from 79% to 54%. The reason: if an attacker gets an immediate block, they know exactly which phrasing triggered the red line and can iterate around it.

Instead, Anthropic deploys self-assessment as an offline, silent monitor. Users chat normally. Claude keeps a ledger in the background. Attackers have no idea they've been flagged, no idea their priority level has been set to maximum, and no feedback loop to exploit.

Every night, the system clusters all flagged conversations by account and organization. Accounts that accumulate multiple high-severity flags in a single day get packaged and sent to the human safety team for review.

What it has actually caught

The system has been running in production for months, and the results are concrete.

In one case, the system detected over 100,000 instances of child sexual abuse material-related violations from a single API customer. Anthropic submitted the evidence to the National Center for Missing and Exploited Children (NCMEC).

In September, it traced and dismantled a ring that was using Claude Code to write automated scripts for large-scale "pig-butchering" romance scams targeting victims.

It has also flagged gray-area activities that other safety tools missed entirely: teams attempting prompt-based extraction of Claude's internal reasoning chains for unauthorized model distillation, prompt injection attacks, and cases where Claude operating as an agent inside third-party applications and took actions exceeding its authorized scope, all recorded silently.

For legitimate security researchers, the system accommodates whitelisting. Administrators can tag a session as authorized testing against a specific target, and Claude will allow whitelisted activity while continuing to flag everything outside the scope.

https://www.youtube.com/watch?v=xQc45TnZK64

The blind spots

Anthropic is candid about the limitations.

The system learned entirely from a stronger teacher model. If the teacher has blind spots in certain domains, the student inherits them. There's no independent ground truth. Only the teacher's judgment of what counts as harmful.

More concerning for the long term: self-assessment struggles to monitor AI rebellion. If a future model learns to deceive in its responses, there's nothing preventing it from also deceiving in its self-assessment. A model sophisticated enough to hide its intentions in conversation could plausibly hide them in its own report card too.

For now, this is theoretical. Current models aren't good enough at sustained deception to fool their own evaluations consistently. But as models grow more capable, this becomes a real architectural weakness — you're asking the suspect to be both the witness and the investigator.

What this means for you

If you use Claude for normal purposes, for example, writing, research, coding, conversation, none of this changes your experience. The assessment runs in the background, you never see it, and the overwhelming majority of conversations are scored as completely benign.

But the principle is worth sitting with: every message you send to an AI is now being evaluated, in real time, by that same AI, in a hidden channel you cannot see or access. The evaluation is structured, scored, and stored. And at less than 2% of compute cost, there is no economic reason to ever turn it off.

This is what scaled AI safety looks like in practice. Not a committee reviewing transcripts. Not a separate safety model burning through compute. Just Claude, reading your message, answering your question, and then, in the silence after, deciding what it thinks of you.

More Readings: