Anthropic’s safety filter was off for eleven months. So was the alarm.

Anthropic published its August risk report this week and volunteered something most companies would have kept quiet about.

From May 2025 until April 2026, all the traffic through the systems it uses to collect human feedback from contractors ran without its blocking biological-safety classifiers. Around 50,000 people, vetted only by outside vendors. Around 133 million exchanges.

That is a bad eleven months. But it is not the part worth your time.

One switch turned off the alarm as well

Here is the sentence I would print out and stick above a desk. The traffic, Anthropic writes, was controlled by

“a flag meant only for internal use, which disabled not only the blocking behavior of the classifiers but also the logging of their flags; as such, traffic that would otherwise have been flagged was not recorded or propagated to any review mechanisms.”

One setting removed the protection and the evidence that the protection was missing. There was nothing to notice. No spike, no empty dashboard, no angry log file. The system looked exactly the way it looks when everything is fine, because “fine” and “not watching” produce identical silence.

Anthropic’s own conclusion is the honest one, and the reason I am writing about it at all:

“The discovery of this gap, however, leads us to believe that there is an increased likelihood of other, similar issues unknown to us.”

While we are here: what the 1,197 number actually is

You may see “1,197 high-risk conversations” going round. It is worth a moment, because the number does not mean what it looks like.

A review model was run over every human turn from the affected period and flagged 1,197 transcripts as high for biological harm. Of those, 757 came from Anthropic’s own internal teams using the same infrastructure, and all but 62 of the remainder came from exercises where red-teamers had been asked to attack the system deliberately. Anthropic manually reviewed all 62 of the non-red-teaming ones, plus a random 30 of the red-teaming ones, and found no clearly concerning misuse — though it does note a handful of potentially dual-use conversations.

So: 1,197 flags, 62 of which were unexplained by someone doing their job. That is a much smaller and much more useful number, and you only get it by reading past the headline. Same lesson as every other percentage that arrives without its denominator.

The same failure, in a business your size

You are not running bioweapon classifiers. You are running things with exactly the same shape:

The backup. It ran nightly for two years. Then a disk filled, or a password changed, and it has been failing silently since March. You will find out on the day you need it.

The enquiry form. It still says “thanks, we’ll be in touch”. The email behind it stopped arriving when someone left and their mailbox was closed. Every enquiry since has gone into the ground, and the silence looks identical to a quiet month.

The spam filter turned off during troubleshooting one afternoon, and never turned back on.

The alert that goes to a phone number nobody has any more.

I can be specific about this rather than lofty, because we did it ourselves a fortnight ago. Our Instagram posting has an access token that lasts sixty days and can be renewed at any point while it is valid. Nothing in our system was renewing it. It expired quietly, and the first indication was the job failing — which is to say, the thing we had built to run every day was also the thing telling us whether it worked. No independent check at all. Precisely the same mistake, several orders of magnitude smaller, and I have no excuse of scale.

How to actually check — four things, one afternoon

1. Trigger it, don’t look at it. A green tick on a dashboard is a claim, not a test. Restore one real file from your backup and open it. Fill in your own contact form from your phone and see whether an email lands. It takes ten minutes and it is the only evidence that counts.

2. Check the log has today in it. Not that a log exists — that it contains recent entries. This is the exact failure above: the absence of alarming records was itself the problem.

3. Ask when the alert last fired. If the answer is “it never has”, you have learned nothing yet. Either nothing has gone wrong, or nothing is watching, and those feel the same from the inside. Cause one deliberately and see if the phone buzzes.

4. Diarise it. Once a quarter, half an hour, the same short list. The value is entirely in the repetition — a check you did once in 2024 is a memory, not a control.

If a machine is doing work for you unattended, the same rule applies with knobs on: it needs a check that is separate from the thing being checked. We wrote about where that line sits in when an AI agent should call a human.

What argues the other way

Anthropic found this itself, published it in detail, quantified it, and said plainly that the discovery makes it less confident there are no other gaps. That is considerably more than most organisations manage, and it would be perverse to treat candour as though it were the scandal. Its stated conclusions are that the gap has been remediated, that the review found no evidence of misuse, and — its words — that there was no impact on customers.

There is also a real limit to the lesson. You can check everything constantly and end up with a wall of alerts nobody reads, which fails in the same way for the opposite reason. Four things you genuinely test beats forty you have merely enabled.

And I would rather this were read as one operator learning from another than as a small company wagging its finger at a large one. We have a token that expired because nobody was watching it. They had classifiers that stopped blocking because a flag was set for something else. The difference is scale, not species.

There is now an official version of this argument. On 20 August the NCSC published interim guidance on running AI agents, and two of its asks are exactly the ones above: keep logs somewhere the agent cannot write to, and be able to stop the thing immediately. I marked our own homework against its checklist and failed most of it.

The short version

A protection you switched on is not the same as a protection that is running. Silence is not evidence of safety when the thing that would have made a noise is the thing that broke. Pick your four, test them properly, and put it in the diary.

Source: Anthropic, Risk Report, August 2026 (redacted public version), read directly. The dates, the ~50,000 contractors and ~133M exchanges, the quoted sentence about the internal-use flag, the 1,197 / 757 / 62 breakdown, the manual-review finding, the remediation, the “no impact on our customers” note and the quoted conclusion about other similar issues are all from that document. Anthropic’s overall chemical-and-biological risk assessment in the report is described as “Low, but higher than our previous estimate due to the gap in our access controls”.