How I Taught AI to Catch Fakes I Can't See
From side project to newsroom tool in twelve months
Imagewhisperer.org, my pet project to detect fake images, is one year old. In January, I wrote about why AI detection fails on the fakes that matter most. Denis Teyssou broke the tool with a Trudeau image. Ronan Le Nagard broke it with an Eiffel Tower composite.
I tried to fix both problem by adding a series of AI detectives to catch what the math missed, and ended the article with: “It’s still cat and mouse. But now the cat has two sets of eyes.” Turns out, two set of eyes is not enough.
What happened in the two months since that article I’d rather write down than explain at dinner parties. I went from three detection models to ten to eighteen. I moved from a not so cheap computer to a dedicated GPU server — which means I train models via my own hardware.
I learned things about how images lie that I didn’t know existed. And then a single AI-generated photograph walked through the front door and made all the then 17 models wrong. None of the checks saw this man as AI generated:
.Black Forest Labs released Flux in 2024. Only recently I found out it uses a different architecture than the generators my models were trained on. The images it produces don’t carry the statistical fingerprints that my existing detectors look for.
A portrait of a man in a park. I knew it was fake. I had the receipt. I stared at those green numbers for a long time. The tool said: real photograph. It was not a real photograph.
Jeongsoo Park and Andrew Owens at the University of Michigan built a detector trained on images from 4,803 different AI generators. Not 4,803 images. 4,803 generators. Their finding changed how I think about detection: accuracy scales with the diversity of generators in the training set, not the volume of images. The more types of AI a detector has seen, the better it recognizes the ones it hasn’t.
That model — CommunityForensics (CommFor) — is one of my new AI fighters. But it gave only a 15% chance that our guy does not exist.
Fabrizio Guillaro and Luisa Verdoliva at the University of Naples built B-Free, also in use by InVid,. They noticed most detectors accidentally learn that AI images tend to show certain subjects — castles, fantasy landscapes, portraits of young women — and then flag those subjects instead of the actual generation artifacts. It’s the equivalent of a drug-sniffing dog that learned to alert on backpacks because the training always hid the drugs in backpacks. B-Free avoids this. Also one of my new friends. The model says there is a 16% chance of AI.
AD
AI tools are evolving every day and the old tricks won’t get you very far anymore. The good news is that Henk van Ess is moving just as fast.
On March 26, he’ll share his latest methods and tools to detect AI-generated content, techniques that can save investigative journalists months of work.
As part of the upcoming GIJN’s Tech Focus Week, this session will present an updated version of his Reporter’s guide to detecting AI-generated content, already a go-to reference for journalists working on OSINT (available on the Global Investigative Journalism Network website)
📅 March 26, 2026 at 3pm CET
Register here:
https://lnkd.in/eZcHmCFg
The specialist screamed
Researchers at the University of Barcelona documented that detection accuracy drops from 79% on 2020-era generators to 38% on 2024-era generators. Every detector they tested was both the best and the worst performer, depending on which generator it was facing. There is a paper from 2024 literally titled “The Cat and Mouse Game.” One from ETH Zurich in 2025 called “The Unwinnable Arms Race.”
Meta trained an AI called DINOv2 to look at 142 million photographs without being told what to look for. No labels. No categories. It just learned to see patterns that differ between how a camera records light and how an AI generator invents it. Think of it as an AI that taught itself to read fingerprints by staring at 142 million hands without being told what a fingerprint was.
I froze that AI — all 300 million of its learned patterns, untouched — and taught it one new trick: spot Flux. By deep learning standards, this is studying for the bar exam by reading a pamphlet. But DINOv2 had already done the studying. It just needed to know what the question was.
Same image. Same park portrait. 18th model added.
The generalists shrugged. The specialist screamed.
Every image that passes through ImageWhisperer is quietly scored by the Flux probe in the background. Every night at 4:00 AM, a script retrains the probe on the expanded dataset, backs up the old model, deploys the new one, and runs a sanity check to make sure it still catches the park portrait.
No human intervention. No restart. No email that says “Henk, please retrain the model.” The first dry run found new Flux images and real images in the shadow archive. The training set grew from 176.000 to 216.000.
The detector wakes up a little smarter every morning. I wake up and check if something broke. So far, nothing. I say “so far” because I’ve learned that confidence in this field has a half-life of about six weeks.
In January, if you uploaded a photo and got a single number — 69% — that was it. You had to decide what to do with a number that was basically a coin flip in a lab coat. Now you get two separate verdicts: “Was this made by AI?” and “Was this photo edited?” Each backed by multiple independent models, plus a vision AI that describes what it actually sees. Plus a check against a database of known fakes scraped daily from PolitiFact, Snopes, and Google Fact Check. Plus web verification. Plus location analysis.
Not a number. A structured investigation.
A section called “What We Did Under the Hood” shows every check that ran. A section called “How We Reached This Verdict” walks through the reasoning in plain language. You see the evidence. You make the call. The tool is a colleague, not an oracle.
And last week, something happened that I did not expect. A major news publisher contacted me to integrate ImageWhisperer into their editorial verification workflow. A year ago this was a side project. Now it’s part of how a newsroom decides what’s real.
I’m not going to pretend that’s not a big deal. For a research specialist, having a publisher say “we want your tool in our pipeline” is like Mario finding out the plumbing company wants to sponsor his side quest.
For the full story of what broke in January: Why AI detection fails on the fakes that matter most. For seven types of image deception: The Hydra Problem.
Try the Detector - now with 21 models :)