Experiment

I Asked 4 AIs to Review Each Other's Work. The Most Brutal Critic Wasn't the One I Expected.

What happens when AI models judge each other instead of just answering your questions?

Illustration of AI models reviewing each other's work
Generated with ChatGPT

I've been seeing a lot of people test how different AI models answer the same question. The usual setup: ask ChatGPT, Claude, and Gemini the same thing, screenshot the results, pick a favorite.

But here's what nobody seems to be doing: asking the models to judge each other's answers.

Not self-review. Not "rate your own output". Actual peer review. Anonymized. Scored. Ranked.

So I ran an experiment.

I gave the same workplace dilemma to four leading AI models — GPT-5.5 Instant, Claude Opus 4.6, Gemini 3.1 Pro, and DeepSeek V4 Pro. The question was deliberately messy, the kind that doesn't have a clean answer:

"Should I tell my boss I use AI to help with reports and presentations?"

All four models gave thoughtful, well-structured responses. They covered policy, confidentiality, framing strategies, sample scripts. On the surface, they looked equally helpful.

Then I did something different.

I anonymized all four answers (labeled A, B, C, D), stripped away any identifying markers, and gave the complete set back to each model with a new instruction:

"You are a serious critic, not a polite collaborator. Evaluate these four answers. Ignore writing style. Focus on reasoning quality, hidden assumptions, ethical seriousness, and practical usefulness. Then rank them from strongest to weakest."

What came back was brutal.

The Experiment: Peer Review at Scale

Here's how the scoring worked. Each judge rated all four answers across five dimensions:

Maximum score: 25 points.

Then each judge had to:

  1. Rank all four from strongest to weakest
  2. Name the single answer they disagreed with most
  3. Identify the most important blind spot across the entire set

I wasn't looking for consensus. I was looking for what each model saw that the others missed.

The Results: When AI Gets Honest

Average peer review scores each model received from the other three models
Average scores each model received from the other three models.

The first thing that jumps out: Answer C (DeepSeek) dominated. Three out of four judges ranked it strongest. GPT gave it a 22/25. DeepSeek gave it a perfect 20/25.

Wait. DeepSeek gave its own answer a perfect score?

Yes. And that's where it gets interesting.

The Self-Preference Problem

Self-review scores vs peer review scores across four AI models
Self-review scores vs. peer review scores across four AI models.

When I mapped out how each judge scored its own anonymized answer versus the others, a clear pattern emerged:

DeepSeek showed the strongest self-preference bias. But here's the twist: all three other judges also ranked DeepSeek's answer as the strongest. So was it bias, or was DeepSeek's answer genuinely better?

This pattern — where systems rate their own output more favorably than peers do — isn't unique to AI. Psychologists call it self-serving bias, and it shows up everywhere from academic peer review to performance evaluations.

And then there's Claude and Gemini — both models that were harder on their own work than on others'. Claude ranked its own answer dead last. Gemini gave itself an Ethics score of 1 out of 5.

That's not modesty. That's self-awareness.

What the Judges Actually Said

DeepSeek on Claude's Answer (Ranked Weakest)

"It doesn't just reach the wrong conclusion — it reaches a dangerous conclusion while presenting itself as measured and balanced. Its central claim that 'the risk of being caught later is usually worse than the risk of sharing now' is the kind of universalizing statement that sounds wise but collapses under scrutiny. It's written from the perspective of someone who has never feared their boss."

Ouch.

Gemini on Its Own Answer (Ethics: 1/5)

"The explicit advice to cover up a data breach is catastrophic. It prioritizes the employee's immediate comfort over the company's legal obligations. Discovering you leaked data and choosing not to report it transforms a fireable mistake into a prosecutable cover-up."

Gemini didn't just critique itself — it identified a legally disastrous flaw in its own reasoning that none of the other judges caught.

GPT on Claude's Answer

"Its optimism about AI becoming mainstream may lull someone into thinking normality equals acceptability. The legality, ethics, and professionalism of AI use depend less on whether AI is popular and more on what data was used, what output was produced, who relied on it, what rules apply, and whether the employee had authority."

Claude on Its Own Answer (Ranked Last)

"The most generic. It reads like a well-formatted blog post written without knowing anything about the user's situation. The 'gentle transparency' concept is never given teeth. The closing chatbot prompt ('Would you like help drafting…?') reveals it as a template, not a judgment."

Claude called itself out for sounding like a chatbot.

The Four Blind Spots: What Each Model Saw That Others Missed

This is where the experiment got genuinely valuable. Each judge identified a different structural blind spot that all four original answers had missed.

DeepSeek: The Collective Action Problem

"None of the four answers engage with the structural labor economics of AI disclosure. When workers voluntarily report that AI makes them '30% faster,' those numbers accumulate. They become the business case for headcount reduction. The individual incentive to disclose (stand out as innovative) conflicts with the collective interest (keep productivity gains opaque to management). This is a prisoner's dilemma, and not one answer names it."

Gemini: The IP & Commoditization Trap

"All four answers falsely equate AI to benign tools like spellcheckers. They completely blind the user to two realities: 1) Companies cannot copyright AI-generated text — legal ownership is compromised even if you edit it. 2) By demonstrating that AI can do 70% of your job, you're inviting management to realize they just need a cheaper junior employee to proofread the output."

GPT: The Escalation Gap

"All four answers discuss policy and confidentiality, but none fully confront the hardest version: the employee may have already used unapproved AI with sensitive data. The real path isn't 'tell your boss' — it's: stop immediately, document what/which tool/when, check policies, use the appropriate escalation channel (IT security, legal, compliance), avoid minimizing it as productivity enhancement, and don't continue behind the scenes."

Claude: The Dependency Illusion

"None of the answers seriously confront the possibility that the user is more dependent on AI than they believe — or that the 'human-in-the-loop' framing may be a comfortable fiction. The most common pattern is graduated dependency: you start editing AI drafts heavily, then edit less, then mostly just prompt and submit. If that's where the user actually is, all four answers help them construct a disclosure narrative that misrepresents their actual workflow. The hardest question — How much of this work is genuinely yours? — is the one none of them ask."

Why This Matters

  1. Every model has a perspective, and a blind spot. DeepSeek sees labor economics. Gemini sees legal liability. GPT sees process gaps. Claude sees self-deception. None of them see all four.
  2. Self-review doesn't work because models inherit their own framing. When you ask an AI to review its own output, it improves clarity and structure. But it doesn't question the premise.
  3. Peer review works because different models have different priors. The value wasn't in the rankings. It was in the four completely different critiques.
  4. The "right answer" isn't a single output — it's the overlap of multiple perspectives.

How I Actually Did This (And How You Can Too)

I used a multi-model workspace so I could ask the same question to four models, switch for the peer-review round, and keep answers and critiques in one place side by side.

Side-by-side comparison view with model selector in a multi-model workspace
Screenshot — side-by-side comparison view with model selector.
  1. Round 1: Asked the same question to GPT, Claude, Gemini, and DeepSeek. Saved all four answers in a project file.
  2. Round 2: Anonymized the answers (A–D) into one document.
  3. Round 3: Gave that document to each model with the peer review prompt.

The key was keeping everything in one place. I didn't have to copy-paste between tabs or lose track of which model said what.

The Takeaway

If you're using AI for anything that matters — decisions, analysis, writing, advice — here's what this experiment taught me:

Don't ask one AI and assume you're done.

Ask multiple models. Make them disagree. Look for what each one sees that the others miss.

The "second opinion" isn't about finding the "better" model. It's about finding the blind spot in the first answer — and the second, and the third.

Because the most dangerous thing AI can do isn't give you a wrong answer.

It's give you a confident, well-structured, persuasive answer that's missing something you didn't know to look for.

Experiment conducted June 2026 using GPT-5.5 Instant, Claude Opus 4.6, Gemini 3.1 Pro, and DeepSeek V4 Pro.

Also on Medium / Activated Thinker.