Picture an owner who has an AI draft the replies to customer emails, then has a second AI call grade each draft before it goes out: "Score this reply from 1 to 10 for accuracy, tone and completeness." The grades come back as 8s and 9s, so the step looks like quality control. The research on AI judges says it is something narrower. A model asked to judge text can often tell which text is its own, and it scores that text higher. Older judges also leaned toward the answer that came first and the longer one; current judges lean hardest toward the version with headings and bullets. None of those is a property of the reply. They are properties of the judge. So "the model checked it" means a judge that leans toward its own kind reviewed its own kind.
What an AI judge is.
An AI judge is a model call whose job is to grade another model's output: score a draft, pick the better of two answers, decide whether a result passes. The pattern spread because one call replaces a review queue, and because the 2023 paper that named it reported GPT-4 agreeing with human voters about as often as humans agreed with each other.[2] That figure is real and most of the trust rests on it.
Two questions decide whether a judge's verdict is worth anything on your work. Does it score the same text differently depending on who wrote it, where it sits in the prompt, or how it is formatted? And when it disagrees with a person, who is right? The studies below answer the first with numbers, and the second only for chat answers and news summaries.
It can tell its own work apart.
Panickssery, Bowman and Feng took three models (GPT-4, GPT-3.5 and Llama 2) and two news-summarization datasets, XSUM and CNN/DailyMail, 1,000 articles each. Each model summarized every article, then was shown pairs of summaries and asked two separate questions. Self-preference: which summary is better? Self-recognition: which summary did you write?[1]
The recognition question is the less obvious one, and the argument rests on it. With no training, GPT-4 identified its own summary against the other models' and a human's 73.5% of the time, and the paper's introduction says all three models beat chance, with one exception in its own table (see fig 1).[1] The authors call this "non-trivial self-recognition capability out of the box". The preference question came out the same way: GPT-4 preferred its own summary 70.5% of the time on XSUM and 91.2% on CNN/DailyMail, where 50% would mean no preference. GPT-3.5 and Llama 2 sat much closer to 50%.
| evaluator | recognizes its own · XSUM / CNN/DM | prefers its own · XSUM | prefers its own · CNN/DM |
|---|---|---|---|
| GPT-4 | 67.2% / 74.7% | 70.5% | 91.2% |
| GPT-3.5 | 53.5% / 48.1%* | 58.2% | 43.1% |
| Llama 2 | 51.4% / 50.5% | 51.1% | 50.5% |
Then the authors did the thing that turns a correlation into an argument. They fine-tuned GPT-3.5 and Llama 2 to get better at recognizing their own output, on as few as 500 examples. The paper's introduction reports recognition rising above 90%, and the preference for their own summaries rose with it, on a straight line: the paper reports "a linear correlation between self-recognition capability and the strength of self-preference bias".[1] To rule out the boring explanation, they also fine-tuned models to recognize length, vowel count and a readability score instead; those fell on the same line, which the authors read as preference tracking recognition rather than any one surface feature.
One caution, and its consequence. The abstract describes self-preference as favoring one's own output when "human annotators consider them of equal quality", but the paper reports no human quality ratings of its own; controlling for generation quality is named as a limitation. So on the raw numbers alone, GPT-4's summaries might simply have been better. What the fine-tuning result adds is that the preference moves with recognition while the summaries stay the same, and that is the part that should bother anyone running a judge.
A 2025 follow-up asked the quality question directly. Chen, Wei, Zhu, Feng and Meng tested evaluators on math, factual and coding tasks where the right answer is known, and found that much of a stronger model's preference for its own answers is earned: its answers were better. The harmful part shows up when the evaluator got the task wrong itself, and stronger models showed more of it in those cases.[7] So ask your judge the narrower question: does it favor itself when it is wrong?
This is a different route into the problem from our note on labels. There, the judge was told whose work it was and the label moved the score. Here, nobody told it anything. It worked out the author from the text.
How big the tilt is, measured against people.
Wataoka, Takahashi and Ri asked the question the summarization paper left open: when a judge favors its own answer, is it wrong? They used Chatbot Arena, 33,000 conversations in which two models answer the same prompt and a human votes for the better one. Eight models served as judges on pairs that sometimes included their own answer.[3]
Their metric is easy to state in words. Take only the pairs where the human preferred one answer. How often did the judge agree with the human when the winning answer was the judge's own, and how often when it was someone else's? For GPT-4 the two numbers were 0.945 and 0.425: it agreed with people 94.5% of the time when they had picked GPT-4's answer, and 42.5% when they had picked another model's. The gap, 0.520, is the paper's headline measure, on a scale where "a value of 0 indicates the absence of bias, while a value close to 1 suggests a high degree of bias".[3] On the same human-labelled pairs, the judge was more than twice as likely to side with the person when the person had sided with it.
The authors' explanation is the useful part. They measured the perplexity of each answer under each judge, roughly how unsurprising the text is to that model, and found that judges "assign significantly higher evaluations to outputs with lower perplexity than human evaluators, regardless of whether the outputs were self-generated".[3] The judge rewards text that reads the way it would have written it, signature or no signature. Its own output is the extreme case; a model trained on similar data, or a draft that has already been through a model, gets some of the same credit.
That matches what the people who named the problem saw and declined to claim. Zheng et al. noticed GPT-4 winning about 10 points more often when GPT-4 judged, and Claude-v1 about 25 points more, but wrote: "Due to limited data and small differences, our study cannot determine whether the models exhibit a self-enhancement bias."[2] By 2024 two separate groups had measured it with controls.
A 2026 preprint complicates the picture. Yang et al. tested 20 current judges on 100 questions, using two other models to decide which pairs were equal in quality, with three experts checking the hard cases (79.8% agreement). Eight judges favored their own answers, nine favored others' and three showed no tilt; the strongest self-preference was LongCat-Flash-Chat at 0.307, and Claude-Sonnet-4.5 leaned the other way at −0.229.[4] Their summary: "stronger quality or higher discriminability does not necessarily imply lower SPB". You cannot guess a judge's tilt from how good it is. The quality labels come from models rather than people and the paper is listed under the journal Information Sciences with no acceptance status we could confirm, so read its values as provisional.
Position, length and formatting move the score too.
Self-preference gets the attention because it sounds like vanity. The other tilts are duller and more common, and the 2023 paper measured them on 80 questions from its MT-Bench set.[2]
Position. Show a judge two answers, swap their order and ask again. A consistent judge gives the same verdict both times. GPT-4 did so 65.0% of the time; GPT-3.5 46.2%; Claude-v1 23.8%, favoring whichever answer came first in 75% of pairs. The authors' line: "Only GPT-4 outputs consistent results in more than 60% of cases." They add that the pairs were close in quality, which is exactly where you would want a judge.
| judge (2023) | same verdict after swap | favored first position | favored second |
|---|---|---|---|
| GPT-4 | 65.0% | 30.0% | 5.0% |
| GPT-3.5 | 46.2% | 50.0% | 1.2% |
| Claude-v1 | 23.8% | 75.0% | 0.0% |
The large follow-up, Shi et al., ran 15 judges over more than 150,000 judgments and put the best 2024 judges (GPT-4, Claude-3.5-Sonnet) at 0.82 consistency on the same benchmark, with weaker judges at 0.57. Their finding about cause is the one to keep: prompt length barely matters, the quality gap matters a great deal. "Answer pairs or series with larger quality disparities are easier to achieve judgment consistency, whereas those of similar quality are difficult to judge."[6] When two drafts are close, the verdict is partly the order you listed them in.
Length. Zheng et al. took 23 answers that contained a numbered list, had GPT-4 pad each list with rephrasings that added nothing, and asked judges to compare the padded answer against the original. Claude-v1 and GPT-3.5 chose the padded version 91.3% of the time; GPT-4, 8.7%.[2] By 2026 the picture had changed shape rather than gone away: Soumik's expansion test found Llama 3.3-70B at +0.44 and Gemini 2.5 Pro at +0.40 in favor of the longer version, GPT-4o at −0.04, and Claude Sonnet 4 at −0.12, the one judge that preferred the shorter answer.[5]
Formatting. This is the bias the 2026 pass found largest. Given the same content as plain prose and as markdown with headings and bullets, every judge leaned toward the markdown: Gemini 2.5 Pro +0.76, Gemini 2.5 Flash +0.72, Claude Sonnet 4 +0.68, Llama 3.3-70B +0.40, GPT-4o +0.10. The author's summary: "Style bias is the dominant bias." Two human annotators shown 30 of the same pairs preferred the markdown 57% of the time, so part of that preference is shared with people and the rest is the judge's.[5]
The same preprint reports position bias at or below 0.04 for all five current judges and calls it negligible.[5] We would still test rather than assume: the formatting result shows a bias can move rather than disappear.
The 85% agreement, and what it hides.
The figure that sells AI judges is in Zheng et al.: GPT-4 agreed with human experts 85% of the time on MT-Bench, "which is even higher than the agreement among humans (81%)".[2] Three things sit inside that number.
It excludes ties. Count the pairs where the human or the judge called it even and the agreement falls to 66%, against 63% between humans. On the crowdsourced Chatbot Arena votes the pattern repeats: 87% without ties, 64% with them. The high figure is the share of clear-cut pairs where GPT-4 sided with the human majority, not the share of its verdicts that are right.
It is agreement on chat answers. MT-Bench is 80 open-ended questions (writing, roleplay, reasoning, coding) and the Arena data is whatever people typed into a public demo. The number says GPT-4 ranks chat answers the way a majority of people do. It says nothing about whether it can tell a correct invoice total from a wrong one. On the paper's own 10 math questions, the default judge failed 70% of comparisons; a reference answer brought that to 15%.[2]
It is the same judge that was 65% position-consistent. A judge can agree with the human majority on clear pairs and still flip on close ones depending on order. The 85% describes the easy cases; the position table describes the hard ones, and the work you bother to review is mostly hard ones.
What to run instead, and what each one costs.
Acceleratech does not build benchmarks. Small businesses do run AI judges over AI drafts: a grader on customer replies, a reviewer on extracted invoice fields, a "does this read well" step before a page goes out. Here is how we would apply the papers to that kind of step. The steps are our reading, not the authors' advice, and each has a cost we state.
- Build a known-good set and a known-bad set before the judge grades anything.
Collect drafts a person you trust has already graded, including some that are wrong in ways that matter (a wrong total, a promise you do not make). Run the judge on them. If it passes the known-bad ones, its 8s and 9s mean nothing. Report two numbers: how often it agrees with the person, and whether that differs between AI-written and human-written items, the split that exposed GPT-4 in the Arena study.[3] Cost: a person's time to label the set, once, and again when the drafts change character. Yang et al. priced human labelling of the same stage (20 models' answers to 100 questions, 2,000 scores) at an estimated US$5,000–7,500 against US$77.81 for the two model judges.[4]
→ the judge is graded before it grades - Give it a reference, not a taste question.
"Is this reply good?" is the exposed surface. "Does this reply contain the three facts in the approved answer, and nothing the policy forbids?" is a comparison. In Zheng et al., a reference answer took the math-grading failure rate from 70% to 15%.[2] Yang et al. found that a forced choice on five named dimensions, instead of one overall score, cut self-preference by 31.5% on average across the four most biased judges, and that asking for step-by-step reasoning sometimes made it worse.[4] Cost: someone writes the reference or the dimensions, and the prompt gets longer. The 2023 paper's few-shot judge raised GPT-4's position consistency from 65.0% to 77.5% at four times the cost per call.[2]
→ comparison over opinion wherever a reference exists - Use a different model family as the judge, and test that one too.
The perplexity finding says a judge rewards text that reads like its own, so a judge from a different vendor than the writer removes the extreme case.[3] It does not remove the tilt: the 2026 study of 20 judges found 17 of 20 judges significantly biased one way or the other.[4] Cost: a second vendor account, a second prompt to maintain, and the step 1 test run again. Soumik's cross-family panel results sit in an appendix that is empty in the version we read, so we cannot tell you what a panel buys.[5]
→ the writer's family never grades the writer - For any A-versus-B call, ask twice, swapped.
Run the pair in both orders. Agreement means the verdict is about the drafts; a flip means they are close and the verdict was about the order, which is when Shi et al. found position bias bites.[6] Cost: twice the calls. Soumik found position bias negligible on five 2025 judges, and found that swapping cost three of the five judges 6.5 to 11.1 points of accuracy on deliberately misleading pairs (the other two moved 1.5 and 4.0 points).[5] So run the swap on your known-good set first; if the judge is consistent there, stop paying for it.
→ order cancelled out, or shown not to matter - Strip formatting and match length before the judge sees anything.
Convert both versions to plain text and bring them to a similar length, then judge. That removes the two biggest tilts the 2026 pass found, markdown and length, for the price of a preprocessing step.[5] If formatting is what you care about, judge it with a rule, not a model.
→ the judge sees content, not layout - Keep a person on a sample, and pick the person carefully.
Have someone read a fixed fraction of what the judge passed, every week, and log where they disagree. Humans disagree with each other too (81% on the clear pairs in Zheng et al.), so pick the person whose call you would defend to a customer.[2] Cost: that person's hours, forever. It is the only check here that does not inherit the judge's taste.
→ the one check with no model in it
If you only do two, do the first and the last. A judge that passed a known-bad answer is not a judge, and a judge nobody audits drifts unnoticed. The known-good test on your best human page from our earlier note is the writing-quality version of step 1.
What this note does not claim.
It does not say AI judges are useless. On clear-cut pairs, GPT-4 agreed with human experts as often as experts agreed with each other, and a reference cut its failure rate sharply.[2] The claim is that a judge's verdict on model-written text, with no reference and no human sample, is weak evidence, and that the weakness is measurable.
The self-preference numbers are from older models and specific tasks. Panickssery et al. used GPT-4, GPT-3.5 and Llama 2 on news summaries; Wataoka et al. used 2023-era Arena models. The 2026 study of 20 judges found current judges tilting both ways, including against their own output.[4] Your judge's tilt could be small or reversed, which is a reason to measure it.
The two 2026 sources carry caveats. Yang et al. is listed under the journal Information Sciences with no acceptance status we could confirm, and its quality labels come from two other models; Soumik's arXiv record says it was published in Transactions on Machine Learning Research (2026), but we read the v2 preprint, which has an empty appendix, runs dated a year before its posting, and some figures that disagree between text and tables. We use them for direction only.
None of this was tested on your work. Every dataset here is English chat answers, news summaries or benchmark questions. Nobody has measured self-preference on invoice extraction or French customer replies. The Monday checks are built to work without that research: they compare the judge against answers you already know.
If an AI judge gates something in one of your workflows and you want to know what its pass rate is worth, describe the step (no credentials or customer data) through the contact form. We will write back, free, with the known-good set we would build for it and which of the six checks we would run first. Building review steps that measure the work rather than the judge is part of our workflow automation services.