Never tell an AI reviewer whose work it is. The label alone moves the score.

A quote goes out of a small firm only after a second AI pass has read it. A proposal section gets a score before a person sees it. An inbox assistant drafts a reply and the same assistant is then asked whether the reply is good enough to send. In each case a sentence at the top of the review prompt says whose work is being judged: "review your draft", "check the assistant's reply", "score the proposal the other model wrote". Few review steps are ever tested for what that sentence does to the verdict. A preprint from five researchers at KAIST, updated in September, measures exactly that on ten AI models, and the sentence does change it.[1]

If you want the test first, skip to the ten-minute version.

The sentence at the top of every review prompt.

Earlier research on AI reviewers (the field calls them LLM judges) had already found that a model tends to rate its own output higher than another model's. Our note on AI judges grading their own homework covered one such result: one LLM reviewer accepted machine-written text 1.42 to 1.91 times as often as human-written text. The trouble with measuring this on generated text is that three things travel together: the judge's own writing style, which it might recognize; the real quality of the work; and any favouritism toward itself. A higher score for its own work could be any of the three.

The KAIST team's move was to take the text away. Instead of judging written output, each of ten models picked 20 items from a shared pool of 200 pre-written one-sentence story constraints (an event, a setting, a character trait), 30 times each, and then every model scored every set of picks on a four-part, 7-point rubric. No model wrote a word, so there is no style to recognize. Yet the picks still carry a fingerprint: a simple classifier could tell which model made a set 50.7% of the time, against a 10% chance rate.[1] That gives a clean instrument. Whatever the judge does with a set of picks, it is not reacting to prose.

Same work, two labels.

The experiment that matters for a business is the second one. For each judge, the team paired its own sets of picks with other models' sets that the same judge had, blind, scored almost identically (within 0.25 on the overall mean and within one point on every part of the rubric). Each set was then scored again under one of two anonymous sentences: "your own selection", or "another language model". No model was named. Each set was scored three times per condition, for 4,800 scored runs in total. Because the true source and the shown label were crossed, half the labels were lies.

Read across fig 1 and the label does the work: 4.92 against 4.50. Read down and the true source does almost nothing: 4.92 against 4.91. In the authors' model the label effect is +0.43 on the 7-point average and significant on every part of the rubric (from +0.29 on coherence to +0.57 on dimensionality, with originality at +0.53 and tellability at +0.32), while the effect of who really produced the set is +0.02 and not significant.[1] The interaction between label and truth is null, which is the detail to keep: a false label moved the score exactly as much as a true one. The judge responded to the sentence, not to any recognition of its own work.

Up, down, or both: it depends on the model.

A single effect size hides the direction. Measured against each judge's own blind baseline, the shift came in two parts that can appear together or alone: scores went up under the "your own" label, and down under the "another model" label. The paper's Table 6 gives both numbers for each judge, and the pattern is model-specific.

Six of the ten judges show the symmetric pattern: a little kinder to "their own" work, harder on "another model's", with the penalty usually the larger half. Two show what the authors call self-boost: Claude Opus 4.7 (+0.275 under the self label, no reliable change under the other label) and DeepSeek-V4-Pro (+0.443, likewise). GPT-5.5 is the mirror image: no self-inflation, but a 0.34-point penalty on anything labelled as another model's. Llama 4 Maverick, one of the two judges the authors set aside in the blind round for barely using the scale, marks down under both labels.[1]

One piece of arithmetic, ours and not the paper's, says what a shift that size does to a gate. If your rule is "send at 5 or better" on a 7-point rubric, anything the reviewer would score 4.6 to 4.9 blind now passes under a self label, and anything it would score 5.0 to 5.4 blind can fail under an other label. The items near the line are exactly the ones a review step exists for.

For a business the pattern matters more than the average, because it says where your review step is exposed. A self-boost judge is the dangerous one in a self-check loop: ask it whether "your draft" is ready and it is a quarter of a point kinder than it would be blind, on a 7-point scale, with nothing about the draft changed. An other-penalty judge is the dangerous one in a hand-off: tell it the draft came from "the assistant" or "the other model" and it marks down work it would otherwise pass. Since the sentence is the only thing that changed, the sentence is the thing to remove.

Blind, self-preference mostly disappears.

The first experiment is the one that keeps the second honest. Before any labels, every judge scored all 300 sets blind, three times each. On the face of it, seven of ten judges rated their own sets higher, by as much as 1.09 points for one model. But strong selectors got high scores from every judge, including themselves, and the authors' statistical model separates that from favouritism by accounting for how harsh each judge is, how good each selector is, and the fact that each set is scored repeatedly.

Once the two judges that barely used the scale are set aside, the self-favouring effect falls to +0.003 and is not significant on any part of the rubric; the only effect left is a small negative one on originality, the opposite of favouritism.[1] Refitting on all ten judges with a different model specification gave the same answer. So in a setting where no style can leak and the quality of the work is accounted for, the judges' favouritism followed the label, not the true source.

Two things this does not say. It does not say self-preference does not exist. The authors' own word is "disappears", on this task, under blind scoring, with a small effect left in the opposite direction; our reading is "small in this setting", not "absent", because the clean part of self-preference, the part not explained by quality and harshness, was measured only where no prose could leak. And it does not touch the number we reported in July. The 1.42 to 1.91 times over-acceptance of machine-written text is a machine-versus-human preference on generated prose, where style can leak, and it measures something different from self-versus-other on identical targets. Both can be true. The reference-anchored judging and the adversarial calibration set from that note stay the fix for a judge that drifts toward machine text; this paper's blind protocol is the fix for a judge that is told whose work it grades.

A second domain, and a twist.

Picking story constraints is a long way from reviewing a proposal, and the authors knew it. Their replication had the same ten models write 90 short research proposals, had a separate model rewrite all of them in one neutral style to flatten each author's voice, scored them blind on a standard five-part review form, then re-scored 85 quality-matched pairs under the same two anonymous labels.

The label effect reproduced on real written work: +0.196 overall on a 10-point scale, with the true source again doing nothing significant.[1] The twist is feasibility, which went the other way. Told a proposal was its own, a judge rated it more novel, more exciting and less feasible. Attribution changes the judgment in ways that depend on what is being judged. Our conclusion from that: strip the label rather than try to correct for it, because a correction would have to know the direction for each criterion in each model, and the direction changes.

The fix is one sentence shorter.

Nothing in the paper requires a new tool. The label was a sentence in the prompt, so the defence is to not write the sentence. "Review this draft against the brief" carries no owner. "Review your draft" is a self label, and in this paper a self label was worth between +0.08 and +0.44 over a blind score, depending on the judge. "Review the assistant's draft" or "check the other model's answer" is an other label: eight of ten judges marked down on it, by up to 0.52. The gap between the two labels was 0.43. The label does not need a model name to bite, and a false label bites as hard as a true one, so the same rule covers a reviewer that is a different model, the same model in a fresh conversation, or the same conversation checking its own work.

Three consequences follow for how review steps get built, and these readings are ours, not the paper's.

First, a self-check in the same conversation ("now review what you just wrote") cannot avoid the self label: the model knows the draft is its own because it just produced it. The authors name self-refinement (a model asked to improve what it just wrote) as the setting where self-preference is known to amplify; that a same-conversation check cannot hide the author is our reading. A reviewer in a fresh conversation, handed the draft with no history, can avoid it. That is the structural reason to prefer a fresh reviewer over self-critique, a point our note on auditing an automation by reading one file made from the other side.

Second, the fresh reviewer only helps if the hand-off prompt does not reintroduce the label: "here is the agent's draft" undoes the separation.

Third, this is a search-and-delete across your prompts, not a project. Each prompt that reviews, approves or gates something in your automations has an opening sentence. Read each one for an owner, and remove it.

Test your own review prompt in ten minutes.

The paper's method scales down to a smoke test any operator can run on a review step they already have. The paper used quality-matched pairs, three repetitions and a statistical model; this version has none of that, and it only tells you whether your reviewer is reading the label.

1. Pick one artifact your reviewer already scores.

   A quote draft, a client email, a section of a proposal, an inbox reply. Keep it fixed for the whole test; only the opening sentence of the review prompt changes.

   → one artifact, unchanged throughout
2. Run the review three ways, three times each.

   Once opening with "review your draft", once with "review the draft another AI assistant wrote", once with no owner at all ("review this draft"). Same rubric, same scale, a fresh conversation every time so no history leaks. Nine runs; on a scored rubric this takes about ten minutes.

   → 3 labels × 3 runs, fresh conversation each
3. Compare the three averages.

   If the "your draft" and "another assistant's draft" averages sit apart by a noticeable fraction of your scale while the artifact never changed, your reviewer is scoring the label. In the paper the gap was about 0.4 on a 7-point scale; your number will differ, and with three runs it is a signal, not a measurement.

   → a gap with nothing changed = the label is scoring
4. Strip the attribution from every review prompt, and keep the no-owner version.

   Then re-run the no-owner prompt a few times to see how much the score moves on its own; that spread is the noise floor any single verdict sits in, and a good reason to pair the reviewer with a reference (the brief, the checklist, the approved example) rather than a bare "is this good".

   → no owner in the prompt, a reference in the rubric

What this does not claim.

One main domain, one replication. The clean result is on picking story constraints, a creative task chosen for control rather than realism. The replication on research proposals is one further domain, and there the direction flipped on one criterion. Whether the effect holds where there is a right answer (a calculation, a rule check) is untested.

Explicit labels only. The paper put the label in the prompt. Real pipelines often carry implicit cues instead: a model's phrasing, a tool's interface, a file name. The authors say plainly that whether those cues bias a judge the same way is not examined here. Removing the sentence removes the cue this paper measured, not every cue.

A rubric nobody validated against people. The four-part rubric is offered as a consistent frame for the judges, not as a measure of quality that humans would agree with. The score shift is real; what a 4.92 means is not established.

English only, ten judges, two of them blunt. Everything ran in English. Two of the ten models (Llama 4 Maverick and Mistral Large 3) barely used the scale, and the headline blind result depends on setting them aside (the authors show the alternative fit as well).

A preprint, version 2. Not peer-reviewed and not rerun by anyone we know of. Treat the numbers as the authors' claims. Our ten-minute test above is a smoke test built on their design, and it has none of their controls.

If you run an AI review step and have not read its opening sentence for an owner, send us the prompt through the contact form and you get a free written first read of where the label sits and what to replace it with. Work like this falls under our assistants and agents services.