Jump to content

LLM-as-a-Judge

From Wikipedia, the free encyclopedia

Basic LLM-as-a-judge workflow: a LLM scores or compares a candidate response against given criteria and returns a rationale.

LLM-as-a-Judge (also LLM-based evaluation or language model-based evaluation) is a technique in natural language processing in which a large language model (LLM) is used to assess the quality, relevance, or correctness of a text output, typically one generated by another model, against predefined criteria. It is used as a scalable, lower-cost alternative to human annotation and to reference-based metrics such as BLEU and ROUGE, which measure word overlap rather than meaning.[1][2]

Definition

[edit]

In LLM-as-a-judge, a large language model acts as an evaluator: it receives an evaluation prompt specifying the target text, the criteria, and the required output format, and returns a judgment such as a numerical score, a categorical label, or a preference between candidates. Because the criteria are expressed in natural language, the same judge can be repurposed across tasks by changing the prompt rather than by retraining.[1][3] Instead of relying solely on human annotators, the approach uses the general language capabilities of large language models to serve as automated judges.[1][2] The judge usually returns a numerical score or a verdict, and in many implementations also produces a natural-language explanation for its decision.[3][2] Sources also call the technique "LLM-based evaluation",[3] or "language model-based evaluation".[4]

Judging tasks are commonly distinguished along two axes. The first contrasts subjective evaluation, in which there is no single ground-truth answer, such as rating the helpfulness or coherence of an open-ended response, with objective evaluation of outputs whose correctness can be verified, such as answers to mathematical or factual questions. The second contrasts reference-free judging, in which the model assesses an output using only the stated criteria, with reference-based (or reference-guided) judging, in which a gold answer or scoring rubric is supplied.[1][2]

Methods

[edit]

Surveys organize the methods for building LLM judges into prompting a general-purpose model and tuning a model specifically for judging.[2][1] The simplest approach uses a general-purpose model as a judge with no additional training, so that the prompt alone specifies the task, the criteria, and the output format. Representative prompting-based methods include G-Eval, which prompts the model to produce chain-of-thought evaluation steps and then fill in a scoring form, and GPTScore, which instead scores a text by the generation likelihood the model assigns to it.[5][6]: 3  A common pattern is to use a more capable model to evaluate the outputs of a weaker one, for example GPT-4 grading a 13-billion-parameter LLaMA model.[7]: 7  A second approach fine-tunes openly available models specifically for judging, which is intended to be cheaper and more reproducible than proprietary judges accessed through an API: PandaLM is trained on judgments produced by GPT-3.5,[8] and JudgeLM on judgments produced by GPT-4,[9] while the Prometheus models are trained to score responses against explicit rubrics and were reported to approach GPT-4 when supplied with a reference answer and a rubric.[10][4]

Judging need not rely on a single text-only model. Multiple judges drawn from different model families can be combined into a panel or jury so that their votes offset individual biases, as in the panel of LLM evaluators, which is also discussed as a means of mitigating bias.[11]: 1  The approach has also been extended to multimodal judges that assess text together with images or video, including the MLLM-as-a-Judge benchmark and the open Prometheus-Vision.[12][13]

Reliability

[edit]

The reliability of an LLM judge is the extent to which its verdicts agree with human judgment and remain stable, unbiased, and reproducible. LLM judges exhibit a range of systematic biases and vulnerabilities to manipulation that complicate their use as a substitute for human evaluation.[2][14]

Biases

[edit]

Surveys of LLM judges describe several recurring biases, among them position (order) bias, verbosity (length) bias, and self-preference (self-enhancement) bias.[2][1] Position, or order, bias is the tendency to favor a response according to its position in the prompt. An early 2023 study found that judges frequently favored the answer shown first and often reversed their verdict when the order was swapped; among the models it tested, even the strongest model tested, GPT-4, reached the same verdict both ways in only about two-thirds of cases.[7]: 5  A later large-scale study found this effect to be systematic rather than a matter of chance, indicating it persists across models.[15] Verbosity, or length, bias is a tendency to prefer longer answers independently of quality. The same 2023 study that documented position bias found that, when answers were rephrased to be longer without adding new information, judges such as Claude and GPT-3.5 still preferred the longer version more than 90% of the time.[7] Self-preference, or self-enhancement, bias is the tendency of a judge to rate its own outputs, or those of its model family, more highly, an effect sometimes called "LLM narcissism".[14][16] GPT-4 and Claude were observed to rate their own outputs more highly, although this early evidence was described as too limited to establish the effect firmly.[7]: 5  Judges also tend to favor well-formatted or authoritative-sounding answers,[1] and one study has proposed a taxonomy of twelve distinct bias types.[17]: 4 

Calibration and reproducibility

[edit]

One survey lists overconfidence among the biases of LLM judges, describing a tendency to exhibit inflated confidence in their judgments.[3] One study reported that judges such as GPT-4o and DeepSeek-R1 are poorly calibrated, expressing more confidence in their verdicts than their accuracy warrants; it measured this with the expected calibration error and the Brier score.[18]: 3  Reproducibility is a further concern. Because language model generation is inherently random, a judge can return different scores for the same input on repeated runs, and its judgments are further destabilized by minor changes in prompt wording.[1] Separately, the black-box nature and version dependency of proprietary judges accessed through frequently updated APIs hinder reproducibility, since the model behind the API may change over time.[14]

Agreement with human judgment

[edit]

A large-scale study across 20 natural language processing tasks concluded that judges were not yet reliable enough to systematically replace human annotators.[19]: 1  On tasks with a verifiable answer the gap can be large: on JudgeBench, a benchmark in which the judge must pick the better of two responses and one response is objectively correct, judges including GPT-4o performed only slightly above chance.[20]: 1  High apparent agreement can be misleading, because a judge may agree with human raters for the wrong reasons, relying on superficial cues rather than the intended criteria.[1] In an extreme demonstration, a constant "null" response that ignores the input obtained high win rates on automatic benchmarks, showing that such scores do not necessarily reflect answer quality.[21]: 1 

Mitigation strategies

[edit]

Surveys describe several techniques for reducing the biases of LLM judges, including swapping the order of candidates, aggregating multiple rounds or multiple judges, rule- and rubric-augmented prompting, and reference-guided judging.[2][1][3] One survey's own meta-evaluation concludes that such strategies are not fully effective at improving performance or mitigating biases.[1] Position bias can be countered by presenting the candidates in both orders and counting a win only when the same answer is preferred both times;[7] length effects can be reduced with controls such as the length-controlled win rate used by AlpacaEval 2.0, an automatic evaluation benchmark.[22]: 1  Pairwise comparison tends to give more reliable results than pointwise scoring,[1] and supplying reference answers with explicit rubrics can bring open judges closer to proprietary ones such as GPT-4.[10][4] Aggregating the verdicts of several judges from different model families, for example by majority voting, can offset the bias of any single model,[14][2] and breaking a judgment into explicit reasoning steps, in the manner of chain-of-thought prompting, is recommended to make judgments more transparent and auditable.[14] Before a judge is trusted to stand in for human raters, its agreement with humans can be checked against a gold standard using statistical tests such as the alternative annotator test (alt-test).[23]: 1  No single mitigation removes bias, and reviews recommend that LLM judges augment rather than replace human evaluation, especially in high-stakes settings.[14][1][19]

Benchmarks

[edit]

Benchmarks associated with LLM-as-a-judge fall into two groups: model leaderboards that rank candidate systems, several of which rely on an LLM judge for scoring, and meta-evaluation benchmarks that assess the judges themselves.[1]

The leaderboards include MT-Bench, a set of 80 multi-turn questions introduced in 2023 whose answers are scored by an LLM judge,[7]: 3  and the human-voted Chatbot Arena.[7]: 4  A 2025 study, "The Leaderboard Illusion", reported that such rankings can be distorted by undisclosed private testing and reuse of arena prompts.[24]: 1 

Other benchmarks instead evaluate the judges themselves. On JudgeBench, which pairs an objectively correct response with an incorrect one, many strong judges identify the correct answer only slightly more often than chance.[20]: 1 

Applications

[edit]

One application is the automatic evaluation of model outputs, including for public leaderboards.[1][3]

LLM judges can also support several production-facing uses. In safety moderation, a model classifies prompts or responses against a safety taxonomy, as in the Llama Guard family.[25] In retrieval-augmented generation, reference-free judges check whether a generated answer is supported by the retrieved context, as in the RAGAS framework.[26]: 1 

[edit]

Compared with earlier reference-based n-gram metrics such as BLEU and ROUGE, which rely on transparent, rule-based overlap with a reference text, an LLM judge can capture deeper semantic agreement but does so through the model's opaque internal reasoning, trading interpretability for sensitivity to meaning.[2] It also differs from a reward model, which is trained to output a scalar preference score used as a training signal, whereas a judge emits a verdict or score, often with an explanation.[27] Human evaluation remains the reference standard against which judges are validated, and reviews generally recommend that LLM judges augment rather than replace it.[1][19]

See also

[edit]

References

[edit]
  1. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 Gu, Jiawei; Jiang, Xuhui; Shi, Zhichao; et al. (2024). "A Survey on LLM-as-a-Judge". arXiv:2411.15594 [cs.CL].
  2. 1 2 3 4 5 6 7 8 9 10 Li, Dawei; Jiang, Bohan; Huang, Liangjie; et al. (2025). From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judge. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing (EMNLP). pp. 2757–2791. arXiv:2411.16594. doi:10.18653/v1/2025.emnlp-main.138.
  3. 1 2 3 4 5 6 Li, Haitao; Dong, Qian; Chen, Junjie; et al. (2024). "LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methods". arXiv:2412.05579 [cs.CL].
  4. 1 2 3 Kim, Seungone; Suk, Juyoung; Longpre, Shayne; et al. (2024). Prometheus 2: An Open Source Language Model Specialized in Evaluating Other Language Models. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP). arXiv:2405.01535.
  5. ↑ Liu, Yang; Iter, Dan; Xu, Yichong; et al. (2023). G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP). arXiv:2303.16634.
  6. ↑ Fu, Jinlan; Ng, See-Kiong; Jiang, Zhengbao; Liu, Pengfei (2024). GPTScore: Evaluate as You Desire. Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). pp. 6556–6576. arXiv:2302.04166. doi:10.18653/v1/2024.naacl-long.365.
  7. 1 2 3 4 5 6 7 Zheng, Lianmin; Chiang, Wei-Lin; Sheng, Ying; et al. (2023). Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena. Advances in Neural Information Processing Systems 36 (NeurIPS 2023), Datasets and Benchmarks Track. arXiv:2306.05685.
  8. ↑ Wang, Yidong; Yu, Zhuohao; Zeng, Zhengran; et al. (2024). PandaLM: An Automatic Evaluation Benchmark for LLM Instruction Tuning Optimization. The Twelfth International Conference on Learning Representations (ICLR). arXiv:2306.05087.
  9. ↑ Zhu, Lianghui; Wang, Xinggang; Wang, Xinlong (2025). JudgeLM: Fine-tuned Large Language Models are Scalable Judges. The Thirteenth International Conference on Learning Representations (ICLR). arXiv:2310.17631.
  10. 1 2 Kim, Seungone; Shin, Jamin; Cho, Yejin; et al. (2024). Prometheus: Inducing Fine-grained Evaluation Capability in Language Models. The Twelfth International Conference on Learning Representations (ICLR). arXiv:2310.08491.
  11. ↑ Verga, Pat; Hofstatter, Sebastian; Althammer, Sophia; et al. (2024). "Replacing Judges with Juries: Evaluating LLM Generations with a Panel of Diverse Models". arXiv:2404.18796 [cs.CL].
  12. ↑ Chen, Dongping; Chen, Ruoxi; Zhang, Shilin; et al. (2024). MLLM-as-a-Judge: Assessing Multimodal LLM-as-a-Judge with Vision-Language Benchmark. Proceedings of the 41st International Conference on Machine Learning (ICML). arXiv:2402.04788.
  13. ↑ Lee, Seongyun; Kim, Seungone; Park, Sue Hyun; et al. (2024). Prometheus-Vision: Vision-Language Model as a Judge for Fine-Grained Evaluation. Findings of the Association for Computational Linguistics: ACL 2024. arXiv:2401.06591.
  14. 1 2 3 4 5 6 Dietz, Laura; Zendel, Oleg; Bailey, Peter; et al. (2025). Principles and Guidelines for the Use of LLM Judges. Proceedings of the 2025 International ACM SIGIR Conference on Innovative Concepts and Theories in Information Retrieval (ICTIR '25). pp. 218–229. arXiv:2504.19076. doi:10.1145/3731120.3744588.
  15. ↑ Shi, Lin; Ma, Chiyu; Liang, Wenhua; et al. (2025). Judging the Judges: A Systematic Study of Position Bias in LLM-as-a-Judge. Proceedings of the 14th International Joint Conference on Natural Language Processing and the 4th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (IJCNLP-AACL). arXiv:2406.07791.
  16. ↑ Yiqi Liu; Nafise Moosavi; Chenghua Lin (2024), LLMs as Narcissistic Evaluators: When Ego Inflates Evaluation Scores, pp. 12688–12701, doi:10.18653/V1/2024.FINDINGS-ACL.753, Wikidata Q135734850
  17. ↑ Ye, Jiayi; Wang, Yanbo; Huang, Yue; et al. (2025). Justice or Prejudice? Quantifying Biases in LLM-as-a-Judge. The Thirteenth International Conference on Learning Representations (ICLR). arXiv:2410.02736.
  18. ↑ Tian, Zailong; Han, Zhuoheng; Chen, Yanzhe; et al. (2025). "Overconfidence in LLM-as-a-Judge: Diagnosis and Confidence-Driven Solution". arXiv:2508.06225 [cs.AI].
  19. 1 2 3 Bavaresco, Anna; Bernardi, Raffaella; Bertolazzi, Leonardo; et al. (2025). LLMs instead of Human Judges? A Large Scale Empirical Study across 20 NLP Evaluation Tasks. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). pp. 238–255. arXiv:2406.18403. doi:10.18653/v1/2025.acl-short.20.
  20. 1 2 Tan, Sijun; Zhuang, Siyuan; Montgomery, Kyle; et al. (2025). JudgeBench: A Benchmark for Evaluating LLM-Based Judges. The Thirteenth International Conference on Learning Representations (ICLR). arXiv:2410.12784.
  21. ↑ Zheng, Xiaosen; Pang, Tianyu; Du, Chao; et al. (2025). Cheating Automatic LLM Benchmarks: Null Models Achieve High Win Rates. The Thirteenth International Conference on Learning Representations (ICLR). arXiv:2410.07137.
  22. ↑ Dubois, Yann; Liang, Percy; Hashimoto, Tatsunori (2024). Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluators. First Conference on Language Modeling (COLM). arXiv:2404.04475.
  23. ↑ Calderon, Nitay; Reichart, Roi; Dror, Rotem (2025). The Alternative Annotator Test for LLM-as-a-Judge: How to Statistically Justify Replacing Human Annotators with LLMs. Proceedings of the Association for Computational Linguistics (ACL). arXiv:2501.10970.
  24. ↑ Singh, Shivalika; Nan, Yiyang; Wang, Alex; et al. (2025). The Leaderboard Illusion. Advances in Neural Information Processing Systems 38 (NeurIPS 2025), Datasets and Benchmarks Track. arXiv:2504.20879.
  25. ↑ Inan, Hakan; Upasani, Kartikeya; Chi, Jianfeng; et al. (2023). "Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations". arXiv:2312.06674 [cs.CL].
  26. ↑ Es, Shahul; James, Jithin; Espinosa-Anke, Luis; Schockaert, Steven (2024). RAGAs: Automated Evaluation of Retrieval Augmented Generation. Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations. pp. 150–158. arXiv:2309.15217. doi:10.18653/v1/2024.eacl-demo.16.
  27. ↑ Lambert, Nathan; Pyatkin, Valentina; Morrison, Jacob; et al. (2025). RewardBench: Evaluating Reward Models for Language Modeling. Findings of the Association for Computational Linguistics: NAACL 2025. pp. 1755–1797. arXiv:2403.13787. doi:10.18653/v1/2025.findings-naacl.96.
[edit]
Retrieved from "https://en.wikipedia.org/w/index.php?title=LLM-as-a-Judge&oldid=1364115937"