{"id":550559,"date":"2026-02-02T19:30:27","date_gmt":"2026-02-02T19:30:27","guid":{"rendered":"https:\/\/Blockchain.News\/news\/open-source-llm-judges-beat-gpt-5-2-dpo-fine-tuning"},"modified":"2026-02-02T19:30:27","modified_gmt":"2026-02-02T19:30:27","slug":"open-source-ai-judges-beat-gpt-5-2-at-15x-lower-cost-using-dpo-fine-tuning","status":"publish","type":"post","link":"https:\/\/e-bitco.in\/index.php\/2026\/02\/02\/open-source-ai-judges-beat-gpt-5-2-at-15x-lower-cost-using-dpo-fine-tuning\/","title":{"rendered":"Open-Source AI Judges Beat GPT-5.2 at 15x Lower Cost Using DPO Fine-Tuning"},"content":{"rendered":"<figure class=\"figure mt-2\">\n<p> <a href=\"https:\/\/blockchain.news\/Profile\/Luisa-Crawford\">Luisa Crawford<\/a> <span class=\"publication-date ml-2\"> Feb 02, 2026 19:30<\/span> <\/p>\n<p class=\"lead\">Together AI demonstrates fine-tuned open-source LLMs can outperform GPT-5.2 as evaluation judges using just 5,400 preference pairs, slashing costs dramatically.<\/p>\n<p> <a href=\"https:\/\/image.blockchain.news:443\/features\/DC3788979712BF4DFF603597AAC46E7C52F8B5EF76BC21453D757F37CDB271FE.jpg\"> <img decoding=\"async\" class=\"rounded\" src=\"https:\/\/image.blockchain.news:443\/features\/DC3788979712BF4DFF603597AAC46E7C52F8B5EF76BC21453D757F37CDB271FE.jpg\" alt=\"Open-Source AI Judges Beat GPT-5.2 at 15x Lower Cost Using DPO Fine-Tuning\"> <\/a> <\/figure>\n<p>Fine-tuned open-source large language models can now outperform OpenAI&#8217;s GPT-5.2 at evaluating AI outputs\u2014at a fraction of the cost. Together AI released research showing their GPT-OSS 120B model achieved 62.63% accuracy on human preference alignment after Direct Preference Optimization training, surpassing GPT-5.2&#8217;s 61.62% baseline while running 14x faster and costing 15x less per token.<\/p>\n<p>The findings matter for any organization running AI evaluation pipelines at scale. GPT-5.2 currently charges $1.75 per million input tokens and $14 per million output tokens. The fine-tuned GPT-OSS 120B? Just $0.15 and $0.60 respectively.<\/p>\n<h2>The Training Approach<\/h2>\n<p>Together AI used DPO, a technique introduced in late 2023 that bypasses the complex reinforcement learning loops of traditional RLHF. Instead of training a separate reward model, DPO directly adjusts the language model&#8217;s weights using preference pairs\u2014one preferred response, one rejected response for each prompt.<\/p>\n<p>The training data came from RewardBench 2, a benchmark containing examples with human-labeled preferred and rejected responses across six categories: safety, factuality, math, precise instruction following, focus, and ties. From roughly 1,500 training examples, the team generated 5,407 preference pairs.<\/p>\n<p>Training took just 1.5 hours for GPT-OSS 120B using LoRA (Low-Rank Adaptation) with a learning rate of 5e-6 over three epochs.<\/p>\n<h2>Where Open Models Excel<\/h2>\n<p>The category-level breakdown reveals where fine-tuning delivered the biggest wins. GPT-OSS 120B after DPO beat GPT-5.2 on math evaluation by 10.3 percentage points and on focus (response quality assessment) by 6.3 points.<\/p>\n<p>Safety evaluation proved easiest across all models, averaging 91.32% accuracy\u2014unsurprising given these models undergo extensive safety training. Factuality detection hit 85.23%. The hardest category? Focus, where models averaged just 10.13% accuracy, highlighting how subjective quality judgments remain challenging.<\/p>\n<p>One wrinkle: Qwen3 235B, which already beat GPT-5.2 out of the box at 62.63%, actually regressed slightly to 61.28% after fine-tuning. Not every model benefits from additional training, reinforcing that validation remains essential.<\/p>\n<h2>The Broader Implications<\/h2>\n<p>The &#8220;LLM-as-a-judge&#8221; paradigm has become standard for evaluating AI outputs at scale because judging is fundamentally simpler than generating. A model generating a response must juggle context, follow multi-step instructions, and synthesize information. Evaluating that response is a focused classification task.<\/p>\n<p>This research suggests organizations can build evaluation pipelines using open-source models they control entirely\u2014no API dependencies, full visibility into model behavior, and the ability to fine-tune for specific domains. The cost savings at production scale are substantial.<\/p>\n<p>Together AI published the full methodology in a <a rel=\"nofollow\" href=\"https:\/\/github.com\/togethercomputer\/together-cookbook\/blob\/main\/Evals\/Optimizing_LLM_Judges.ipynb\">cookbook notebook<\/a> for teams wanting to replicate the approach with their own preference data.<\/p>\n<p><span><i>Image source: Shutterstock<\/i><\/span> <!-- Divider --> <!-- Author info END --> <!-- Divider --> <a href=\"https:\/\/blockchain.news\/\">Source<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Luisa Crawford Feb 02, 2026 19:30 Together AI demonstrates fine-tuned open-source LLMs can outperform GPT-5.2 as evaluation judges using just 5,400 preference pairs, slashing costs dramatically. Fine-tuned open-source large language models can now outperform OpenAI&#8217;s GPT-5.2 at evaluating AI outputs\u2014at a fraction of the cost. Together AI released research showing their GPT-OSS 120B model achieved [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":550560,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[12],"tags":[1129,17401,16487,2572,25,2747],"class_list":{"0":"post-550559","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-blockchain","8":"tag-ai","9":"tag-dpo","10":"tag-llm","11":"tag-machine-learning","12":"tag-news","13":"tag-open-source"},"_links":{"self":[{"href":"https:\/\/e-bitco.in\/index.php\/wp-json\/wp\/v2\/posts\/550559","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/e-bitco.in\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/e-bitco.in\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/e-bitco.in\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/e-bitco.in\/index.php\/wp-json\/wp\/v2\/comments?post=550559"}],"version-history":[{"count":0,"href":"https:\/\/e-bitco.in\/index.php\/wp-json\/wp\/v2\/posts\/550559\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/e-bitco.in\/index.php\/wp-json\/wp\/v2\/media\/550560"}],"wp:attachment":[{"href":"https:\/\/e-bitco.in\/index.php\/wp-json\/wp\/v2\/media?parent=550559"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/e-bitco.in\/index.php\/wp-json\/wp\/v2\/categories?post=550559"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/e-bitco.in\/index.php\/wp-json\/wp\/v2\/tags?post=550559"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}