General-purpose large language models outperform specialized clinical AI tools on medical benchmarks.

Comparative evaluation of general-purpose LLMs (GPT-5.2, Gemini 3.1 Pro, Claude Opus 4.6) vs specialized clinical AI tools on medical benchmarks

This Nature Medicine study quantitatively evaluates two specialized clinical AI tools (OpenEvidence and UpToDate Expert AI) against three frontier general-purpose LLMs (GPT-5.2, Gemini 3.1 Pro, and Claude Opus 4.6) on standardized medical benchmarks including 500 MedQA questions. General-purpose LLMs outperform specialized clinical AI tools across all evaluated dimensions, questioning the added value of clinical model specialization.

Synthesis written by Geno'X. For the full original abstract, please refer to the source publication.

A counterintuitive and important finding: general-purpose LLMs outperform specialized clinical AI tools built on these same LLMs. This raises questions about the real value of clinical specialization layers added by vendors — and about the criteria to use when selecting an AI tool in medical practice.

Clinical impact: 2/3 · Evidence strength: 2/3 · Novelty: 1/2 · Sample size: 1/1 · Publication status: 1/1 → Total: 7/10