A study published in Nature has found that agreement among large language models (LLMs) does not serve as a reliable indicator of medical accuracy.
Researchers investigating the use of artificial intelligence for verifying health information discovered that when multiple models reach a consensus, it often fails to track medical truth, a phenomenon the study authors term a “consensus mirage.”
The evaluation involved 10 different LLMs, including frontier, medical-specialized, and Russian-specific families. These models were tested against RuHealth, a corpus consisting of 108 health claims that had been verified through a multi-stage physician review process. The claims included 40 true statements, 40 false statements, and 28 disputed ones. To ensure comprehensive testing, the models assessed these claims in both Russian and English across various decoding temperatures and multiple stochastic runs, totaling approximately 90,000 individual judgments.
According to the findings, while frontier models demonstrated the highest levels of accuracy, none of the tested systems proved reliable. The models struggled significantly with identification, systematically missing false claims. Furthermore, the researchers said that the category of “disputed” was assigned to 59.8% of judgments, a figure that far exceeded the 26% true base rate of the dataset.
The study also examined the effectiveness of majority voting as a verification strategy. The data showed that even when three frontier models reached a unanimous decision on 90 of the 108 claims, 31% of those unanimous verdicts were incorrect.
Additionally, the models frequently misclassified claims, with 26 of the 28 claims labeled as “disputed” actually being either true or false. The researchers concluded that this lack of correlation between consensus and correctness persists regardless of language or the specific wording of prompts.
