A study conducted by Mass General Brigham in Massachusetts, published in Jama Network Open, highlighted that common chatbots, such as ChatGPT and Gemini, make incorrect diagnoses in over 80% of cases when they do not have sufficient information about patients. The study tested 21 language models, including those developed by OpenAI, Anthropic, Google, xAI, and DeepSeek, using 29 clinical vignettes based on reference medical texts. Even when they had access to complete information, the chatbots had an error rate of over 40%, although in some cases they managed to provide the correct diagnosis for 90% of patients. The conclusion of the experiments is that the performance of these models significantly depends on the volume of available information, and hallucinations, that is, the invention of information, remain a major problem in generating correct responses.
Sources
Latest News
11:00
Brad Pitt announces that filming for “Ocean’s 14” could begin toward the end of next year
10:49
Florin Manole supports indexing pensions by 7.3% in 2027 despite budget constraints
10:46
Protest at Nikolaus Lenau High School in Timișoara: Students refuse to attend classes after a knife incident
10:42
Two trains collided head-on in Prague / Dozens of people were injured
10:38
Jorge Linares, former world boxing champion, was detained by ICE in Texas
See more news