A study shows that AI models accept convincingly presented false medical information, putting health safety at risk.

A study published in The Lancet Digital Health highlights a major problem related to the use of artificial intelligence (AI) in the healthcare field: AI language models can accept and repeat false medical information when presented in credible language. Researchers analyzed over a million requests across 20 language models, including ChatGPT and Llama, to determine whether these models contest or repeat convincingly formulated false medical claims. The results showed that 32% of cases were accepted as valid information, with significant variations between models. Smaller models accepted false information in over 60% of cases, while advanced models, such as GPT-4, had an acceptance rate of about 10%. A notable aspect was that models specialized for medical use performed worse than some general models. The study suggests that the way information is formulated may influence its acceptance more than its truthfulness. Additionally, the models were influenced by faulty logical arguments, such as appeals to authority, which increased the acceptance rate of false information.