According to the results of a recent study, ChatGPT beats doctors when it comes to making a diagnosis. The worst part is that the chatbot does better even when practitioners use AI.
Whether we approve of its use or not, it is impossible to deny that artificial intelligence advances medicine. Filled with millions of studies and data, a chatbot has knowledge that no human being can hope to retain over the course of their life. Where this is most evident is on the diagnosis of diseases. Not only is AI capable of identifying pathologies that leave practitioners perplexed, but it can also do so remotely, without invasive examination.
Doctors themselves recognize theusefulness of being accompanied by an AI. So, when Doctor Adam Rodman launches a study to report on these benefits, he is almost sure of the results he will obtain: his colleagues with the help of ChatGPT-4 will make better diagnoses than those who think alone. Not only was this not the case, but what he discovered truly left him speechless.
ChatGPT AI makes better diagnoses than doctors according to this study
50 doctors from American hospitals, both internal and external, are participating in the research. We present to them 6 files from which they must formulate a diagnosiswhile explaining their reasoning. The cases are real and are among 105 used in various studies since the 1990s.
They were of course never published, which means that neither the participants nor ChatGPT could have had prior knowledge of it. Only one of the 6 files was disclosed in advance with examples of correct and other wrong answers, to make it clear what was expected.
Read also – AI detects this disease in children in 100% of cases
The 50 doctors were randomly divided into 2 groups. The first uses ChatGPT to identify the disease, while the second must work without. The capabilities of the chatbot alone were also measured. The conclusions are surprising.
The control group, without AI assistance, obtains a average score of 74%. Practitioners using chatbots do barely better, 76%. As for ChatGPT, it shatters these scores by reaching an average of 90%. How to explain this?
Why does ChatGPT beat doctors in making a diagnosis?
After the initial shock, Doctor Rodman tries to understand what happened. It highlights 2 main points. First of all, doctors do not trust AI if it suggests something that contradicts them. On the contrary, seeing ChatGPT formulate different hypotheses reinforces theirs. Often wrongly as we can see here.
Read also – To detect a fatal disease, this Google AI has an original method
An observation that makes sense for Laura Zwaan, who studies clinical reasoning and diagnostic errors: “People are usually overconfident when they think they are right“. Concerning the other factor explaining the results of the study, it is much more down to earth: many doctors do not know how to use ChatGPT to the best of its possibilities.
Dr. Jonathan H. Chen, one of the authors, notes that the majority use AI as a search engineasking questions like “Is cirrhosis a risk factor for cancer?“. In the end, “only a fraction of doctors realized that they could literally copy and paste the entire file into the chatbot and simply ask it to give a complete answer to the entire question“.
The problem raised is therefore twofold. In addition to train practitioners in the use of artificial intelligence in their workwe will have to succeed in making them accept that the latter can sometimes find an answer that eludes them, despite their years of experience.
Source : The New York Times
