The AI chatbot ChatGPT could be better at following treatment standards for depression than human doctors, a study suggests.
The technology could improve decision making in primary care, researchers said, as it is capable of following recognised treatment standards without any gender or social class biases that are sometimes a factor between humans.
But more work is needed to assess any potential risks or ethical issues that could stem from its use in practice, the researchers added.
A team in Israel gave two versions of ChatGPT – 3.5 and 4 – brief descriptions of hypothetical patients showing symptoms of depression during initial consultations.
There were eight distinct characters which varied by gender, socioeconomic status and depression severity.
Symptoms included sadness, problems sleeping and loss of appetite in the three weeks leading up to the appointment, as well as a diagnosis of mild to moderate depression.
The information about each hypothetical patient was fed into ChatGPT 10 times and its answers were compared to 1,249 French primary care doctors, 73% of whom were women.
For mild depression, ChatGPT-3.5 recommended psychotherapy in 95% of cases and ChatGPT-4 in 97.5% of cases.
Primary care doctors however recommended it in only 4.3% of cases, opting for drugs 48% of the time, or psychotherapy plus prescribed drugs 32.5% of the time.
For severe cases of depression, 44.5% of doctors recommended psychotherapy plus prescribed drugs, while the two versions of ChatGPT recommended this method in 72% and 100% of cases respectively.
When it came to the type of medicine that was recommended, ChatGPT favoured exclusive use of antidepressants in 74% and 68% of cases, while human doctors leaned towards a mix of antidepressants and anxiolytics/hypnotics in 67.4% of cases.
Read more:
ChatGPT creator expresses concern about ‘under-regulation’
British judge admits using ‘jolly useful’ ChatGPT to write ruling
Please use Chrome browser for a more accessible video player
Is AI an existential threat?
ChatGPT ‘has potential to enhance decision making in primary healthcare’
The researchers said their findings, published in the journal Family Medicine and Community Health, showed ChatGPT “aligned well with accepted guidelines for managing mild and severe depression, without showing the gender or socioeconomic biases observed among primary care physicians”.
They added: “ChatGPT-4 demonstrated greater precision in adjusting treatment to comply with clinical guidelines.
“The study suggests that ChatGPT… has the potential to enhance decision making in primary healthcare.”
But they said despite the potential benefits of using AI chatbots such as ChatGPT, “further research is needed to refine AI recommendations for severe cases and to consider potential risks and ethical issues”.
