https://www.ft.com/content/128ee880-acdb-42fb-8bc0-ea9b71ca11a8
"Artificial intelligence tools used by doctors risk leading to worse health
outcomes for women and ethnic minorities, as a growing body of research shows
that many large language models downplay the symptoms of these patients.
A series of recent studies have found that the uptake of AI models across the
healthcare sector could lead to biased medical decisions, reinforcing patterns
of under treatment that already exist across different groups in western
societies.
The findings by researchers at leading US and UK universities suggest that
medical AI tools powered by LLMs have a tendency to not reflect the severity of
symptoms among female patients, while also displaying less “empathy” towards
Black and Asian ones.
The warnings come as the world’s top AI groups such as Microsoft, Amazon,
OpenAI and Google rush to develop products that aim to reduce physicians’
workloads and speed up treatment, all in an effort to help overstretched health
systems around the world.
Many hospitals and doctors globally are using LLMs such as Gemini and ChatGPT
as well as AI medical note-taking apps from start-ups including Nabla and Heidi
to auto-generate transcripts of patient visits, highlight medically relevant
details and create clinical summaries.
In June, Microsoft revealed it had built an AI-powered medical tool it claims
is four times more successful than human doctors at diagnosing complex
ailments.
But research by the MIT’s Jameel Clinic in June found that AI models, such as
OpenAI’s GPT-4, Meta’s Llama 3 and Palmyra-Med — a healthcare- focused LLM —
recommended a much lower level of care for female patients, and suggested some
patients self-treat at home instead of seeking help.
A separate study by the MIT team showed that OpenAI’s GPT-4 and other models
also displayed answers that had less compassion towards Black and Asian people
seeking support for mental health problems.
That suggested “some patients could receive much less supportive guidance based
purely on their perceived race by the model”, said Marzyeh Ghassemi, associate
professor at MIT’s Jameel Clinic."
Via Susan ****
Cheers,
*** Xanni ***
--
mailto:xanni@xanadu.net Andrew Pam
http://xanadu.com.au/ Chief Scientist, Xanadu
https://glasswings.com.au/ Partner, Glass Wings
https://sericyb.com.au/ Manager, Serious Cybernetics
