FDA highlights the need to address bias in AI – Healthcare IT News
The U.S. Food and Drug Administration on Thursday convened a public meeting of its Patient Engagement Advisory Committee to discuss issues regarding artificial intelligence and machine learning in medical devices.
"Devices using AI and ML technology will transform healthcare delivery by increasing efficiency in key processes in the treatment of patients," said Dr. Paul Conway, PEAC chair and chair of policy and global affairs of the American Association of Kidney Patients.
As Conway and others noted during the panel, AI and ML systems may have algorithmic biases and lack transparency potentially leading, in turn, to an undermining of patient trust in devices.
Medical device innovation has already ramped up in response to the COVID-19 crisis, with Center for Devices and Radiological Health Director Dr. Jeff Shuren noting that 562 medical devices have already been granted emergency use authorization by the FDA.
It's imperative, said Shuren, that patients' needs be considered as part of the creation process.
"We continue to encourage all members of the healthcare ecosystem to strive to understand patients' perspective and proactively incorporate them into medical device development, modification and evaluation," said Shuren. "Patients are truly the inspiration for all the work we do."
"Despite the global challenges with the COVID-19 public health emergency ... the patient's voice won't be stopped," Shuren added. "And if anything, there is even more reason for it to be heard."
However, said Pat Baird, regulatory head of global software standards at Philips, facilitating patient trust also means acknowledging the importance of robust and accurate data sets.
"To help support ourpatients, we need to become more familiar with them, their medical conditions, their environment, and their needs and wantsto be able to better understand the potentially confounding factors that drive some of the trends in the collected data," said Baird.
"An algorithm trained on one subset of the population might not be relevant for a different subset," Baird explained.
For instance, if a hospital needed a device that would serve its population of seniors at a Florida retirement community, an algorithm trained on recognizing healthcare needs of teens in Maine would not be effective.Not every population will have the same needs.
"This bias in the data is not intentional, but can be hard to identify," he continued. He encouraged the development of a taxonomy of bias types that would be made publicly available.
Ultimately, he said, people won't use what they don't trust. "We need to use our collective intelligence to help produce better artificial intelligence populations," he said.
Captain Terri Cornelison, chief medical officer and director for thehealth of women at CDRH, noted that demographic identifiers can be medically significant due to genetics and social determinants of health, among other factors.
"Science is showing us that these are not just categorical identifiers but actually clinically relevant," Cornelison said.
She pointed out that a clinical study that does not identify patients' sex may mask different results for people with different chromosomes.
"In many instances, AI and ML devices may be learning a worldview that is narrow in focus, particularly in the available training data, if the available training data do not represent a diverse set of patients," she said.
"More simply, AI and ML algorithms may not represent you if the data do not include you," she said.
"Advances in artificial intelligence are transforming our health systems and daily lives," Cornelison continued. "Yet despite these significant achievements, most ignore the sex, gender, age, race [and] ethnicity dimensions and their contributions to health and disease differences among individuals."
The committee also examined how informed consent might play a role in algorithmic training.
"If I give my consent to be treated by an AI/ML device, I have the right to know whether there were patients like me ... in the data set," said Bennet Dunlap, a health communications consultant. "I think the FDA should not be accepting or approving a medical device that does not have patient engagement" of the kind outlined in committee meetings, he continued.
"You need to know what your data is going to be used for," he reiterated. "I have white privilege. I can just assume old white guys are in [the data sets]. That's where everybody starts. But that should not be the case."
Dr. Monica Parker, assistant professor in neurology and education core member of the Goizueta Alzheimers Disease Research Center at Emory University, pointed out that diversifying patient data requires turning to trusted entities within communities.
"If people are developing these devices, in the interest of being more broadly diverse, is there some question about where these things were tested?" She raised the issue of testing taking place in academic medical centers or technology centers on the East or West Coast, versus "real-world data collection from hospitals that may be using some variation of the device for disease process.
"Clinicians who are serving the population for which the device is needed" provide accountability and give the device developer a better sense of whothey're treating, Parker said. She also reminded fellow committee members that members of different demographic groups are not uniform.
Philip Rutherford, director of operation at Faces and Voices Recovery, pointed out that it's not just enough to prioritize diversity in data sets.The people in charge of training the algorithm must also not be homogenous.
"If we want diversity in our data, we have to seek diversity in the people that are collecting the data," said Rutherford.
The committee called on the FDA to take a strong role in addressing algorithmic bias in artificial intelligence and machine learning.
"At the end of the day, diversity validation and unconscious bias all these things can be addressed if there's strong leadership from the start," said Conway.
Kat Jercich is senior editor of Healthcare IT News.Twitter: @kjercichEmail: kjercich@himss.orgHealthcare IT News is a HIMSS Media publication.
Read the original here:
FDA highlights the need to address bias in AI - Healthcare IT News
- Senate bill targets the use of artificial intelligence in Mississippi - WAPT Jackson - March 15th, 2025 [March 15th, 2025]
- will.i.am and Sean Paul on embracing artificial intelligence in music - Sky News - March 15th, 2025 [March 15th, 2025]
- As Artificial Intelligence (AI) Stocks Fall, Investors Should Consider Buying More of These 2 Powerhouses - The Motley Fool - March 15th, 2025 [March 15th, 2025]
- FSUs Graduate School hosts workshop on artificial intelligence to explore the impact of AI on teaching and research - Florida State News - March 15th, 2025 [March 15th, 2025]
- Artificial Intelligence Optimization (AIO): The Next Frontier in SEO - hackernoon.com - March 15th, 2025 [March 15th, 2025]
- The U.S. Army, Artificial Intelligence, and Mission Command - War On The Rocks - March 15th, 2025 [March 15th, 2025]
- 2 Stock-Split Artificial Intelligence (AI) Stocks to Buy Before They Zoom Higher, According to Wall Street Analysts - The Motley Fool - March 15th, 2025 [March 15th, 2025]
- AI in the classroom: How artificial intelligence is helping teachers - Bay News 9 - March 15th, 2025 [March 15th, 2025]
- Video: Daryl Morey Says 76ers Use Artificial Intelligence for Input on Team Decisions - Bleacher Report - March 15th, 2025 [March 15th, 2025]
- Better Artificial Intelligence Stock: AT&T vs. Verizon Communications - The Motley Fool - March 15th, 2025 [March 15th, 2025]
- NVIDIA GTC 25: What To Expect From the Years Biggest Artificial Intelligence Event - BizTech Magazine - March 15th, 2025 [March 15th, 2025]
- 2 Stock-Split Artificial Intelligence (AI) Stocks to Buy Before They Zoom Higher, According to Wall Street Analysts - MSN - March 15th, 2025 [March 15th, 2025]
- DEI Erosion Threatens Safety. Artificial Intelligence May Help Pick Up the Slack. - Inbound Logistics - March 15th, 2025 [March 15th, 2025]
- Saint Vincent conference will examine broad impact of artificial intelligence - TribLIVE - March 15th, 2025 [March 15th, 2025]
- As Artificial Intelligence (AI) Stocks Fall, Investors Should Consider Buying More of These 2 Powerhouses - MSN - March 15th, 2025 [March 15th, 2025]
- 3 Artificial Intelligence (AI) Stocks You Can Buy and Hold for the Next Decade - The Motley Fool - March 15th, 2025 [March 15th, 2025]
- This Artificial Intelligence (AI) Stock Has Surged 1,000% Over the Past 2 Years. Is a Split on the Horizon? - The Motley Fool - March 15th, 2025 [March 15th, 2025]
- Move Over, Artificial Intelligence (AI) -- Businesses Are on Pace to Spend More Than $10 Trillion on This Trend Over the Next Decade - The Motley Fool - March 15th, 2025 [March 15th, 2025]
- Here Is My Top Artificial Intelligence (AI) Stock to Buy Right Now - The Motley Fool - March 15th, 2025 [March 15th, 2025]
- Nasdaq Sell-Off: 2 Artificial Intelligence (AI) Stocks Down 20% and 49% to Buy Hand Over Fist on the Dip - The Motley Fool - March 15th, 2025 [March 15th, 2025]
- 1 Top Artificial Intelligence (AI) Chip Stock to Buy Hand Over Fist Before It Jumps - The Motley Fool - March 15th, 2025 [March 15th, 2025]
- [Latest] How Artificial Intelligence is Shaping the Emc Shielding And Test Equipment Market - openPR - March 15th, 2025 [March 15th, 2025]
- Nasdaq Correction: 5 Reasons Nvidia Stock Is Still a Top Artificial Intelligence (AI) Stock to Buy Right Now - The Motley Fool - March 15th, 2025 [March 15th, 2025]
- Artificial Intelligence (AI) Adoption Rates Appear Low, but This Technology Leader Could See a Massive Surge in Demand in the Next 10 Years. Here's... - March 15th, 2025 [March 15th, 2025]
- [Latest] The Influence of Artificial Intelligence on the Augmented Reality (Ar) Shopping Market - openPR - March 15th, 2025 [March 15th, 2025]
- Oracle Just Shared Some Amazing News for the Entire Artificial Intelligence (AI) Industry - The Motley Fool - March 15th, 2025 [March 15th, 2025]
- 1 No-Brainer Artificial Intelligence (AI) ETF to Buy With $40 During the Nasdaq Sell-Off - The Motley Fool - March 15th, 2025 [March 15th, 2025]
- Inside The Artificial Intelligence That Can Clone Your Mind - Forbes - March 15th, 2025 [March 15th, 2025]
- Why Tesla Could Be a Massive Winner in the Artificial Intelligence (AI) Race - The Motley Fool - March 15th, 2025 [March 15th, 2025]
- 4 Artificial Intelligence (AI) Stocks Worth Buying in the Tech Sell-Off - The Motley Fool - March 15th, 2025 [March 15th, 2025]
- Impact of Artificial Intelligence (AI) on Media and Creative Industries - EDMO - March 1st, 2025 [March 1st, 2025]
- 1 Artificial Intelligence (AI) Stock That Could Be Bigger Than Nvidia in 5 Years - The Motley Fool - March 1st, 2025 [March 1st, 2025]
- This Artificial Intelligence (AI) Stock Is Up 15% in 2025 Already. It Is Still a Solid Buy? - The Motley Fool - March 1st, 2025 [March 1st, 2025]
- Can artificial intelligence be the future solution to the enormous challenges and suffering caused by Schizophrenia? - Nature.com - March 1st, 2025 [March 1st, 2025]
- Applications of Artificial Intelligence in Medical Education: A Systematic Review - Cureus - March 1st, 2025 [March 1st, 2025]
- This Artificial Intelligence (AI) Stock Is Up 15% in 2025 Already. It Is Still a Solid Buy? - AOL - March 1st, 2025 [March 1st, 2025]
- Federal Executive Forum Artificial Intelligence Strategies in Government Progress and Best Practices 2025 - Federal News Network - March 1st, 2025 [March 1st, 2025]
- Introduction to Artificial Intelligence for General Surgeons: A Narrative Review - Cureus - March 1st, 2025 [March 1st, 2025]
- How is Artificial Intelligence Affecting Health Care? - Workers Comp Forum - March 1st, 2025 [March 1st, 2025]
- 1 Spectacular Artificial Intelligence (AI) Stock to Buy With $50 Right Now - The Motley Fool - March 1st, 2025 [March 1st, 2025]
- 20+ Advantages and Disadvantages of AI | Pros of Artificial Intelligence - Simplilearn - March 1st, 2025 [March 1st, 2025]
- Prediction: This Top Artificial Intelligence (AI) Stock Will Start Skyrocketing After March 6 - The Motley Fool - March 1st, 2025 [March 1st, 2025]
- 1 Surprising Stock Harnessing the Power of Artificial Intelligence (AI) - The Motley Fool - March 1st, 2025 [March 1st, 2025]
- 2 Top Artificial Intelligence (AI) Stocks to Buy On the Dip Amid Nasdaq Selloff - Yahoo Finance - March 1st, 2025 [March 1st, 2025]
- Review: Artificial intelligence is shaping the future of diabetes care - News-Medical.Net - March 1st, 2025 [March 1st, 2025]
- Prediction: This Artificial Intelligence (AI) Stock -- a 1,020% Gainer Since Its IPO -- Won't Split Its Stock in 2025. Here's Why - The Motley Fool - March 1st, 2025 [March 1st, 2025]
- A Nobel laureate on the economics of artificial intelligence - MIT Technology Review - March 1st, 2025 [March 1st, 2025]
- Prediction: This Top Artificial Intelligence (AI) Stock Will Start Skyrocketing After March 6 - Nasdaq - March 1st, 2025 [March 1st, 2025]
- Meta Platforms Just Caused This Crucial Artificial Intelligence (AI) Stock to Plummet. Should You Buy the Dip? - The Motley Fool - March 1st, 2025 [March 1st, 2025]
- Artificial Intelligence - AI Update, February 28, 2025: AI News and Views From the Past Week - MarketingProfs.com - March 1st, 2025 [March 1st, 2025]
- The Ultimate Artificial Intelligence (AI) ETF to Buy With $50 Right Now - The Motley Fool - March 1st, 2025 [March 1st, 2025]
- Prediction: This Artificial Intelligence (AI) Company Will Split Its Stock in 2025 - The Motley Fool - March 1st, 2025 [March 1st, 2025]
- Should You Forget Nvidia and Buy 2 Artificial Intelligence (AI) Stocks Instead? - The Motley Fool - March 1st, 2025 [March 1st, 2025]
- Why Artificial Intelligence Stocks SoundHound AI, IonQ, and C3.ai Are Struggling Today - The Motley Fool - March 1st, 2025 [March 1st, 2025]
- 2 Top Artificial Intelligence (AI) Stocks Ready for a Bull Run - The Motley Fool - March 1st, 2025 [March 1st, 2025]
- AI Cant Do This Anymore: The Dangers of Artificial Intelligence in Academia - Skidmore News - March 1st, 2025 [March 1st, 2025]
- Whats Next in Artificial Intelligence: Agents that can do more than chatbots - Pittsburgh Post-Gazette - February 9th, 2025 [February 9th, 2025]
- Geopolitics of artificial intelligence to be focus of major summit in Paris; AP explains - Yahoo - February 9th, 2025 [February 9th, 2025]
- Geopolitics of artificial intelligence to be focus of major summit in Paris; AP explains - The Associated Press - February 9th, 2025 [February 9th, 2025]
- 3 Top Artificial Intelligence Stocks to Buy in February - MSN - February 9th, 2025 [February 9th, 2025]
- Geopolitics of artificial intelligence to be focus of major summit in Paris; AP explains - Lufkin Daily News - February 9th, 2025 [February 9th, 2025]
- 2 of the Hottest Artificial Intelligence (AI) Stocks on the Planet Can Plunge Up to 94%, According to Select Wall Street Analysts - The Motley Fool - February 9th, 2025 [February 9th, 2025]
- These 2 Stocks Are Leading the Data Center Artificial Intelligence (AI) Trend, but Are They Buys Right Now? - The Motley Fool - February 9th, 2025 [February 9th, 2025]
- Book Review | Genesis: Artificial Intelligence, Hope, and the Human Spirit - LSE - February 9th, 2025 [February 9th, 2025]
- The Artificial Intelligence Action Summit In France: Maintaining The Dialogue On Global AI Regulation - Forrester - February 9th, 2025 [February 9th, 2025]
- Is prediction the next frontier for artificial intelligence? - Healthcare IT News - February 9th, 2025 [February 9th, 2025]
- The Artificial Intelligence in Medicines Market Is Set to Reach $18,119 Million | CAGR of 49.6% - openPR - February 9th, 2025 [February 9th, 2025]
- Geopolitics of artificial intelligence to be focus of major summit in Paris; AP explains - The Audubon County Advocate Journal - February 9th, 2025 [February 9th, 2025]
- Around and About with Richard McCarthy: Asking AI about itself: Will artificial intelligence ever surpass humankind? - GazetteNET - February 9th, 2025 [February 9th, 2025]
- Will the Paris artificial intelligence summit set a unified approach to AI governanceor just be another conference? - Bulletin of the Atomic... - February 9th, 2025 [February 9th, 2025]
- Apple Stock Jumps on Artificial Intelligence (AI) Driving iPhone Sales. Here's Why It's Not Getting Crushed by the DeepSeek Launch. - The Motley Fool - February 9th, 2025 [February 9th, 2025]
- Who will win the race to Artificial General Intelligence? - The Indian Express - February 9th, 2025 [February 9th, 2025]
- Prediction: This Artificial Intelligence (AI) Chip Stock Will Win Big From DeepSeek's Feat - The Motley Fool - February 9th, 2025 [February 9th, 2025]
- Prediction: 2 Artificial Intelligence (AI) Stocks That Will Be Worth More Than Nvidia 3 Years From Now - The Motley Fool - February 9th, 2025 [February 9th, 2025]
- State of Louisiana Launches Innovation Brand, Announces Creation of $50 Million Growth Fund and Artificial Intelligence Research Institute - Louisiana... - February 9th, 2025 [February 9th, 2025]
- Using smart technologies and artificial intelligence in food packaging can reduce food waste - Yahoo News Canada - February 9th, 2025 [February 9th, 2025]
- BigBear.ai Wins Department of Defense Contract to Prototype Near-Peer Adversary Geopolitical Risk Analysis for Chief Digital and Artificial... - February 9th, 2025 [February 9th, 2025]
- Should Investors Change Their Artificial Intelligence (AI) Investment Strategy After the DeepSeek Launch? - The Motley Fool - February 9th, 2025 [February 9th, 2025]
- 1 Unstoppable Artificial Intelligence (AI) Stock to Buy Before It Punches Its Ticket to the $4 Trillion Club - The Motley Fool - February 9th, 2025 [February 9th, 2025]
- Got 10 Years and $1000? These 3 Artificial Intelligence (AI) Stocks Are Set to Soar. - The Motley Fool - February 9th, 2025 [February 9th, 2025]