Grant: Discovery Grants Program - Individual
York University
$31,000.00
Mitigating the unfairness of AI-based medical image diagnostic tools
FY 2024-25
Summary
Significant efforts are underway to develop artificial intelligence (AI) tools that are trained to yield diagnostic recommendations on medical images (e.g., X-rays, MRI, CT). Current AI tools have reached performance on par with human radiologists making them viable for real-world deployment. However, performance is often reported for a general population and not specific demographics. My research has shown that AI models can unintentionally be biased for particular populations - for example, disproportionately misreporting Black patients as healthy (vs. White patients), leading to a denial of critical care for an already underserved community. Such unfairness in AI-based medical imaging is a critical issue that must be understood and addressed before large-scale adoption.
My research program's long-term goal targets mitigating the unfairness (debiasing) of AI medical image diagnostic tools to maintain high performance while being fair for all patients regardless of their membership to different subpopulations of socially sensitive attributes such as race, sex, socioeconomic status, or non-social factors (sensitive attributes) such as disease-dependent (e.g. disease appearance), instrumental (e.g. imaging devices), and environmental (e.g. data sources) factors. In that scope, we divide the program into three main short-term objectives, understanding the root causes of unfairness to remove them, mitigating the unfairness algorithmically without necessarily knowing the root causes, rethinking the fairness of the AI models, and standardizing the reporting scheme.
Impact: Our research program proposes considerable advancement in bias mitigation in AI medical image diagnostic tools, a severe barrier in AI model deployment in radiology. Our goal is to provide methods, algorithms, and insight to ensure AI medical image diagnostic tools reach high performance for all patients. Our program also provides awareness of AI models' strengths and weaknesses and guides policymakers and developers to standardize AI algorithms' fairness checks.
Innovation for Canada: There is a massive public health insurance cost for the government of Canada, burnout in Canadian radiology, and geographical disparities in access to care. AI in radiology is a potential solution to reduce healthcare costs, reduce radiologist workload, and increase healthcare access in a broader community if deployed fairly. There are many startup companies in Toronto focusing on AI and medical imaging. Hospitals actively do research to deploy AI in the diagnostic imaging pipeline. High-tech companies at the international level have invested tremendously in AI & radiology. Notably, in Canada and Ontario, several companies work on developing medical image diagnostic tools. Successfully delivering my program's targets and mitigating the proposed unfairness is vital for AI in radiology products of these industries and provides international competitive advantages for the Canadian industry.
Program
Discovery Grants Program - Individual
Recipient
Location
Ontario, Canada
Recipient Type
Academia
Type
Grant
Source
NSERC's Awards Data
As published
Application id
RGPIN-2023-05094
Name nom
Seyyed-Kalantari, Laleh
Department département
Electrical Engineering and Computer Science
Organization id
38
Institution établissement
York University
Province en
Ontario
Province fr
Ontario
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2024
Competition year année de concours
2023
Award amount
31000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
1507
Committee name en
Computer Science
Committee name fr
Sciences informatiques
Area of application code
1102
Area of application group en
Health, education and social services
Area of application group fr
Santé publique, éducation et services sociaux
Area of application en
Biomedical engineering
Area of application fr
Génie biomédical
Research subject code
2800
Research subject group en
Artificial intelligence
Research subject group fr
Intelligence artificielle
Research subject en
Artificial intelligence (computer vision, use2603)
Research subject fr
Intelligence artificielle (vision artificielle, utiliser 2603)
Field of research list names en
Not available
Field of research list names fr
Non disponible
Socioeconomic objective list names en
Not available
Socioeconomic objective list names fr
Non disponible
Application title
Mitigating the unfairness of AI-based medical image diagnostic tools
Keywords
artificial intelligence; medical imaging; fairness; bias; mitigating unfairness; decision making; medical image; trustworthy ai; ai diagnostic tools; ai and ethic
Application summary
Significant efforts are underway to develop artificial intelligence (AI) tools that are trained to yield diagnostic recommendations on medical images (e.g., X-rays, MRI, CT). Current AI tools have reached performance on par with human radiologists making them viable for real-world deployment. However, performance is often reported for a general population and not specific demographics. My research has shown that AI models can unintentionally be biased for particular populations - for example, disproportionately misreporting Black patients as healthy (vs. White patients), leading to a denial of critical care for an already underserved community. Such unfairness in AI-based medical imaging is a critical issue that must be understood and addressed before large-scale adoption.
My research program's long-term goal targets mitigating the unfairness (debiasing) of AI medical image diagnostic tools to maintain high performance while being fair for all patients regardless of their membership to different subpopulations of socially sensitive attributes such as race, sex, socioeconomic status, or non-social factors (sensitive attributes) such as disease-dependent (e.g. disease appearance), instrumental (e.g. imaging devices), and environmental (e.g. data sources) factors. In that scope, we divide the program into three main short-term objectives, understanding the root causes of unfairness to remove them, mitigating the unfairness algorithmically without necessarily knowing the root causes, rethinking the fairness of the AI models, and standardizing the reporting scheme.
Impact: Our research program proposes considerable advancement in bias mitigation in AI medical image diagnostic tools, a severe barrier in AI model deployment in radiology. Our goal is to provide methods, algorithms, and insight to ensure AI medical image diagnostic tools reach high performance for all patients. Our program also provides awareness of AI models' strengths and weaknesses and guides policymakers and developers to standardize AI algorithms' fairness checks.
Innovation for Canada: There is a massive public health insurance cost for the government of Canada, burnout in Canadian radiology, and geographical disparities in access to care. AI in radiology is a potential solution to reduce healthcare costs, reduce radiologist workload, and increase healthcare access in a broader community if deployed fairly. There are many startup companies in Toronto focusing on AI and medical imaging. Hospitals actively do research to deploy AI in the diagnostic imaging pipeline. High-tech companies at the international level have invested tremendously in AI & radiology. Notably, in Canada and Ontario, several companies work on developing medical image diagnostic tools. Successfully delivering my program's targets and mitigating the proposed unfairness is vital for AI in radiology products of these industries and provides international competitive advantages for the Canadian industry.