Canada Spends Logo

Explorez notre analyse du budget 2026

Grant: Discovery Grants Program - Individual

University of Toronto

$29,000.00

Human-Centered Algorithm Design for High Stakes Decision-Making in Public Services

EF 2024-25

Résumé
Decades of neoliberal politics in North America centered on austerity and privatization have led to public sector agencies increasingly looking towards predictive algorithms and models built with artificial intelligence and machine learning technologies as a means to reduce costs, improve decision-making processes as well as provide greater efficiencies in public policy and social services delivery. Algorithms in the public sector have generally been adopted in the form of risk assessment algorithms with their primary purpose being the preemptive estimation of 'risk'. This has centered resources towards risk management based on individual client characteristics while driving attention away from structural and societal problems. Over the past two decades, several high-stakes decision-making domains such as the child-welfare system (CWS), criminal justice system, education, and healthcare have increasingly turned towards risk assessment algorithms. For instance, within CWS, there is growing public and media pressure because of the harm caused to children (especially minorities and Indigenous children given Canada's history of residential schools) who are removed from the care of their parents as well as where the system failed to remove and protect a child. However, the public sector poses its own challenges with respect to technical (quality of data), social and cultural (workers' interaction with algorithms), theoretical (what is risk assessment?), and societal (impact of algorithms on communities) implications of algorithmic decision-making. This research program will combine concepts from human-computer interaction (HCI), machine learning (ML) and participatory design (PD) to engage in the design and development of human-centered algorithms for high stakes decision making in the public sector. Particularly, the lens chosen will be through integrating participatory design strategies of relevant stakeholders with machine learning models that leverage historical, unstructured narratives to develop a Participatory Machine Learning (PML) framework. Further, this framework will advance the development of strength-based, holistic assessments that aim to produce positive outcomes for people as opposed to the current norm of narrow, deficit-based risk assessments that measure risk to the government. Finally, the PML framework will be validated in the public sector in two crucial areas of Canadian importance - child welfare and criminal justice. Through graduate student-led projects, this research program will advance insights into algorithmic fairness, bias and transparency issues in public services from a socio-technical perspective in order to train the next generation of researchers engaged in developing computational technologies for the social good. Ultimately the insights gleaned from this research program will be integrated into public sector programs for the purposes of better, high stakes algorithmic decision-making.
Programme
Discovery Grants Program - Individual
Bénéficiaire
Emplacement
Ontario, Canada
Type de bénéficiaire
Academia
Type
Grant
Source
NSERC's Awards Data

Tel que publié

Application id
RGPIN-2022-04570
Name nom
Guha, Shion
Department département
Faculty of Information
Organization id
31
Institution établissement
University of Toronto
Province en
Ontario
Province fr
Ontario
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2024
Competition year année de concours
2022
Award amount
29000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
1507
Committee name en
Computer Science
Committee name fr
Sciences informatiques
Area of application code
1100
Area of application group en
Health, education and social services
Area of application group fr
Santé publique, éducation et services sociaux
Area of application en
Health, education and social services
Area of application fr
Santé publique, éducation et services sociaux
Research subject code
2710
Research subject group en
Information technology
Research subject group fr
Technologie de l'information
Research subject en
Information systems design
Research subject fr
Conception de systèmes d'information
Field of research list names en
Not available
Field of research list names fr
Non disponible
Socioeconomic objective list names en
Not available
Socioeconomic objective list names fr
Non disponible
Application title
Human-Centered Algorithm Design for High Stakes Decision-Making in Public Services
Keywords
child welfare system; algorithmic risk assessment; participatory design; ai and society; human-centered ai; human-centered data science; human-centered machine learning; human-ai interaction; algorithmic decision-making; human-computer interaction
Application summary
Decades of neoliberal politics in North America centered on austerity and privatization have led to public sector agencies increasingly looking towards predictive algorithms and models built with artificial intelligence and machine learning technologies as a means to reduce costs, improve decision-making processes as well as provide greater efficiencies in public policy and social services delivery. Algorithms in the public sector have generally been adopted in the form of risk assessment algorithms with their primary purpose being the preemptive estimation of 'risk'. This has centered resources towards risk management based on individual client characteristics while driving attention away from structural and societal problems. Over the past two decades, several high-stakes decision-making domains such as the child-welfare system (CWS), criminal justice system, education, and healthcare have increasingly turned towards risk assessment algorithms. For instance, within CWS, there is growing public and media pressure because of the harm caused to children (especially minorities and Indigenous children given Canada's history of residential schools) who are removed from the care of their parents as well as where the system failed to remove and protect a child. However, the public sector poses its own challenges with respect to technical (quality of data), social and cultural (workers' interaction with algorithms), theoretical (what is risk assessment?), and societal (impact of algorithms on communities) implications of algorithmic decision-making. This research program will combine concepts from human-computer interaction (HCI), machine learning (ML) and participatory design (PD) to engage in the design and development of human-centered algorithms for high stakes decision making in the public sector. Particularly, the lens chosen will be through integrating participatory design strategies of relevant stakeholders with machine learning models that leverage historical, unstructured narratives to develop a Participatory Machine Learning (PML) framework. Further, this framework will advance the development of strength-based, holistic assessments that aim to produce positive outcomes for people as opposed to the current norm of narrow, deficit-based risk assessments that measure risk to the government. Finally, the PML framework will be validated in the public sector in two crucial areas of Canadian importance - child welfare and criminal justice. Through graduate student-led projects, this research program will advance insights into algorithmic fairness, bias and transparency issues in public services from a socio-technical perspective in order to train the next generation of researchers engaged in developing computational technologies for the social good. Ultimately the insights gleaned from this research program will be integrated into public sector programs for the purposes of better, high stakes algorithmic decision-making.