Canada Spends Logo

Explorez notre analyse du budget 2026

Grant: Discovery Grants Program - Individual

University of Waterloo

$48,000.00

Accurate User Modeling of Search and Decision Making Tasks for Improved Offline Evaluation of Information Retrieval

EF 2020-21

Résumé
My research program's objective is to make the search for information safer and more effective. Information retrieval, commonly known as search, is prevalent today with the ubiquitous use of web search engines (e.g. Google) and organizations providing information access via search engines. Engineers and researchers use test collections and offline effectiveness measures to improve the quality of search results. Offline evaluation allows engineers to affordably and efficiently simulate the effect of their algorithmic changes and measure the extent to which these changes help or harm search users. My research program works to improve the predictive ability of offline effectiveness measures by more accurately modeling user behaviour. Near-term objectives include 1) creating new methods to efficiently construct test collections that support user-model-based effectiveness measures and 2) improved prediction accuracy for effectiveness measures by modeling user interaction beyond a single query and search results pair. In the next 5 years and beyond, my research program will move to emphasize research on the evaluation of search in terms of the task outcome for a user rather than simply the documents retrieved by the user. For example, many search users turn to web search to help them make decisions about health-related issues. Unfortunately, users can perceive documents containing incorrect information as relevant to their decision-making tasks. We have shown that search results biased towards incorrect information can significantly reduce searchers' decision accuracy. Other researchers have shown that major search engines are biased and users may only view correct answers 45-52% of the time for health related searches. When search engines lead people to incorrect decisions about their health care, not only can money be wasted on scam treatments, but people's health can be harmed. My research program focuses on measuring search engine effectiveness so that researchers and engineers have the correct measurements to guide their work. I have proposed a line of work to study decision-making with search engines and to predict the accuracy of the decisions people reach when they use search engines for decision support. Our initial domain of study will be health-related search. The TREC Decision Track, which I co-organize as a part of my research program, provides a venue for researchers to improve the quality of search engines and reduce people's exposure to medical misinformation and improve their decisions. This proposed research can lead to improvements for everyone who uses search engines by helping them avoid misinformation and make better decisions. The research program also provides training for students in high value areas such as data analytics, user modeling, machine learning, and large scale data processing.
Programme
Discovery Grants Program - Individual
Emplacement
Ontario, Canada
Type de bénéficiaire
Academia
Type
Grant
Source
NSERC's Awards Data

Tel que publié

Application id
RGPIN-2020-04665
Name nom
Smucker, Mark
Department département
Management Sciences
Organization id
33
Institution établissement
University of Waterloo
Province en
Ontario
Province fr
Ontario
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2020
Competition year année de concours
2020
Award amount
48000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
Discovery grants programs
Group fr
Programmes de subventions à la découverte
Committee code
1507
Committee name en
Computer Science
Committee name fr
Sciences informatiques
Area of application code
802
Area of application group en
Information and communication services
Area of application group fr
Services d'information et de télécommunications
Area of application en
Information systems and technology
Area of application fr
Systèmes d'information et technologie de l'information
Research subject code
2710
Research subject group en
Information technology
Research subject group fr
Technologie de l'information
Research subject en
Information systems design
Research subject fr
Conception de systèmes d'information
Installment
inst:1-5
Partie
5
Nb partie
5
Application title
Accurate User Modeling of Search and Decision Making Tasks for Improved Offline Evaluation of Information Retrieval
Keyword
NSERC
Application summary
My research program's objective is to make the search for information safer and more effective. Information retrieval, commonly known as search, is prevalent today with the ubiquitous use of web search engines (e.g. Google) and organizations providing information access via search engines. Engineers and researchers use test collections and offline effectiveness measures to improve the quality of search results. Offline evaluation allows engineers to affordably and efficiently simulate the effect of their algorithmic changes and measure the extent to which these changes help or harm search users. My research program works to improve the predictive ability of offline effectiveness measures by more accurately modeling user behaviour. Near-term objectives include 1) creating new methods to efficiently construct test collections that support user-model-based effectiveness measures and 2) improved prediction accuracy for effectiveness measures by modeling user interaction beyond a single query and search results pair. In the next 5 years and beyond, my research program will move to emphasize research on the evaluation of search in terms of the task outcome for a user rather than simply the documents retrieved by the user. For example, many search users turn to web search to help them make decisions about health-related issues. Unfortunately, users can perceive documents containing incorrect information as relevant to their decision-making tasks. We have shown that search results biased towards incorrect information can significantly reduce searchers' decision accuracy. Other researchers have shown that major search engines are biased and users may only view correct answers 45-52% of the time for health related searches. When search engines lead people to incorrect decisions about their health care, not only can money be wasted on scam treatments, but people's health can be harmed. My research program focuses on measuring search engine effectiveness so that researchers and engineers have the correct measurements to guide their work. I have proposed a line of work to study decision-making with search engines and to predict the accuracy of the decisions people reach when they use search engines for decision support. Our initial domain of study will be health-related search. The TREC Decision Track, which I co-organize as a part of my research program, provides a venue for researchers to improve the quality of search engines and reduce people's exposure to medical misinformation and improve their decisions. This proposed research can lead to improvements for everyone who uses search engines by helping them avoid misinformation and make better decisions. The research program also provides training for students in high value areas such as data analytics, user modeling, machine learning, and large scale data processing.