Canada Spends Logo

Explore our Budget 2026 analysis

Grant: Discovery Grants Program - Individual

University of Toronto

$62,000.00

Robust and Effective Neural Retrieval Techniques

FY 2024-25

Summary
Information Retrieval (IR) methods power commercial search engines and offer the capacity to efficiently and effectively identify relevant information for users. Major search engines process 40,000 queries every second, which translates into 1.2 trillion queries per year. Users rely significantly on the results from these queries to become informed, make decisions, and take action, and as such, IR methods have the capacity to impact the fabric of our society. Robust and effective IR methods can provide accurate and reliable information to users, empowering them to act on trustworthy knowledge. On such a basis, the long-term vision of this proposed research program is to develop IR methods that exhibit robust and effective performance without bias or prejudice. This is a major undertaking as the state-of-the-art IR methods that rely on Large-Language Models (LLMs) have been shown to carry preferences towards certain user queries at the cost of others. This has important socio-economic implications, as user queries that are not addressed properly will present users with undesirable information that can have dire consequences for their decision. To address such a challenge, we will pursue two concrete short-term research objectives: (1) we will develop methods and tools that are able to both identify the preferences and biases encoded in LLMs and also determine subspaces with the LLM embedding space that may have encoded these preferences; and (2) we will propose LLM finetuning techniques that would consider the trade-off between exploration and exploitation when finetuning an LLM such that the LLM can capitalize on its strengths and break-free from its preferences. These two short-term objectives are highly synergistic and allow for the identification of preferences within LLMs as well as addressing these preferences through the fine-tuning process. The goal will be to develop effective IR methods that are able to address the spectrum of user queries regardless of the preferences encoded in the LLM. This research program is highly innovative as it moves beyond treating LLMs as blackboxes that can be finetuned for particular downstream tasks. Rather it provides a systemic approach for dealing with the preferential biases of LLMs so that more effective models are created across the board. The program will have ground-breaking socio-economic impacts as it speaks to the increasingly important issue of preferential biases encoded in LLMs and explores how such preferences can be effectively addressed. It will lead to more effective IR methods that are able to address a broad range of queries, leading to higher user satisfaction and, hence, more informed decision-making for the users. In addition, the program will train 16 HQP in the areas of Information Retrieval, Large-Language Models, and Applied Artificial Intelligence. There is a significant talent gap in these areas in Canada, and hence this program directly contributes to satisfying this need.
Program
Discovery Grants Program - Individual
Location
Ontario, Canada
Recipient Type
Academia
Type
Grant
Source
NSERC's Awards Data

As published

Application id
RGPIN-2024-04733
Name nom
Bagheri, Ebrahim
Department département
Faculty of Information
Organization id
31
Institution établissement
University of Toronto
Province en
Ontario
Province fr
Ontario
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2024
Competition year année de concours
2024
Award amount
62000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
1507
Committee name en
Computer Science
Committee name fr
Sciences informatiques
Area of application code
802
Area of application group en
Information and communication services
Area of application group fr
Services d'information et de télécommunications
Area of application en
Information systems and technology
Area of application fr
Systèmes d'information et technologie de l'information
Research subject code
2700
Research subject group en
Information technology
Research subject group fr
Technologie de l'information
Research subject en
Information technology
Research subject fr
Technologie de l'information
Field of research list names en
Not available
Field of research list names fr
Non disponible
Socioeconomic objective list names en
Not available
Socioeconomic objective list names fr
Non disponible
Application title
Robust and Effective Neural Retrieval Techniques
Keywords
information retrieval; search systems; neural ranking; query performance prediction; retrieval effectiveness; large language models
Application summary
Information Retrieval (IR) methods power commercial search engines and offer the capacity to efficiently and effectively identify relevant information for users. Major search engines process 40,000 queries every second, which translates into 1.2 trillion queries per year. Users rely significantly on the results from these queries to become informed, make decisions, and take action, and as such, IR methods have the capacity to impact the fabric of our society. Robust and effective IR methods can provide accurate and reliable information to users, empowering them to act on trustworthy knowledge. On such a basis, the long-term vision of this proposed research program is to develop IR methods that exhibit robust and effective performance without bias or prejudice. This is a major undertaking as the state-of-the-art IR methods that rely on Large-Language Models (LLMs) have been shown to carry preferences towards certain user queries at the cost of others. This has important socio-economic implications, as user queries that are not addressed properly will present users with undesirable information that can have dire consequences for their decision. To address such a challenge, we will pursue two concrete short-term research objectives: (1) we will develop methods and tools that are able to both identify the preferences and biases encoded in LLMs and also determine subspaces with the LLM embedding space that may have encoded these preferences; and (2) we will propose LLM finetuning techniques that would consider the trade-off between exploration and exploitation when finetuning an LLM such that the LLM can capitalize on its strengths and break-free from its preferences. These two short-term objectives are highly synergistic and allow for the identification of preferences within LLMs as well as addressing these preferences through the fine-tuning process. The goal will be to develop effective IR methods that are able to address the spectrum of user queries regardless of the preferences encoded in the LLM. This research program is highly innovative as it moves beyond treating LLMs as blackboxes that can be finetuned for particular downstream tasks. Rather it provides a systemic approach for dealing with the preferential biases of LLMs so that more effective models are created across the board. The program will have ground-breaking socio-economic impacts as it speaks to the increasingly important issue of preferential biases encoded in LLMs and explores how such preferences can be effectively addressed. It will lead to more effective IR methods that are able to address a broad range of queries, leading to higher user satisfaction and, hence, more informed decision-making for the users. In addition, the program will train 16 HQP in the areas of Information Retrieval, Large-Language Models, and Applied Artificial Intelligence. There is a significant talent gap in these areas in Canada, and hence this program directly contributes to satisfying this need.