Canada Spends Logo

Explore our Budget 2026 analysis

Grant: Discovery Grants Program - Individual

University of Waterloo

$52,000.00

Impact of decision theoretic models in information elicitation

FY 2022-23

Summary
Prediction markets and scoring rules are used daily to elicit personal probability estimates on the outcome of future events. For example, “Will project B be complete by June 1, 2018?” is one security that may be traded in a prediction market, or have agents report their beliefs in a scoring rule. Prediction markets and scoring rules are used for questions and instances in which there is no historical data, and the only way to forecast the outcome of a future event is to ask experts, workers, or anyone having an opinion, thereby leveraging the “wisdom of the crowd.”When initially proposed, both prediction markets and scoring rules were shown to work for agents that are risk-neutral (are indifferent between getting $50 or $100 with probability 0.5, and nothing otherwise), myopic (do not take future payoffs into account), and rational (are maximizing their total reward). Risk-neutrality, rationality, and myopic behavior are known not to hold in practice, and more realistic models of human decision-making have been proposed and verified in various laboratory and field studies. In the proposed research program, we will characterize the efficacy of prediction markets and scoring rules when humans, agents, are modeled using these more realistic and modern decision models. In particular, we will characterize how well prediction markets and scoring rules aggregate information when participants are cumulative prospect theory agents, when external incentives exist, when participants are ambiguity averse agents, and when agents have hyperbolic discounting. All of these models have been shown to exist in humans in practice, but they have not been considered in prediction markets, and some have not been considered in scoring rules. The results of this research program will be new prediction market and scoring rule mechanisms that improve the accuracy of these probability elicitation methods when faced with agents modeled using the decision models discussed above. The outcomes of the proposed research program will be more accurate forecasts from prediction markets and scoring rules in settings that occur more often in practice than the settings considered when these mechanisms were proposed. This research will further highlight the limitations and benefits of prediction markets and scoring rules. In addition, new information elicitation mechanisms may be developed for the settings considered. Canada stands to benefit from the proposed research as we will work to make the developed methods available in software that may be deployed in Canadian organizations, thereby providing them with a distinct competitive advantage of having more accurate estimates on the likelihood of a future event, leading to faster and more accurate decisions. HQP trained under the developed program may deploy the developed mechanisms or use their analysis and technical skills in industry, education, or public sector throughout Canada.
Program
Discovery Grants Program - Individual
Location
Ontario, Canada
Recipient Type
Academia
Type
Grant
Source
NSERC's Awards Data

As published

Application id
RGPIN-2018-04005
Name nom
Dimitrov, Stanko
Department département
Management Sciences - Management Sciences
Organization id
33
Institution établissement
University of Waterloo
Province en
Ontario
Province fr
Ontario
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2022
Competition year année de concours
2018
Award amount
52000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
1509
Committee name en
Civil, Industrial and Systems Engineering
Committee name fr
Génie civil, industriel et des systèmes
Area of application code
802
Area of application group en
Information and communication services
Area of application group fr
Services d'information et de télécommunications
Area of application en
Information systems and technology
Area of application fr
Systèmes d'information et technologie de l'information
Research subject code
1600
Research subject group en
Industrial engineering
Research subject group fr
Génie industriel
Research subject en
Industrial engineering
Research subject fr
Génie industriel
Installment
inst:5;6
Partie
2
Nb partie
6
Application title
Impact of decision theoretic models in information elicitation
Application summary
Prediction markets and scoring rules are used daily to elicit personal probability estimates on the outcome of future events. For example, “Will project B be complete by June 1, 2018?” is one security that may be traded in a prediction market, or have agents report their beliefs in a scoring rule. Prediction markets and scoring rules are used for questions and instances in which there is no historical data, and the only way to forecast the outcome of a future event is to ask experts, workers, or anyone having an opinion, thereby leveraging the “wisdom of the crowd.”When initially proposed, both prediction markets and scoring rules were shown to work for agents that are risk-neutral (are indifferent between getting $50 or $100 with probability 0.5, and nothing otherwise), myopic (do not take future payoffs into account), and rational (are maximizing their total reward). Risk-neutrality, rationality, and myopic behavior are known not to hold in practice, and more realistic models of human decision-making have been proposed and verified in various laboratory and field studies. In the proposed research program, we will characterize the efficacy of prediction markets and scoring rules when humans, agents, are modeled using these more realistic and modern decision models. In particular, we will characterize how well prediction markets and scoring rules aggregate information when participants are cumulative prospect theory agents, when external incentives exist, when participants are ambiguity averse agents, and when agents have hyperbolic discounting. All of these models have been shown to exist in humans in practice, but they have not been considered in prediction markets, and some have not been considered in scoring rules. The results of this research program will be new prediction market and scoring rule mechanisms that improve the accuracy of these probability elicitation methods when faced with agents modeled using the decision models discussed above. The outcomes of the proposed research program will be more accurate forecasts from prediction markets and scoring rules in settings that occur more often in practice than the settings considered when these mechanisms were proposed. This research will further highlight the limitations and benefits of prediction markets and scoring rules. In addition, new information elicitation mechanisms may be developed for the settings considered. Canada stands to benefit from the proposed research as we will work to make the developed methods available in software that may be deployed in Canadian organizations, thereby providing them with a distinct competitive advantage of having more accurate estimates on the likelihood of a future event, leading to faster and more accurate decisions. HQP trained under the developed program may deploy the developed mechanisms or use their analysis and technical skills in industry, education, or public sector throughout Canada.