Canada Spends Logo

Explore our Budget 2026 analysis

Grant: Department of National Defence / NSERC Research Partnership

Université Laval

$123,391.00

Cognitive Shadow: capturing judgement policies for real-time decision support

FY 2019-20

Summary
Formally capturing expert knowledge is a major challenge in the field of decision support. Policy capturing is a judgment analysis method that infers a decision policy based on past examples. A ""judgmental bootstrapping"" effect can be observed when a model actually outperforms the human decision maker on which the model was based (typically because the model is not prone to fatigue or distraction). A prototype policy capturing tool called the Cognitive Shadow has been developed to automatically learn an operator's decision pattern and provide warnings in real-time when a mismatch is detected between the user's decision and the inferred decision policy. This project aims at investigating critical human factors and cognitive engineering issues for implementing the Cognitive Shadow capability in the context of naval air defence. The specific objectives are to 1) assess factors influencing operator trust in the cognitive assistant, 2) assess impacts on workload,behavior and performance measures, and 3) understand the different impacts of two possible modes of operation: a shadowing mode where the operator receives an alert only when a mismatch with the decision model is detected, versus a recommendation mode where the operator receives information about the model's expected decision. A series of human-in-the-loop experiments will be performed in two phases in order to systematically investigate these issues. In Phase 1 the Simulated Combat Control System testbed is used as a functional simulation of naval air defence practiced aboard the Canadian Navy's Halifax class frigates. Phase 1 will involve two laboratory experiments involving implementations of the Cognitive Shadow hypothesized to differentially impact workload, trust, performance and competency preservation. Phase 2 will involve using the Cognitive Shadow with domain experts in the context of a real-world combat management system. This project will help Thales Research and Technology Canada take a leap forward in maturing the new Cognitive Shadowtechnology, and Defence Research and Development Canada in leveraging the technological advances for the Royal Canadian Navy.
Program
Department of National Defence / NSERC Research Partnership
Location
Quebec, Canada
Recipient Type
Academia
Type
Grant
Source
NSERC's Awards Data

As published

Application id
518400-2017
Name nom
Tremblay, Sébastien
Department département
Psychologie, École de
Organization id
51
Institution établissement
Université Laval
Province en
Québec
Province fr
Québec
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2019
Competition year année de concours
2017
Award amount
123391
Program id
DNDPJ
Program name en
Department of National Defence / NSERC Research Partnership
Program name fr
Partenariat de recherche du ministère de la Défense nationale et du CRSNG
Group en
RESEARCH PARTNERSHIPS
Group fr
PARTENARIATS DE RECHERCHE
Committee code
96
Committee name en
RPP Internal Decision Cttee
Committee name fr
Cté de décision interne - PPR
Area of application code
802
Area of application group en
Information and communication services
Area of application group fr
Services d'information et de télécommunications
Area of application en
Information systems and technology
Area of application fr
Systèmes d'information et technologie de l'information
Research subject code
3607
Research subject group en
Inorganic chemistry
Research subject group fr
Chimie inorganique
Research subject en
Catalysis
Research subject fr
Catalyse
Installment
inst:2 - 2
Partie
2
Nb partie
2
Application title
Cognitive Shadow: capturing judgement policies for real-time decision support
Keyword
NSERC
Application summary
Formally capturing expert knowledge is a major challenge in the field of decision support. Policy capturing is a judgment analysis method that infers a decision policy based on past examples. A ""judgmental bootstrapping"" effect can be observed when a model actually outperforms the human decision maker on which the model was based (typically because the model is not prone to fatigue or distraction). A prototype policy capturing tool called the Cognitive Shadow has been developed to automatically learn an operator's decision pattern and provide warnings in real-time when a mismatch is detected between the user's decision and the inferred decision policy. This project aims at investigating critical human factors and cognitive engineering issues for implementing the Cognitive Shadow capability in the context of naval air defence. The specific objectives are to 1) assess factors influencing operator trust in the cognitive assistant, 2) assess impacts on workload,behavior and performance measures, and 3) understand the different impacts of two possible modes of operation: a shadowing mode where the operator receives an alert only when a mismatch with the decision model is detected, versus a recommendation mode where the operator receives information about the model's expected decision. A series of human-in-the-loop experiments will be performed in two phases in order to systematically investigate these issues. In Phase 1 the Simulated Combat Control System testbed is used as a functional simulation of naval air defence practiced aboard the Canadian Navy's Halifax class frigates. Phase 1 will involve two laboratory experiments involving implementations of the Cognitive Shadow hypothesized to differentially impact workload, trust, performance and competency preservation. Phase 2 will involve using the Cognitive Shadow with domain experts in the context of a real-world combat management system. This project will help Thales Research and Technology Canada take a leap forward in maturing the new Cognitive Shadowtechnology, and Defence Research and Development Canada in leveraging the technological advances for the Royal Canadian Navy.