Grant: Discovery Grants Program - Individual

Online learning environments aim to deliver efficacious instructions, but rarely take into consideration motivational factors like engagement that plays an important role in effective learning. Standard methods such as self-reporting and observational checklists assess, through questionnaires, learners' reported levels of attention, distraction, excitement, or boredom in their learning activities. However, these methods require considerable time and effort from both learners and observers and may also present issues with validity since the results depend on learners willingness to report their emotions and the accuracy of their perceptions about their emotions. While widely used in class-room settings, these methods are not effective in online learning settings. The purpose of this research is to design a system for automatic detection of learners' engagement and provide pedagogical intervention during online learning. Cognitive science and psychology concepts, as well as computer vision and artificial intelligence techniques will be used to assess learners' level of engagement interpreting facial expressions, gesture, and posture, and design pedagogical intervention based on that information for the online learners. Specifically, our three objectives are as follows: � Design a new system for learner's engagement detection, one that considers engagement levels, intensity, and frequency of reports, authentic emotions, and correlations with task performance � Develop an affective pedagogical agent to inhabit in the learning environment that can respond appropriately to affective states of engagement of the online learners. � Integrate and evaluate the performance of engagement detection and pedagogical agent in a real-life online learning setting. Research shows that engagement is malleable and proper pedagogical interventions, learning designs, and feedback can enhance learner engagement. This research will take an innovative approach to assessments of such engagement, with the potential to shift the standard approaches currently in use. The high reward of this research will be found in the many stakeholders-a broad range of online learners and instructors-who will benefit from automatic engagement detection and personalized pedagogical support. These will generate significant learning gains, increased productivity, and higher retention rates in online courses. This project will be further extended for the students' engagement detection and pedagogical intervention design using multi-modal approaches. Information from facial expression, gesture, and posture will be combined to students' online activities and the course forum posts. We will also investigate augmented reality and virtual reality techniques to improve the pedagogical agents. We expect that this research will eventually contribute to achieving our long-term goal of developing an effective framework of engagement detection and pedagogical intervention design for the online learners.