Canada Spends Logo

Explore our Budget 2026 analysis

Grant: Discovery Grants Program - Individual

York University

$33,000.00

Natural Language Interactions with Visual Analytics

FY 2020-21

Summary
Visual analytics has become a popular way for people to explore and understand data. It combines computational tools with information visualization techniques to facilitate the analytical reasoning process. While interaction is at the core of visual data analysis, the way people interact with data has largely remained the same over the years. Typically, people perform visual data analysis via mouse and touch interactions which are often inefficient when the interface requires many steps to complete a complex task. A promising way to support the user more effectively is to introduce natural language as a complementary input modality for visual analytics, as language can be more expressive and accessible compared to other modalities. The long-term goal of this research is to create the next generation of multimodal interfaces for visual analytics by incorporating natural language as an interaction modality which will enable people to perform complex data analysis tasks faster and more effectively. Using such futuristic interfaces, people will be able to express their complex information needs through a novel combination of input modalities including natural language, touch, gesture, and gaze, enabling rich interaction cycles with visual analytics systems. In response, the system will automatically generate answers combining visual representations with natural language explanations for effective comprehension. To this end, this proposal will focus on the following objectives. First, drawing from latest advances in automatic question answering research, we will develop methods for analyzing the syntactic, semantic, and pragmatic aspects of the question to determine what results should be presented to the user. Second, we will leverage natural language generation techniques to automatically create a multimedia summary by combining visualizations and text that effectively answers the question. Here, the text would not only facilitate users to comprehend the results more effectively by explaining key points but also enhance the transparency of the algorithm by conveying how the results were computed. Finally, we will investigate how combining natural language with other modalities affect the user while interacting with visual analytics and communicating insights. The research will open up new ways of analyzing large and complex datasets, improving the ability of Canadians to discover critical insights and make informed decisions in domains ranging from health informatics to business intelligence to customer analytics. The research program has great potential for supporting data democratization, enabling novice users and people who are visually impaired or who have cognitive difficulties to comprehend charts and perform analytical tasks through natural language. In addition to scientific impact, the program will provide specialized training for graduate and undergraduate students in data science and analytics for their future career.
Program
Discovery Grants Program - Individual
Location
Ontario, Canada
Recipient Type
Academia
Type
Grant
Source
NSERC's Awards Data

As published

Application id
RGPIN-2019-06340
Name nom
Prince, Md Enamul Hoque
Department département
School of Information Technology
Organization id
38
Institution établissement
York University
Province en
Ontario
Province fr
Ontario
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2020
Competition year année de concours
2019
Award amount
33000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
Discovery grants programs
Group fr
Programmes de subventions à la découverte
Committee code
1507
Committee name en
Computer Science
Committee name fr
Sciences informatiques
Area of application code
800
Area of application group en
Information and communication services
Area of application group fr
Services d'information et de télécommunications
Area of application en
Information and communication services
Area of application fr
Services d'information et de télécommunications
Research subject code
2801
Research subject group en
Artificial intelligence
Research subject group fr
Intelligence artificielle
Research subject en
Natural language and speech understanding
Research subject fr
Langage naturel et reconnaissance de la parole
Installment
inst:2-5
Partie
4
Nb partie
5
Application title
Natural Language Interactions with Visual Analytics
Keyword
NSERC
Application summary
Visual analytics has become a popular way for people to explore and understand data. It combines computational tools with information visualization techniques to facilitate the analytical reasoning process. While interaction is at the core of visual data analysis, the way people interact with data has largely remained the same over the years. Typically, people perform visual data analysis via mouse and touch interactions which are often inefficient when the interface requires many steps to complete a complex task. A promising way to support the user more effectively is to introduce natural language as a complementary input modality for visual analytics, as language can be more expressive and accessible compared to other modalities. The long-term goal of this research is to create the next generation of multimodal interfaces for visual analytics by incorporating natural language as an interaction modality which will enable people to perform complex data analysis tasks faster and more effectively. Using such futuristic interfaces, people will be able to express their complex information needs through a novel combination of input modalities including natural language, touch, gesture, and gaze, enabling rich interaction cycles with visual analytics systems. In response, the system will automatically generate answers combining visual representations with natural language explanations for effective comprehension. To this end, this proposal will focus on the following objectives. First, drawing from latest advances in automatic question answering research, we will develop methods for analyzing the syntactic, semantic, and pragmatic aspects of the question to determine what results should be presented to the user. Second, we will leverage natural language generation techniques to automatically create a multimedia summary by combining visualizations and text that effectively answers the question. Here, the text would not only facilitate users to comprehend the results more effectively by explaining key points but also enhance the transparency of the algorithm by conveying how the results were computed. Finally, we will investigate how combining natural language with other modalities affect the user while interacting with visual analytics and communicating insights. The research will open up new ways of analyzing large and complex datasets, improving the ability of Canadians to discover critical insights and make informed decisions in domains ranging from health informatics to business intelligence to customer analytics. The research program has great potential for supporting data democratization, enabling novice users and people who are visually impaired or who have cognitive difficulties to comprehend charts and perform analytical tasks through natural language. In addition to scientific impact, the program will provide specialized training for graduate and undergraduate students in data science and analytics for their future career.
York University - $33,000 Grant (2020-21) | Canada Spends