Canada Spends Logo

Explore our Budget 2026 analysis

Grant: Discovery Launch Supplement

Institut national de la recherche scientifique

$12,500.00

Automatic Speaker Recognition Robustness Towards Spontaneous Emotional Speech

FY 2024-25

Summary
Identity theft is the act of using someone's personal identification without authorization. We have witnessed a steady increase of such problem, with its peak in 2020, during the COVID-19 pandemic. Identity frauds can lead to billions of dollars in financial losses and is a major problem for consumers, industries and governments. Thus, more secure identity verification, such as those based on biometrics, is an alternative to add an extra layer of security to traditional authentication methods (e.g., passwords and PINs). In fact, financial institutions have identified voice biometrics as an opportunity to increase security, lower authentication time and provide cost-effective access control to their services. This is motivated by the fact that voice biometrics has significantly matured over the last few years due to recent advancements in machine learning (ML) and artificial intelligence (AI). Although the deployment of voice recognition solutions has become a reality in many applications, such as telephone banking, forensics and voice-based assistants, the robustness of such systems towards variability caused by spontaneous emotional speech is still a challenge.  Automatic speaker recognition, for instance, is known to perform well when emotionally-neutral speech is present in both train and test data. Nevertheless, this is not realistic in real-world scenarios where human beings are constantly changing their emotional state. For example, using our voice to authenticate ourselves in our bank account after going through a stressful situation will likely lead to involuntary variations in the physiology of the speech production (i.e., changes in our vocal tract configuration). Hence, features extracted from emotional speech deviate from the ones extracted from neutral speech signals. This creates a mismatch between train and test speech data and leads to significant performance degradation of voice recognition systems. Thus, the overarching goal of this research program is to develop a robust speaker recognition system that can reliably be used in everyday environments. We will explore voice conversion methods based on deep neural network (DNN) to improve the robustness of voice biometrics in emotional environments. This will result in voice recognition models capable of minimizing the mismatch between neutral and emotional speech. To assure robustness in more realistic scenarios, we will also consider emotional speech signals captured in diverse acoustic environments, where background noise and reverberation are often present. As voice conversion techniques can be maliciously used to generate deepfake (DF) audios, we will also investigate countermeasures to mitigate the vulnerability of voice biometric systems to attacks based on DF. Another concern will be speech data privacy. To achieve that, we will focus on investigating decentralized training as well as anonymization techniques.
Program
Discovery Launch Supplement
Location
Quebec, Canada
Recipient Type
Academia
Type
Grant
Source
NSERC's Awards Data

As published

Application id
DGECR-2024-00023
Name nom
Avila, Anderson
Department département
Énergie Matériaux Télécommunications Research Centre
Organization id
56
Institution établissement
Institut national de la recherche scientifique
Province en
Québec
Province fr
Québec
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2024
Competition year année de concours
2024
Award amount
12500
Program id
DGECR
Program name en
Discovery Launch Supplement
Program name fr
Supplément Tremplin vers la découverte
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
1510
Committee name en
Electrical and Computer Engineering
Committee name fr
Génie électrique et informatique
Area of application code
1207
Area of application group en
Advancement of knowledge
Area of application group fr
Avancement des connaissances
Area of application en
Information, computer and communication technologies
Area of application fr
Technologie de l'information, des ordinateurs et des réseaux de communications
Research subject code
2514
Research subject group en
Electrical and electronic engineering
Research subject group fr
Génie électrique et électronique
Research subject en
Digital signal processing
Research subject fr
Traitement des signaux numériques
Field of research list names en
Not available
Field of research list names fr
Non disponible
Socioeconomic objective list names en
Not available
Socioeconomic objective list names fr
Non disponible
Application title
Automatic Speaker Recognition Robustness Towards Spontaneous Emotional Speech
Keywords
automatic speaker recognition; voice biometrics; emotional voice conversion; robustness; deepfake
Application summary
Identity theft is the act of using someone's personal identification without authorization. We have witnessed a steady increase of such problem, with its peak in 2020, during the COVID-19 pandemic. Identity frauds can lead to billions of dollars in financial losses and is a major problem for consumers, industries and governments. Thus, more secure identity verification, such as those based on biometrics, is an alternative to add an extra layer of security to traditional authentication methods (e.g., passwords and PINs). In fact, financial institutions have identified voice biometrics as an opportunity to increase security, lower authentication time and provide cost-effective access control to their services. This is motivated by the fact that voice biometrics has significantly matured over the last few years due to recent advancements in machine learning (ML) and artificial intelligence (AI). Although the deployment of voice recognition solutions has become a reality in many applications, such as telephone banking, forensics and voice-based assistants, the robustness of such systems towards variability caused by spontaneous emotional speech is still a challenge.  Automatic speaker recognition, for instance, is known to perform well when emotionally-neutral speech is present in both train and test data. Nevertheless, this is not realistic in real-world scenarios where human beings are constantly changing their emotional state. For example, using our voice to authenticate ourselves in our bank account after going through a stressful situation will likely lead to involuntary variations in the physiology of the speech production (i.e., changes in our vocal tract configuration). Hence, features extracted from emotional speech deviate from the ones extracted from neutral speech signals. This creates a mismatch between train and test speech data and leads to significant performance degradation of voice recognition systems. Thus, the overarching goal of this research program is to develop a robust speaker recognition system that can reliably be used in everyday environments. We will explore voice conversion methods based on deep neural network (DNN) to improve the robustness of voice biometrics in emotional environments. This will result in voice recognition models capable of minimizing the mismatch between neutral and emotional speech. To assure robustness in more realistic scenarios, we will also consider emotional speech signals captured in diverse acoustic environments, where background noise and reverberation are often present. As voice conversion techniques can be maliciously used to generate deepfake (DF) audios, we will also investigate countermeasures to mitigate the vulnerability of voice biometric systems to attacks based on DF. Another concern will be speech data privacy. To achieve that, we will focus on investigating decentralized training as well as anonymization techniques.
Institut national de la recherche scientifique - $12,… | Canada Spends