Canada Spends Logo

Explorez notre analyse du budget 2026

Grant: Discovery Grants Program - Individual

Université Laval

$31,000.00

Closing the Reality Gap in Machine Learning for Speech and Audio

EF 2023-24

Résumé
With the proliferation of deep learning, machine learning models have realized very substantial performance improvements on bench-marking datasets. Examples for this kind of success include domains such as computer vision, speech recognition, natural language processing, speech enhancement-separation, reinforcement learning, and so on. However, the impressive performance obtained on the bench-marking datasets does not necessarily translate to real-life performance. For instance, machine learning models trained under a fixed set-of-conditions would not necessarily perform well under previously unseen conditions. For example a personal assistant might not perform well in reverberant rooms if it is trained under non-reverberant rooms, or speech recognizer in a smart phone might not work well in various types of environmental noise. With the support of this Discovery Grant, I would like to therefore establish a 5 year research program that would addresses the aforementioned shortcomings to increase and accurately measure the generalization performance of neural networks in real-life deployment for speech and audio applications. To progress towards this goal, my first aim is to collect datasets that are representative to the challenges of real-life. As we have shown in our recent work, `REAL-M:Towards Speech Separation on Real Mixtures', it is possible to collect data in a scalable way through crowdsourcing, which promotes diversity in acoustic conditions as well. We will aim to collect data that can be used for tasks such as speech enhancement, speech separation, automatic speech recognition, multi-speaker speech recognition, speech diarization, sound and speech generation. After completing this first step of data collection, we will investigate ways to improve performance under real-life conditions by improving upon the techniques from the established machine learning fields such as domain adaptation and domain generalization. In the later phases of the project we will also investigate the case where we would like to train models under evolving conditions, in order to aim to develop techniques that would effectively alleviate forgetting and improve domain generalization under the continual learning setup. The potential outcome of this research project aims to significantly improve the performance of machine learning based speech and audio technologies under real-life conditions, for which there currently is a large commercial interest from big technology companies, as showcased by the prevalence of voice assistants. This program presents possibility of training 4 Highly Qualified Personnels (2 PhD students, 2 MSc students). These HQPs will develop an expertise in machine learning for speech/audio, which are currently in-demand competencies in the job market of Canada.
Programme
Discovery Grants Program - Individual
Bénéficiaire
Emplacement
Quebec, Canada
Type de bénéficiaire
Academia
Type
Grant
Source
NSERC's Awards Data

Tel que publié

Application id
RGPIN-2023-05759
Name nom
Subakan, Yusuf Cem
Department département
Sciences et génie, faculté des - Sciences et génie, faculté des
Organization id
51
Institution établissement
Université Laval
Province en
Québec
Province fr
Québec
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2023
Competition year année de concours
2023
Award amount
31000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
1507
Committee name en
Computer Science
Committee name fr
Sciences informatiques
Area of application code
1207
Area of application group en
Advancement of knowledge
Area of application group fr
Avancement des connaissances
Area of application en
Information, computer and communication technologies
Area of application fr
Technologie de l'information, des ordinateurs et des réseaux de communications
Research subject code
9999
Research subject group en
Not available
Research subject group fr
Not available
Research subject en
Non disponible
Research subject fr
Non disponible
Application title
Closing the Reality Gap in Machine Learning for Speech and Audio
Keyword
Continual Learning; Conversational AI; Deep Learning; Domain Adaptation; Domain Generalization; Machine Learning for Signal Processing; Machine Learning for Speech and Audio; Out-of-domain generalization; Speech Enhancement and Separation; Speech Processing
Application summary
With the proliferation of deep learning, machine learning models have realized very substantial performance improvements on bench-marking datasets. Examples for this kind of success include domains such as computer vision, speech recognition, natural language processing, speech enhancement-separation, reinforcement learning, and so on. However, the impressive performance obtained on the bench-marking datasets does not necessarily translate to real-life performance. For instance, machine learning models trained under a fixed set-of-conditions would not necessarily perform well under previously unseen conditions. For example a personal assistant might not perform well in reverberant rooms if it is trained under non-reverberant rooms, or speech recognizer in a smart phone might not work well in various types of environmental noise. With the support of this Discovery Grant, I would like to therefore establish a 5 year research program that would addresses the aforementioned shortcomings to increase and accurately measure the generalization performance of neural networks in real-life deployment for speech and audio applications. To progress towards this goal, my first aim is to collect datasets that are representative to the challenges of real-life. As we have shown in our recent work, `REAL-M:Towards Speech Separation on Real Mixtures', it is possible to collect data in a scalable way through crowdsourcing, which promotes diversity in acoustic conditions as well. We will aim to collect data that can be used for tasks such as speech enhancement, speech separation, automatic speech recognition, multi-speaker speech recognition, speech diarization, sound and speech generation. After completing this first step of data collection, we will investigate ways to improve performance under real-life conditions by improving upon the techniques from the established machine learning fields such as domain adaptation and domain generalization. In the later phases of the project we will also investigate the case where we would like to train models under evolving conditions, in order to aim to develop techniques that would effectively alleviate forgetting and improve domain generalization under the continual learning setup. The potential outcome of this research project aims to significantly improve the performance of machine learning based speech and audio technologies under real-life conditions, for which there currently is a large commercial interest from big technology companies, as showcased by the prevalence of voice assistants. This program presents possibility of training 4 Highly Qualified Personnels (2 PhD students, 2 MSc students). These HQPs will develop an expertise in machine learning for speech/audio, which are currently in-demand competencies in the job market of Canada.