Grant: Discovery Grants Program - Individual
The University of British Columbia
$36,000.00
Computational Approaches to Massively Multilingual Speech Recognition, Retrieval and Alignment
EF 2024-25
Résumé
In the past decades, modern societies have become increasingly multilingual thanks to the free flow of people and information across the globe. While speech technologies have great potential to support and facilitate multilingual communications, recent advancement in speech technology has overwhelmingly concentrated in several high-resource languages. It remains incredibly challenging to develop accurate and inclusive speech processing systems for all 7000 languages in the world. In this proposal, I propose three research threads that have the potential to build fairer and more inclusive speech technologies for all languages.
Designing effective multilingual speech representations: The first barrier to multilingual speech processing is how to effectively represent the diversity of world languages. To tackle this challenge, I plan not only to gather and annotate speech datasets to cover more low-resource languages, but also investigate effective multilingual representations with both linguistic knowledge (e.g., International Phonetic Alphabet) and engineered systems (data-driven discrete representations).
Advancing state-of-the-art speech models for retrieval, recognition and alignment: I will develop speech technologies essential to the collection, annotation and exploration of multilingual speech data. The technology will assist users to transcribe speech recordings (recognition), and obtain fine-grained annotations of speech at sentence, word or phonetic levels (segmentation and alignment). I also aim to build information retrieval systems to query multilingual speech recordings through semantic content or acoustic forms (retrieval) to satisfy users' diverse information needs.
Developing software toolkit to facilitate the processing of large-scale speech data: To disseminate research outcomes, I plan to deliver the speech technologies developed in this proposal as an easy-to-use and publicly available software toolkit for all speech practioners.
The outcome of this proposal will advance multilingual speech technologies and create a publicly available toolkit for multilingual speech processing. The proposed project has great potential to empower speech engineers, linguists and speech pathologists to acquire high-quality annotations in more languages and more efficiently and deliver fairer and more inclusive speech applications that support and benefit Canada as an increasingly multilingual society.
Programme
Discovery Grants Program - Individual
Bénéficiaire
Emplacement
British Columbia, Canada
Type de bénéficiaire
Academia
Type
Grant
Source
NSERC's Awards Data
Tel que publié
Application id
RGPIN-2024-04431
Name nom
Zhu, Jian
Department département
Department of Linguistics
Organization id
2
Institution établissement
The University of British Columbia
Province en
British Columbia
Province fr
Colombie-Britannique
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2024
Competition year année de concours
2024
Award amount
36000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
1507
Committee name en
Computer Science
Committee name fr
Sciences informatiques
Area of application code
1207
Area of application group en
Advancement of knowledge
Area of application group fr
Avancement des connaissances
Area of application en
Information, computer and communication technologies
Area of application fr
Technologie de l'information, des ordinateurs et des réseaux de communications
Research subject code
2801
Research subject group en
Artificial intelligence
Research subject group fr
Intelligence artificielle
Research subject en
Natural language and speech understanding
Research subject fr
Langage naturel et reconnaissance de la parole
Field of research list names en
Not available
Field of research list names fr
Non disponible
Socioeconomic objective list names en
Not available
Socioeconomic objective list names fr
Non disponible
Application title
Computational Approaches to Massively Multilingual Speech Recognition, Retrieval and Alignment
Keywords
automatic speech recognition; multilingual speech processing; phoneme recognition; deep learning; spoken language processing; language documentation; natural language processing
Application summary
In the past decades, modern societies have become increasingly multilingual thanks to the free flow of people and information across the globe. While speech technologies have great potential to support and facilitate multilingual communications, recent advancement in speech technology has overwhelmingly concentrated in several high-resource languages. It remains incredibly challenging to develop accurate and inclusive speech processing systems for all 7000 languages in the world. In this proposal, I propose three research threads that have the potential to build fairer and more inclusive speech technologies for all languages.
Designing effective multilingual speech representations: The first barrier to multilingual speech processing is how to effectively represent the diversity of world languages. To tackle this challenge, I plan not only to gather and annotate speech datasets to cover more low-resource languages, but also investigate effective multilingual representations with both linguistic knowledge (e.g., International Phonetic Alphabet) and engineered systems (data-driven discrete representations).
Advancing state-of-the-art speech models for retrieval, recognition and alignment: I will develop speech technologies essential to the collection, annotation and exploration of multilingual speech data. The technology will assist users to transcribe speech recordings (recognition), and obtain fine-grained annotations of speech at sentence, word or phonetic levels (segmentation and alignment). I also aim to build information retrieval systems to query multilingual speech recordings through semantic content or acoustic forms (retrieval) to satisfy users' diverse information needs.
Developing software toolkit to facilitate the processing of large-scale speech data: To disseminate research outcomes, I plan to deliver the speech technologies developed in this proposal as an easy-to-use and publicly available software toolkit for all speech practioners.
The outcome of this proposal will advance multilingual speech technologies and create a publicly available toolkit for multilingual speech processing. The proposed project has great potential to empower speech engineers, linguists and speech pathologists to acquire high-quality annotations in more languages and more efficiently and deliver fairer and more inclusive speech applications that support and benefit Canada as an increasingly multilingual society.