Grant: Strategic Projects - Group
McGill University
$194,470.00
ANR Multimodal analysis and knowledge inference for musical orchestration (MAKIMOno)
EF 2017-18
Résumé
This project addresses scientifically one of the most complex aspects of music: the use of timbre to shape music through various modes of orchestration. In close interaction with a Canadian company, OrchPlayMusic Inc., this first-of-its-kind project will lead to the creation of information technologies for human interaction with digital media that will radically change orchestration pedagogy, provide better tools for the computer-aided interactive creation of musical content, and lead to a better understanding of perceptual principles underlying orchestration practice. Timbre is the complex set of tone colours that distinguish sounds emanating from different instruments or their blended combinations. Orchestration is the art of writing music that combines different instruments to achieve various sonic goals. Orchestration pedagogy focuses solely on describing how composers score instruments rather than understanding why they made such choices. We will address the why scientifically with the methods of computer science, digital signal processing, and experimental psychology. Drawing from the results of the co-applicants' previous research, this project will create efficient novel analysis, learning, and interaction techniques that reveal the underlying theoretical bases for orchestration. These techniques include multivariate time series analysis of musical signals and knowledge inference through deep representational learning that will decipher automatically the structure of multimodal musical representations (music symbols, acoustic properties, perceptual results) in order to provide optimal descriptors for the understanding of orchestration principles. The project will rely on both solid perceptual principles and empirically characterized orchestration examples to build a scientifically grounded theory of musical orchestration from a large existing multimodal database, to be extended within the course of the research. Solving these complex issues in a strong international collaboration with the French ANR will also lead to generic multimodal learning and analysis techniques broadly applicable to the fields of machine learning and computational perception that will place Canada at the forefront of interdisciplinary innovation.
Programme
Strategic Projects - Group
Bénéficiaire
Emplacement
Quebec, Canada
Type de bénéficiaire
Academia
Type
Grant
Source
NSERC's Awards Data
Tel que publié
Application id
507004-2017
Name nom
McAdams, Stephen
Department département
Music Research
Organization id
61
Institution établissement
McGill University
Province en
Québec
Province fr
Québec
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2017
Competition year année de concours
2017
Award amount
194470
Program id
STPGP
Program name en
Strategic Projects - Group
Program name fr
Projets stratégiques - De groupe
Group en
Research partnerships programs
Group fr
Programmes de partenariat
Committee code
1588
Committee name en
Strategic Projects - ICT
Committee name fr
Projets stratégiques - ICT
Area of application code
802
Area of application group en
Information and communication services
Area of application group fr
Services d'information et de télécommunications
Area of application en
Information systems and technology
Area of application fr
Systèmes d'information et technologie de l'information
Research subject code
6301
Research subject group en
Psychology
Research subject group fr
Psychologie
Research subject en
Sensory systems and perception
Research subject fr
Systèmes sensoriels et perception
Partie
1
Nb partie
3
Application title
ANR Multimodal analysis and knowledge inference for musical orchestration (MAKIMOno)
Keyword
NSERC
Application summary
This project addresses scientifically one of the most complex aspects of music: the use of timbre to shape music through various modes of orchestration. In close interaction with a Canadian company, OrchPlayMusic Inc., this first-of-its-kind project will lead to the creation of information technologies for human interaction with digital media that will radically change orchestration pedagogy, provide better tools for the computer-aided interactive creation of musical content, and lead to a better understanding of perceptual principles underlying orchestration practice. Timbre is the complex set of tone colours that distinguish sounds emanating from different instruments or their blended combinations. Orchestration is the art of writing music that combines different instruments to achieve various sonic goals. Orchestration pedagogy focuses solely on describing how composers score instruments rather than understanding why they made such choices. We will address the why scientifically with the methods of computer science, digital signal processing, and experimental psychology. Drawing from the results of the co-applicants' previous research, this project will create efficient novel analysis, learning, and interaction techniques that reveal the underlying theoretical bases for orchestration. These techniques include multivariate time series analysis of musical signals and knowledge inference through deep representational learning that will decipher automatically the structure of multimodal musical representations (music symbols, acoustic properties, perceptual results) in order to provide optimal descriptors for the understanding of orchestration principles. The project will rely on both solid perceptual principles and empirically characterized orchestration examples to build a scientifically grounded theory of musical orchestration from a large existing multimodal database, to be extended within the course of the research. Solving these complex issues in a strong international collaboration with the French ANR will also lead to generic multimodal learning and analysis techniques broadly applicable to the fields of machine learning and computational perception that will place Canada at the forefront of interdisciplinary innovation.