Canada Spends Logo

Explorez notre analyse du budget 2026

Grant: Discovery Grants Program - Individual

University of Calgary

$46,000.00

Spatial Analysis of Audio Images for Computational Musicology

EF 2023-24

Résumé
Musicologists pursue the scholarly analysis of music with a variety of goals, including the desire to explore and understand creative musical processes. The evolving field of computational musicology brings computational methods to this pursuit. The proposed research program will develop computational methods around the investigation of creative musical processes in contemporary live electronic music that features spatial sound (i.e., rendered so that the listener can perceive the direction and location of a sound source, normally done with a surround speaker system) as part of the music. It builds on prior work done with musicologist colleagues (funded by SSHRC) and NSERC-DG-funded work on image and signal processing, and sound synthesis. The emphasis on spatial sound means that work will focus on sound-field data - in our case captured by a high-order ambisonic microphone (i.e., spherical array with a large number of microphone elements distributed over the sphere). To illustrate, consider the work of Clarke et al. (2013) - they analyze a spectrogram (an image showing sound decomposed in frequency and time) of a fragment of a musical performance by manually identifying parts of the image that correspond to salient musical objects, ultimately building a symbolic representation of the music. The proposed research follows Clarke's paradigm, but with the following important differences. First, we will bring our experience in computer vision and image/signal processing to automate the identification of salient musical objects. The notion of saliency is not trivial - I will be guided by both musicological knowledge of the subject matter and progress in the field of computer vision. Second, we will focus on spatial sound, building and relying on a microphone array and high-order ambisonic recordings. This allows us to follow the lead of O'Donovan et al. (2007) in using the microphone as an audio camera, making it possible to process two-dimensional audio data on a sphere as an image. Finally, we will be exploiting our growing database of live-spatial-electronic music recordings. We now have close to 80 GiB of high-order ambisonic recordings of contemporary spatial electronic music, which constitutes a unique resource in this type of research. While the focus of the research is computational musicology, the research has broader applications. HQP will be well equipped to work in any of the emerging fields that need sound field analysis.  Two important application areas are environmental noise assessment and amelioration, and virtual reality (VR) systems. A local sound engineering firm is already engaging with us to collaborate on measurement and analysis of industrial noise. Sound systems that will accompany VR graphics are in their infancy. HQP expertise in sound fields will be essential in the future of VR, and my HQP will have the added advantage of collaboration opportunities with experts in cultural applications of VR.
Programme
Discovery Grants Program - Individual
Bénéficiaire
Emplacement
Alberta, Canada
Type de bénéficiaire
Academia
Type
Grant
Source
NSERC's Awards Data

Tel que publié

Application id
RGPIN-2019-03974
Name nom
Boyd, Jeffrey
Department département
Computer Science
Organization id
11
Institution établissement
University of Calgary
Province en
Alberta
Province fr
Alberta
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2023
Competition year année de concours
2019
Award amount
46000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
1507
Committee name en
Computer Science
Committee name fr
Sciences informatiques
Area of application code
802
Area of application group en
Information and communication services
Area of application group fr
Services d'information et de télécommunications
Area of application en
Information systems and technology
Area of application fr
Systèmes d'information et technologie de l'information
Research subject code
2514
Research subject group en
Electrical and electronic engineering
Research subject group fr
Digital signal processing
Research subject en
Génie électrique et électronique
Research subject fr
Traitement des signaux numériques
Application title
Spatial Analysis of Audio Images for Computational Musicology
Keyword
ambisonics; computational musicology; computer vision; sound images; spatial audio
Application summary
Musicologists pursue the scholarly analysis of music with a variety of goals, including the desire to explore and understand creative musical processes. The evolving field of computational musicology brings computational methods to this pursuit. The proposed research program will develop computational methods around the investigation of creative musical processes in contemporary live electronic music that features spatial sound (i.e., rendered so that the listener can perceive the direction and location of a sound source, normally done with a surround speaker system) as part of the music. It builds on prior work done with musicologist colleagues (funded by SSHRC) and NSERC-DG-funded work on image and signal processing, and sound synthesis. The emphasis on spatial sound means that work will focus on sound-field data - in our case captured by a high-order ambisonic microphone (i.e., spherical array with a large number of microphone elements distributed over the sphere). To illustrate, consider the work of Clarke et al. (2013) - they analyze a spectrogram (an image showing sound decomposed in frequency and time) of a fragment of a musical performance by manually identifying parts of the image that correspond to salient musical objects, ultimately building a symbolic representation of the music. The proposed research follows Clarke's paradigm, but with the following important differences. First, we will bring our experience in computer vision and image/signal processing to automate the identification of salient musical objects. The notion of saliency is not trivial - I will be guided by both musicological knowledge of the subject matter and progress in the field of computer vision. Second, we will focus on spatial sound, building and relying on a microphone array and high-order ambisonic recordings. This allows us to follow the lead of O'Donovan et al. (2007) in using the microphone as an audio camera, making it possible to process two-dimensional audio data on a sphere as an image. Finally, we will be exploiting our growing database of live-spatial-electronic music recordings. We now have close to 80 GiB of high-order ambisonic recordings of contemporary spatial electronic music, which constitutes a unique resource in this type of research. While the focus of the research is computational musicology, the research has broader applications. HQP will be well equipped to work in any of the emerging fields that need sound field analysis.  Two important application areas are environmental noise assessment and amelioration, and virtual reality (VR) systems. A local sound engineering firm is already engaging with us to collaborate on measurement and analysis of industrial noise. Sound systems that will accompany VR graphics are in their infancy. HQP expertise in sound fields will be essential in the future of VR, and my HQP will have the added advantage of collaboration opportunities with experts in cultural applications of VR.