Grant: Discovery Grants Program - Individual

Together, neurons and synapses in a typical human brain expend energy at a rate of approximately 20?W. Contrastingly, a digital simulation of a commensurate artificial neural network consumes 7.9?MW, larger by a million fold. Within the perspective of the multi-million dollar energy budget for training Generative Pre-trained Transformers (GPTs), such a gap between biological and artificial energy expenditure poses challenges us to find a more efficient solution. It is estimated that between 10-1000 times less energy can be achieved by transitioning to spiking neuromorphic architectures where memory and computation are colocalized. Currently, however, learning algorithms to exploit these new devices for machine learning are lacking. Previous NSERC-funded research in our lab uncovered a learning algorithm based on the physiology of neurons, dendrites and synapses. Here, we propose to build on the core idea that animates this `burstprop' algorithm, and engineer adaptations for neuromorphic hardware. Ultimately, the algorithm would aim to reduce the energy cost of training large-scale machine learning models such those used in computer vision. To achieve this goal, I propose to integrate three mechanisms: homeostatic plasticity for enhancing stability, attention mechanisms for enhancing computational efficiency, and predictive coding for enhancing coding efficiency. These mechanisms are known in either the neuroscience literature or the machine learning literature to be highly important for allowing neural networks to perform. These research goals will be pursued by researchers joining a team that values collegiality, diversity and openness. Our international and gender-diverse team combines researchers with expertise in physics, mathematics, neuroscience and computer science. As central part of the Center for Neural Dynamics and AI, our team interacts closely with numerous other interdisciplinary groups at the University of Ottawa and beyond, providing a fertile training ground for highly qualified personnel. The outcomes of the proposed research will allow us to better evaluate the energy efficiency gains to be expected from dendrocentric learning algorithms and spiking neural networks. These results, when combined with actual neuromorphic hardware, may lead to further reductions in the energy consumption of many artificial intelligence models.