Canada Spends Logo

Explorez notre analyse du budget 2026

Grant: Discovery Grants Program - Individual

University of British Columbia

$29,000.00

Tractable Big Data and Big Models in Machine Learning

EF 2015-16

Résumé
In nearly all fields of science and engineering, the amount of data we collect is growing at unprecedented rates. We no longer produce data sizes from megabytes to gigabytes, but rather from terabytes to petabytes (and beyond). Machine learning is one of the key tools we use to make sense of these ever-growing quantities of data ('big data'), and it is now being used to solve very complicated tasks by fitting increasingly-complicated models to these huge data sets ('big models'). Machine learning is used in everyday technologies like e-mail spam filtering, product recommendation systems, advertisement ranking systems, new motion sensing devices, and recent improvements in speech recognition. There remains a huge potential for machine learning to impact applications ranging from physics to biology and from education technology to human-computer interaction. <span style=""line-height: 1.5em;"">The successes and potential of large-scale machine learning are driving the need to develop techniques that can consider constantly-increasing data and model sizes, and the objective<span style=""line-height: 1.5em;""> of this proposal is to advance the state of the art in fitting big models to big data sets. Building on my existing work showing that special model structures can be used to give order-of-magnitude improvements in runtimes,  the outcome of this research will be new techniques that are substantially faster than existing techniques (e.g., polynomial-time instead of exponential-time, or improving runtime by a<span style=""line-height: 1.5em;""> factor that may be as large as the data or model size). In particular, the research will focus on: <span style=""line-height: 1.5em;"">1. Improved incremental gradient methods: this extends our recent work on exponentially-convergent stochastic gradient methods, leading to methods that have faster convergence rates, as well as memory-free methods that avoid expensive full passes through the data. <span style=""line-height: 1.5em;"">2. Exploiting new problem structures: this thread seeks out new problem structures to exploit (such as graphs and non-quadratic surrogate functions), which will lead to the discovery of new tractable problem classes. <span style=""line-height: 1.5em;"">3. Parallel and distributed methods: this thread focuses on developing methods that have appealing theoretical and practical properties when implemented in a parallel and distributed settings, allowing scaling to much larger datasets. By focusing on improving the scalability of core enabling technologies behind data-driven applications, this work could affect a wide variety of scientific and engineering disciplines that produce huge datasets, in both academia and industry. <span style=""line-height: 1.5em;"">Further, this research will provide key training to graduate and undergraduate students. It will produce highly-qualified personnel with skills and experience in large-scale data analysis, with the theoretical background required to solve ever-larger problems and model even more complex phenomena. These skills will prove to be a major asset in the expansion of Canada's growing knowledge-based economy.
Programme
Discovery Grants Program - Individual
Emplacement
British Columbia, Canada
Type de bénéficiaire
Academia
Type
Grant
Source
NSERC's Awards Data

Tel que publié

Application id
RGPIN-2015-06068
Name nom
Schmidt, Mark
Department département
Computer Science - Computer Science
Organization id
2
Institution établissement
University of British Columbia
Province en
British Columbia
Province fr
Colombie-Britannique
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2015
Competition year année de concours
2015
Award amount
29000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
Discovery grants programs
Group fr
Programmes de subventions à la découverte
Committee code
1507
Committee name en
Computer Science
Committee name fr
Sciences informatiques
Area of application code
1207
Area of application group en
Advancement of knowledge
Area of application group fr
Avancement des connaissances
Area of application en
Information, computer and communication technologies
Area of application fr
Technologie de l'information, des ordinateurs et des réseaux de communications
Research subject code
2805
Research subject group en
Artificial intelligence
Research subject group fr
Intelligence artificielle
Research subject en
Learning and inference theories
Research subject fr
Théories de l'apprentissage et l'inférence
Partie
1
Nb partie
5
Application title
Tractable Big Data and Big Models in Machine Learning
Keyword
NSERC
Application summary
<p>In nearly all fields of science and engineering, the amount of data we collect is growing at unprecedented rates. We no longer produce data sizes from megabytes to gigabytes, but rather from terabytes to petabytes (and beyond). Machine learning is one of the key tools we use to make sense of these ever-growing quantities of data ('big data'), and it is now being used to solve very complicated tasks by fitting increasingly-complicated models to these huge data sets ('big models'). Machine learning is used in everyday technologies like e-mail spam filtering, product recommendation systems, advertisement ranking systems, new motion sensing devices, and recent improvements in speech recognition. There remains a huge potential for machine learning to impact applications ranging from physics to biology and from education technology to human-computer interaction. </p> <p><span style=""line-height: 1.5em;"">The successes and potential of large-scale machine learning are driving the need to develop techniques that can consider constantly-increasing data and model sizes, and the </span>objective<span style=""line-height: 1.5em;""> of this proposal is to advance the state of the art in fitting big models to big data sets. Building on my existing work showing that special model structures can be used to give order-of-magnitude improvements in runtimes,  the outcome of this research will be new techniques that are substantially faster than existing techniques (e.g., polynomial-time instead of exponential-time, or improving runtime </span>by a<span style=""line-height: 1.5em;""> factor that may be as large as the data or model size). In particular, the research will focus on:</span></p> <p><span style=""line-height: 1.5em;"">1. Improved incremental gradient methods: this extends our recent work on exponentially-convergent stochastic gradient methods, leading to methods that have faster convergence rates, as well as memory-free methods that avoid expensive full passes through the data.</span></p> <p><span style=""line-height: 1.5em;"">2. Exploiting new problem structures: this thread seeks out new problem structures to exploit (such as graphs and non-quadratic surrogate functions), which will lead to the discovery of new tractable problem classes.</span></p> <p><span style=""line-height: 1.5em;"">3. Parallel and distributed methods: this thread focuses on developing methods that have appealing theoretical and practical properties when implemented in a parallel and distributed settings, allowing scaling to much larger datasets.</span></p> <p>By focusing on improving the scalability of core enabling technologies behind data-driven applications, this work could affect a wide variety of scientific and engineering disciplines that produce huge datasets, in both academia and industry. <span style=""line-height: 1.5em;"">Further, this research will provide key training to graduate and undergraduate students. It will produce highly-qualified personnel with skills and experience in large-scale data analysis, with the theoretical background required to solve ever-larger problems and model even more complex phenomena. These skills will prove to be a major asset in the expansion of Canada's growing knowledge-based economy.</span></p>