Canada Spends Logo

Explore our Budget 2026 analysis

Grant: Discovery Grants Program - Individual

University of British Columbia

$29,000.00

Tractable Big Data and Big Models in Machine Learning

FY 2017-18

Summary
In nearly all fields of science and engineering, the amount of data we collect is growing at unprecedented rates. We no longer produce data sizes from megabytes to gigabytes, but rather from terabytes to petabytes (and beyond). Machine learning is one of the key tools we use to make sense of these ever-growing quantities of data ('big data'), and it is now being used to solve very complicated tasks by fitting increasingly-complicated models to these huge data sets ('big models'). Machine learning is used in everyday technologies like e-mail spam filtering, product recommendation systems, advertisement ranking systems, new motion sensing devices, and recent improvements in speech recognition. There remains a huge potential for machine learning to impact applications ranging from physics to biology and from education technology to human-computer interaction. The successes and potential of large-scale machine learning are driving the need to develop techniques that can consider constantly-increasing data and model sizes, and the  objective  of this proposal is to advance the state of the art in fitting big models to big data sets. Building on my existing work showing that special model structures can be used to give order-of-magnitude improvements in runtimes,  the outcome of this research will be new techniques that are substantially faster than existing techniques (e.g., polynomial-time instead of exponential-time, or improving runtime  by a  factor that may be as large as the data or model size). In particular, the research will focus on: 1. Improved incremental gradient methods: this extends our recent work on exponentially-convergent stochastic gradient methods, leading to methods that have faster convergence rates, as well as memory-free methods that avoid expensive full passes through the data. 2. Exploiting new problem structures: this thread seeks out new problem structures to exploit (such as graphs and non-quadratic surrogate functions), which will lead to the discovery of new tractable problem classes. 3. Parallel and distributed methods: this thread focuses on developing methods that have appealing theoretical and practical properties when implemented in a parallel and distributed settings, allowing scaling to much larger datasets. By focusing on improving the scalability of core enabling technologies behind data-driven applications, this work could affect a wide variety of scientific and engineering disciplines that produce huge datasets, in both academia and industry.  Further, this research will provide key training to graduate and undergraduate students. It will produce highly-qualified personnel with skills and experience in large-scale data analysis, with the theoretical background required to solve ever-larger problems and model even more complex phenomena. These skills will prove to be a major asset in the expansion of Canada's growing knowledge-based economy.
Program
Discovery Grants Program - Individual
Location
British Columbia, Canada
Recipient Type
Academia
Type
Grant
Source
NSERC's Awards Data

As published

Application id
RGPIN-2015-06068
Name nom
Schmidt, Mark
Department département
Computer Science - Computer Science
Organization id
2
Institution établissement
University of British Columbia
Province en
British Columbia
Province fr
Colombie-Britannique
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2017
Competition year année de concours
2015
Award amount
29000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
Discovery grants programs
Group fr
Programmes de subventions à la découverte
Committee code
1507
Committee name en
Computer Science
Committee name fr
Sciences informatiques
Area of application code
1207
Area of application group en
Advancement of knowledge
Area of application group fr
Avancement des connaissances
Area of application en
Information, computer and communication technologies
Area of application fr
Technologie de l'information, des ordinateurs et des réseaux de communications
Research subject code
2805
Research subject group en
Artificial intelligence
Research subject group fr
Intelligence artificielle
Research subject en
Learning and inference theories
Research subject fr
Théories de l'apprentissage et l'inférence
Partie
3
Nb partie
5
Application title
Tractable Big Data and Big Models in Machine Learning
Keyword
NSERC
Application summary
In nearly all fields of science and engineering, the amount of data we collect is growing at unprecedented rates. We no longer produce data sizes from megabytes to gigabytes, but rather from terabytes to petabytes (and beyond). Machine learning is one of the key tools we use to make sense of these ever-growing quantities of data ('big data'), and it is now being used to solve very complicated tasks by fitting increasingly-complicated models to these huge data sets ('big models'). Machine learning is used in everyday technologies like e-mail spam filtering, product recommendation systems, advertisement ranking systems, new motion sensing devices, and recent improvements in speech recognition. There remains a huge potential for machine learning to impact applications ranging from physics to biology and from education technology to human-computer interaction. The successes and potential of large-scale machine learning are driving the need to develop techniques that can consider constantly-increasing data and model sizes, and the  objective  of this proposal is to advance the state of the art in fitting big models to big data sets. Building on my existing work showing that special model structures can be used to give order-of-magnitude improvements in runtimes,  the outcome of this research will be new techniques that are substantially faster than existing techniques (e.g., polynomial-time instead of exponential-time, or improving runtime  by a  factor that may be as large as the data or model size). In particular, the research will focus on: 1. Improved incremental gradient methods: this extends our recent work on exponentially-convergent stochastic gradient methods, leading to methods that have faster convergence rates, as well as memory-free methods that avoid expensive full passes through the data. 2. Exploiting new problem structures: this thread seeks out new problem structures to exploit (such as graphs and non-quadratic surrogate functions), which will lead to the discovery of new tractable problem classes. 3. Parallel and distributed methods: this thread focuses on developing methods that have appealing theoretical and practical properties when implemented in a parallel and distributed settings, allowing scaling to much larger datasets. By focusing on improving the scalability of core enabling technologies behind data-driven applications, this work could affect a wide variety of scientific and engineering disciplines that produce huge datasets, in both academia and industry.  Further, this research will provide key training to graduate and undergraduate students. It will produce highly-qualified personnel with skills and experience in large-scale data analysis, with the theoretical background required to solve ever-larger problems and model even more complex phenomena. These skills will prove to be a major asset in the expansion of Canada's growing knowledge-based economy.