Canada Spends Logo

Explore our Budget 2026 analysis

Grant: Discovery Grants Program - Individual

University of Toronto

$5,843.00

Practical and scalable algorithms for machine learning

FY 2009-10

Summary
Machine learning and statistical pattern recognition have been extremely influential in many scientific and engineering fields which require the analysis and interpretation of large amounts of data. However, many of the state of the art methods in statistical learning involve extremely complex algorithms which require large amounts of computation, both while building the systems ("training") and at implementation or runtime ("testing"). This significantly limits the scope of problem to which these techniques can be applied, because most algorithms simply cannot be run on large problems in reasonable time or with reasonable storage requirements even on state of the art computing platforms. While much has been gained from sophisticated treatments of machine learning problems, including Bayesian statistics, convex optimization, and Markov chain Monte Carlo computations, these paradigms have overshadowed the strengths of older, simpler but still extremely powerful approaches. In fact, in many large-scale real world applications such as web search, spam detection and computer vision the methods of choice are extremely simple but often customized to fit the particular domain. The objective of this research proposal is to focus on *extremely simple*, *highly scalable* and yet competitive and effective methods for machine learning problems including classification, clustering, alignment, record linkage, denoising and outlier detection. The approach will be to start with classic algorithms, such as nearest neighbour classification, logistic regression, Parzen density estimation, multi-layer backprop networks and to carefully examine under what circumstances their performance is significantly inferior to more sophisticated "state of the art" methods. Having identified their shortcomings, we will focus on understanding the fundamental origin of these deficiencies and correcting them by making modifications (potentially small but important) to the classic algorithms or by deriving new algorithms inspired by the simplicity and scalability of the old ones but designed to avoid their weaknesses.
Program
Discovery Grants Program - Individual
Location
Ontario, Canada
Recipient Type
Academia
Type
Grant
Source
NSERC's Awards Data

As published

Application id
250382-2007
Name nom
Roweis, Sam
Department département
Computer Science
Organization id
31
Institution établissement
University of Toronto
Province en
Ontario
Province fr
Ontario
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2009
Competition year année de concours
2007
Award amount
5843
Program id
RGPIN
Program nane en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
Discovery grants programs
Group fr
Programmes de subventions à la découverte
Committee code
331
Committee name en
Computing and Information Sciences - B
Committee name fr
Informatique - B
Area of application code
802
Area of application group en
Information and communication services
Area of application group fr
Services d'information et de télécommunications
Area of application en
Information systems and technology
Area of application fr
Systèmes d'information et technologie de l'information
Research subject code
2805
Research subject group en
Artificial intelligence
Research subject group fr
Intelligence artificielle
Research subject en
Learning and inference theories
Research subject fr
Théories de l'apprentissage et l'inférence
Partie
3
Nb partie
3
Application title
Practical and scalable algorithms for machine learning
Keyword
NA
Application summary
Machine learning and statistical pattern recognition have been extremely influential in many scientific and engineering fields which require the analysis and interpretation of large amounts of data. However, many of the state of the art methods in statistical learning involve extremely complex algorithms which require large amounts of computation, both while building the systems ("training") and at implementation or runtime ("testing"). This significantly limits the scope of problem to which these techniques can be applied, because most algorithms simply cannot be run on large problems in reasonable time or with reasonable storage requirements even on state of the art computing platforms. While much has been gained from sophisticated treatments of machine learning problems, including Bayesian statistics, convex optimization, and Markov chain Monte Carlo computations, these paradigms have overshadowed the strengths of older, simpler but still extremely powerful approaches. In fact, in many large-scale real world applications such as web search, spam detection and computer vision the methods of choice are extremely simple but often customized to fit the particular domain. The objective of this research proposal is to focus on *extremely simple*, *highly scalable* and yet competitive and effective methods for machine learning problems including classification, clustering, alignment, record linkage, denoising and outlier detection. The approach will be to start with classic algorithms, such as nearest neighbour classification, logistic regression, Parzen density estimation, multi-layer backprop networks and to carefully examine under what circumstances their performance is significantly inferior to more sophisticated "state of the art" methods. Having identified their shortcomings, we will focus on understanding the fundamental origin of these deficiencies and correcting them by making modifications (potentially small but important) to the classic algorithms or by deriving new algorithms inspired by the simplicity and scalability of the old ones but designed to avoid their weaknesses.