Grant: Engage Grants Program
University of New Brunswick
$25,000.00
Automatically building vocabularies from web forum text
FY 2015-16
Summary
A staggering amount of text is written online every day, much of it in discussions taking place in online forums
focused on topics such as motorsports, pets, and collectibles. This provides an opportunity for businesses to
analyze this text to glean insights into consumer demands and opinions, which can then give a business a
competitive advantage. Due to the large volume of data, the analyses of such conversations would ideally be
done automatically. The key to building an automatic text analysis system is a vocabulary of important terms.
Such a vocabulary could be taken from a dictionary; however, because of the specialized topics that online
forums focus on, many important terms are not listed in general dictionaries. For example, online forums
contain many non-standard words, including intentional variant spellings (e.g., ""coooolll"") and novel
combinations (e.g., ""defuel""), as well as noise (e.g., usernames) and unintentional spelling errors.
The goal of this project is to identify the above categories of words in order to automatically build a vocabulary
of key terms for an online forum community. The project will apply a range of statistical techniques that draw
on information including word frequency, character sequences in words, and the contexts in which words
appear. The automatic construction of such a vocabulary will help improve the performance of the subsequent
text processing and analysis tools that try to zero in on relevant and important pieces of information given the
vast amounts of noisy and unstructured data in online forums. In turn, such tools can improve the quality of the
online services that VerticalScope provides to its forum users, as well as the attractiveness of VerticalScope's
services to various businesses. These help VerticalScope become a leading player in data science research and
development, hence benefiting the Canadian economy. This will further benefit the broader public by enabling
businesses to better understand consumer needs to develop more-attractive products.
Program
Engage Grants Program
Recipient
Location
New Brunswick, Canada
Recipient Type
Academia
Type
Grant
Source
NSERC's Awards Data
As published
Application id
485322-2015
Name nom
Cook, ChristopherPaul
Department département
Computer Science, Faculty of
Organization id
75
Institution établissement
University of New Brunswick
Province en
New Brunswick
Province fr
Nouveau-Brunswick
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2015
Competition year année de concours
2015
Award amount
25000
Program id
EGP
Program name en
Engage Grants Program
Program name fr
Programme de subventions d'engagement partenariat
Group en
Research partnerships programs
Group fr
Programmes de partenariat
Committee code
1551
Committee name en
Atlantic Internal Decision Committee
Committee name fr
Comité de décision interne pour l'Atlantique
Area of application code
1207
Area of application group en
Advancement of knowledge
Area of application group fr
Avancement des connaissances
Area of application en
Information, computer and communication technologies
Area of application fr
Technologie de l'information, des ordinateurs et des réseaux de communications
Research subject code
2801
Research subject group en
Artificial intelligence
Research subject group fr
Intelligence artificielle
Research subject en
Natural language and speech understanding
Research subject fr
Langage naturel et reconnaissance de la parole
Partie
1
Nb partie
1
Application title
Automatically building vocabularies from web forum text
Keyword
NSERC
Application summary
A staggering amount of text is written online every day, much of it in discussions taking place in online forums
focused on topics such as motorsports, pets, and collectibles. This provides an opportunity for businesses to
analyze this text to glean insights into consumer demands and opinions, which can then give a business a
competitive advantage. Due to the large volume of data, the analyses of such conversations would ideally be
done automatically. The key to building an automatic text analysis system is a vocabulary of important terms.
Such a vocabulary could be taken from a dictionary; however, because of the specialized topics that online
forums focus on, many important terms are not listed in general dictionaries. For example, online forums
contain many non-standard words, including intentional variant spellings (e.g., ""coooolll"") and novel
combinations (e.g., ""defuel""), as well as noise (e.g., usernames) and unintentional spelling errors.
The goal of this project is to identify the above categories of words in order to automatically build a vocabulary
of key terms for an online forum community. The project will apply a range of statistical techniques that draw
on information including word frequency, character sequences in words, and the contexts in which words
appear. The automatic construction of such a vocabulary will help improve the performance of the subsequent
text processing and analysis tools that try to zero in on relevant and important pieces of information given the
vast amounts of noisy and unstructured data in online forums. In turn, such tools can improve the quality of the
online services that VerticalScope provides to its forum users, as well as the attractiveness of VerticalScope's
services to various businesses. These help VerticalScope become a leading player in data science research and
development, hence benefiting the Canadian economy. This will further benefit the broader public by enabling
businesses to better understand consumer needs to develop more-attractive products.