Canada Spends Logo

Explore our Budget 2026 analysis

Grant: Discovery Grants Program - Individual

University of Waterloo

$28,000.00

Characterizing and Improving Data Quality in Very Large Databases

FY 2015-16

Summary
Data mining and data analysis have become standard practice in business and government organizations. However, analysis results are only as good as the input data. Unfortunately, the size and complexity of modern databases and information systems make it difficult to ensure data quality. In particular, inadequate understanding of the data semantics (leading to poor database design), data evolution over time (e.g., due to integrating new data sources) and error-prone collection may lead to data that are inconsistent, incorrect and incomplete. Understanding and monitoring data quality to avoid the ""garbage-in-garbage-out"" problem is an important and challenging issue. An additional challenge is that business-critical applications often collect continuous streams of data, which must be processed ""on the fly"" to support real-time decision making. Several data management technologies have recently been proposed to handle streaming data, including data stream management systems, stream data warehouses and event-processing systems. This proposal establishes a new research direction into methodologies and tools for streaming data quality. This includes models that describe the semantics of streaming data, and algorithms that incrementally detect and resolve data quality issues as new data arrive. Improving data stream quality will make stream processing systems more usable. Thus, the proposed research is of interest to businesses and government organizations that can gain a competitive advantage by making data-driven decisions in real time (e.g., healthcare, financial institutions, telecommunications companies, Web-based companies such as Google, Yahoo!, Facebook and Twitter, law enforcement, smart power grid management, and highway traffic management), and to database and information systems vendors (e.g., IBM, Oracle, Microsoft, Teradata, SAP, StreamBase Systems).
Program
Discovery Grants Program - Individual
Location
Ontario, Canada
Recipient Type
Academia
Type
Grant
Source
NSERC's Awards Data

As published

Application id
418547-2012
Name nom
Golab, Lukasz
Department département
Management Sciences
Organization id
33
Institution établissement
University of Waterloo
Province en
Ontario
Province fr
Ontario
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2015
Competition year année de concours
2012
Award amount
28000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
Discovery grants programs
Group fr
Programmes de subventions à la découverte
Committee code
1507
Committee name en
Computer Science
Committee name fr
Sciences informatiques
Area of application code
802
Area of application group en
Information and communication services
Area of application group fr
Services d'information et de télécommunications
Area of application en
Information systems and technology
Area of application fr
Systèmes d'information et technologie de l'information
Research subject code
2711
Research subject group en
Information technology
Research subject group fr
Technologie de l'information
Research subject en
Database management
Research subject fr
Gestion de bases de données
Partie
4
Nb partie
5
Application title
Characterizing and Improving Data Quality in Very Large Databases
Keyword
NSERC
Application summary
Data mining and data analysis have become standard practice in business and government organizations. However, analysis results are only as good as the input data. Unfortunately, the size and complexity of modern databases and information systems make it difficult to ensure data quality. In particular, inadequate understanding of the data semantics (leading to poor database design), data evolution over time (e.g., due to integrating new data sources) and error-prone collection may lead to data that are inconsistent, incorrect and incomplete. Understanding and monitoring data quality to avoid the ""garbage-in-garbage-out"" problem is an important and challenging issue. An additional challenge is that business-critical applications often collect continuous streams of data, which must be processed ""on the fly"" to support real-time decision making. Several data management technologies have recently been proposed to handle streaming data, including data stream management systems, stream data warehouses and event-processing systems. This proposal establishes a new research direction into methodologies and tools for streaming data quality. This includes models that describe the semantics of streaming data, and algorithms that incrementally detect and resolve data quality issues as new data arrive. Improving data stream quality will make stream processing systems more usable. Thus, the proposed research is of interest to businesses and government organizations that can gain a competitive advantage by making data-driven decisions in real time (e.g., healthcare, financial institutions, telecommunications companies, Web-based companies such as Google, Yahoo!, Facebook and Twitter, law enforcement, smart power grid management, and highway traffic management), and to database and information systems vendors (e.g., IBM, Oracle, Microsoft, Teradata, SAP, StreamBase Systems).