Canada Spends Logo

Explorez notre analyse du budget 2026

Grant: Engage Grants Program

École Polytechnique de Montréal

$25,000.00

Reverse Engineering a Data Lake for Data Science******

EF 2018-19

Résumé
As a major private clinic treating HIV, hepatitis and drug addiction, the Clinique Médecine Urbaine of the Quartier Latin (""CMUQL""), envisions to systematically identify epidemiological trends about their patients' diseases, without human intervention. Concretely, CMUQL plans to migrate its legacy research infrastructure to a data science platform (DSP) based on a modern lambda architecture, advanced distributed computation technologies like MapReduce and Spark, and on a data lake.* Unfortunately, this migration faces a number of important challenges in order not to end up with a ""big data graveyard"". First, reverse engineering towards a data lake is still in its infancy. Second, CMUQL's data lake needs to integrate data from a third party provider with at least 2 other data sources: a decade-old paper archive of hand-written, copied patient documents (including medical images), with inconsistent format and terminology; and an ad hoc Microsoft Access/FileMaker database, partially based on the other two data sources. Third, due to medical reasons, for any transformation, calculation or analysis made on the (anonymized) patient data, an audit trail should be traceable back to the input data.* As a first step towards a modern DSP, we aim to mine and reverse-engineer the current data sources into a NoSQL-based data lake. One PhD student and one professional Master's student will:* 1. make an inventory of data sources and their structure* 2. design the basic data lake architecture* 3. prototype a data lake for the existing Access database* The data inventory and data lake prototype delivered by this project will enable CMUQL to explore and start understand diseases menacing thousands of Canadian patients. Second, the project allows 2 HQP to obtain unique research experience with reverse engineering towards a data lake, modern data science architectures and mining (un)structured data.*
Programme
Engage Grants Program
Emplacement
Quebec, Canada
Type de bénéficiaire
Academia
Type
Grant
Source
NSERC's Awards Data

Tel que publié

Application id
537315-2018
Name nom
Adams, Bram
Department département
Génie informatique et génie logiciel
Organization id
64
Institution établissement
École Polytechnique de Montréal
Province en
Québec
Province fr
Québec
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2018
Competition year année de concours
2018
Award amount
25000
Program id
EGP
Program name en
Engage Grants Program
Program name fr
Programme de subventions d'engagement partenariat
Group en
Research partnerships programs
Group fr
Programmes de partenariat
Committee code
1552
Committee name en
Quebec Internal Decision Committee
Committee name fr
Comité de décision interne pour le Québec
Area of application code
802
Area of application group en
Information and communication services
Area of application group fr
Services d'information et de télécommunications
Area of application en
Information systems and technology
Area of application fr
Systèmes d'information et technologie de l'information
Research subject code
2706
Research subject group en
Information technology
Research subject group fr
Technologie de l'information
Research subject en
Software engineering
Research subject fr
Génie logiciel
Partie
1
Nb partie
1
Application title
Reverse Engineering a Data Lake for Data Science******
Keyword
NSERC
Application summary
As a major private clinic treating HIV, hepatitis and drug addiction, the Clinique Médecine Urbaine of the Quartier Latin (""CMUQL""), envisions to systematically identify epidemiological trends about their patients' diseases, without human intervention. Concretely, CMUQL plans to migrate its legacy research infrastructure to a data science platform (DSP) based on a modern lambda architecture, advanced distributed computation technologies like MapReduce and Spark, and on a data lake.* Unfortunately, this migration faces a number of important challenges in order not to end up with a ""big data graveyard"". First, reverse engineering towards a data lake is still in its infancy. Second, CMUQL's data lake needs to integrate data from a third party provider with at least 2 other data sources: a decade-old paper archive of hand-written, copied patient documents (including medical images), with inconsistent format and terminology; and an ad hoc Microsoft Access/FileMaker database, partially based on the other two data sources. Third, due to medical reasons, for any transformation, calculation or analysis made on the (anonymized) patient data, an audit trail should be traceable back to the input data.* As a first step towards a modern DSP, we aim to mine and reverse-engineer the current data sources into a NoSQL-based data lake. One PhD student and one professional Master's student will:* 1. make an inventory of data sources and their structure* 2. design the basic data lake architecture* 3. prototype a data lake for the existing Access database* The data inventory and data lake prototype delivered by this project will enable CMUQL to explore and start understand diseases menacing thousands of Canadian patients. Second, the project allows 2 HQP to obtain unique research experience with reverse engineering towards a data lake, modern data science architectures and mining (un)structured data.*