Canada Spends Logo

Explore our Budget 2026 analysis

Grant: Discovery Grants Program - Individual

Université Laval

$42,000.00

Innovative deep learning solutions leveraging unlabeled and multi-source annotated datasets to segment and monitor change in large scale 3D point clouds for Digital Twin Cities

FY 2024-25

Summary
A Digital Twin City (DTC) is a virtual representation of physical assets across their respective life cycles in a city. This is an emerging concept that has become the centre of attention for academia in more recent years. The rapid development of mobile LiDAR system (MLS) provides large-scale, accurate and affordable 3D point clouds (PCs) at a city-scale. These data offer high-quality geometric measurements but limited non-geometric semantics, such as object type and topological relationships. Therefore, semantic segmentation is a fundamental perception task for the massive datasets used in DTC creation. In recent years, the development of deep neural networks (DNNs) has led to great success in 3D PC semantic segmentation while focusing on designing fully supervised networks for densely annotated data. However, such massive 3D point-wise annotation is time-consuming, labour-intensive, and error-prone. As a result, current effort should be in developing efficient methods for creating more labels or learning from cheap supervision (i.e. little annotation or simulated data). DTC involves 4D dynamic objects. Since LiDAR provides a snapshot of the DTC at a certain timestamp, there is a need to detect and understand changes in multi-temporal LiDAR datasets. Performing change detection in street-level PC streams is often a significantly complex task. One must expect a large variety of object shapes and appearances, numerous occlusion artifacts between the different objects, and a lack of accurate registration between the compared datasets. There is a crucial lack of studies using DNNs to address this problem. Furthermore, DTC creation should be addressed with a focus on automatic updating because a city is constantly changing. Existing urban 3D modelling methods are limited in this respect. By extending our on-going research, the grand goal for this research program is to design a deep learning-based framework for large-scale outdoor point clouds to accommodate the need for Digital Twin City creation and monitoring over time. The intent is: 1) to develop innovative deep learning approaches to be less dependent on annotated data and better able to leverage datasets with different characteristics, even unlabelled or simulated ones; 2) to propose solutions to detect and monitor changes in urban environments at the point level; 3) to design new approaches able to learn the spatial and dynamic structures of urban environments in order to propose suitable and consistent 3D reconstructions. This innovative research will be instrumental towards the knowledge advancement of deep learning applied to large-scale outdoor PCs and 3D PCs contexts that are under-explored in literature often dedicated to autonomous vehicles. This should build new capacity to create digital twin cities enabling better, faster decisions for various applications (ex. disaster recovery operation; carbon neutral mobility).
Program
Discovery Grants Program - Individual
Location
Quebec, Canada
Recipient Type
Academia
Type
Grant
Source
NSERC's Awards Data

As published

Application id
RGPIN-2023-04507
Name nom
Daniel, Sylvie
Department département
sciences géomatiques
Organization id
51
Institution établissement
Université Laval
Province en
Québec
Province fr
Québec
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2024
Competition year année de concours
2023
Award amount
42000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
1509
Committee name en
Civil, Industrial and Systems Engineering
Committee name fr
Génie civil, industriel et des systèmes
Area of application code
604
Area of application group en
Construction, urban and rural planning
Area of application group fr
Construction, urbanisme et aménagement rural
Area of application en
Surveying and photogrammetry
Area of application fr
Arpentage et photogrammétrie
Research subject code
4102
Research subject group en
Geographical information
Research subject group fr
Information géographique
Research subject en
Remote sensing
Research subject fr
Télédétection
Field of research list names en
Not available
Field of research list names fr
Non disponible
Socioeconomic objective list names en
Not available
Socioeconomic objective list names fr
Non disponible
Application title
Innovative deep learning solutions leveraging unlabeled and multi-source annotated datasets to segment and monitor change in large scale 3D point clouds for Digital Twin Cities
Keywords
3d point cloud; semantic segmentation; change detection; occlusion detection; deep learning; unsupervised learning; domain adaptation; digital twin city; 3d mapping; mobile laser scanning
Application summary
A Digital Twin City (DTC) is a virtual representation of physical assets across their respective life cycles in a city. This is an emerging concept that has become the centre of attention for academia in more recent years. The rapid development of mobile LiDAR system (MLS) provides large-scale, accurate and affordable 3D point clouds (PCs) at a city-scale. These data offer high-quality geometric measurements but limited non-geometric semantics, such as object type and topological relationships. Therefore, semantic segmentation is a fundamental perception task for the massive datasets used in DTC creation. In recent years, the development of deep neural networks (DNNs) has led to great success in 3D PC semantic segmentation while focusing on designing fully supervised networks for densely annotated data. However, such massive 3D point-wise annotation is time-consuming, labour-intensive, and error-prone. As a result, current effort should be in developing efficient methods for creating more labels or learning from cheap supervision (i.e. little annotation or simulated data). DTC involves 4D dynamic objects. Since LiDAR provides a snapshot of the DTC at a certain timestamp, there is a need to detect and understand changes in multi-temporal LiDAR datasets. Performing change detection in street-level PC streams is often a significantly complex task. One must expect a large variety of object shapes and appearances, numerous occlusion artifacts between the different objects, and a lack of accurate registration between the compared datasets. There is a crucial lack of studies using DNNs to address this problem. Furthermore, DTC creation should be addressed with a focus on automatic updating because a city is constantly changing. Existing urban 3D modelling methods are limited in this respect. By extending our on-going research, the grand goal for this research program is to design a deep learning-based framework for large-scale outdoor point clouds to accommodate the need for Digital Twin City creation and monitoring over time. The intent is: 1) to develop innovative deep learning approaches to be less dependent on annotated data and better able to leverage datasets with different characteristics, even unlabelled or simulated ones; 2) to propose solutions to detect and monitor changes in urban environments at the point level; 3) to design new approaches able to learn the spatial and dynamic structures of urban environments in order to propose suitable and consistent 3D reconstructions. This innovative research will be instrumental towards the knowledge advancement of deep learning applied to large-scale outdoor PCs and 3D PCs contexts that are under-explored in literature often dedicated to autonomous vehicles. This should build new capacity to create digital twin cities enabling better, faster decisions for various applications (ex. disaster recovery operation; carbon neutral mobility).