Grant: Alliance Grants
McGill University
$45,000.00
Upsampling of low-resolution/large-volume 3D tomographic images using generative adversarial neural networks applied to biological anthropology, medical imaging, and evolutionary biology
EF 2022-23
Résumé
'It is vital to remember that information is not knowledge, that knowledge is not wisdom, and that wisdom is not foresight. But information is the first essential step to all of these', Sir Arthur C. Clarke.Large-volume versus high-resolution information is an inherent conundrum in all domains of research that rely on imaging; you can achieve either one, or the other, but never both. The incentives to increase resolution while keeping large context are many, from comprehensive analysis of basic biological phenomena, to maintaining radiation safety standards in imaging, to preserving the integrity of rare specimens of cultural value. However, extrapolating nonexisting spatial information has always been an ill-posed problem because its solution is not unique ' until now with the application of deep learning using neural networks. Although there is ample literature on upscaling single images using super-resolution convolutional neural networks (CNN), recurrent neural networks (RNN) or generative adversarial networks (GAN), most algorithms are designed for (and validated on) synthetically downsampled 2D images. Here we merge expertise in deep learning, tomographic X-ray imaging, biomineralization, biological anthropology, and dental radiology to design and validate an open-ended upsampling algorithm using 3D images of historical (ancient mummies), clinical (dental cone-beam computed tomography) and basic science (bird eggshell) samples. Unique to our project is the availability of original multi-scale 3D image sets that span multiple resolution/volume scales and which can be accurately superimposed (registered) in 3D, and can be expertly segmented (having meaningful features identified and assigned to the target class on a voxel basis). Images are hierarchical, meaning that the neighborhood of local features is as important as the features themselves; indeed, that is the basis of the CNN operation which identifies and labels features based on their context (and the context of context). GANs include a generator algorithm that constructs artificial features, and a discriminator algorithm that compares artificial and true features: iteration of the two leads to convergence, and to construction of realistic artificial spatial information. By applying GAN-CNN to a low-resolution/large-volume image using a high-resolution/low-volume image as ground truth, we will achieve 3D image upsampling ×n. To circumvent the inevitable increase of the data size (×n3) we will implement a parallel segmentation algorithm that reduces voxel depth, because the ultimate objective is the segmentation of upsampled images. Finally, we will explore the limit of image upsampling in 3D using registered series 3D images of ancient mummies, bird eggshells and human craniofacial complex, acquired at multiple magnifications. This open-ended study will bring biological anthropology, zoology and clinical radiology towards the new tier in bioimaging and 3D image analysis.
Programme
Alliance Grants
Bénéficiaire
Emplacement
Quebec, Canada
Type de bénéficiaire
Academia
Type
Grant
Source
NSERC's Awards Data
Tel que publié
Application id
571519-2021
Name nom
Reznikov, Natalie N
Department département
Bioengineering
Organization id
61
Institution établissement
McGill University
Province en
Québec
Province fr
Québec
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2022
Competition year année de concours
2021
Award amount
45000
Program id
ALLRP
Program name en
Alliance Grants
Program name fr
Subventions Alliance
Group en
RESEARCH PARTNERSHIPS
Group fr
PARTENARIATS DE RECHERCHE
Committee code
96
Committee name en
RPP Internal Decision Cttee
Committee name fr
Cté de décision interne - PPR
Area of application code
9999
Area of application group en
Not available
Area of application group fr
Non disponible
Area of application en
Not available
Area of application fr
Non disponible
Research subject code
8000
Research subject group en
Social sciences and humanities
Research subject group fr
Sciences humaines
Research subject en
Social sciences and humanities
Research subject fr
Sciences humaines
Installment
inst:2
Partie
1
Nb partie
2
Application title
Upsampling of low-resolution/large-volume 3D tomographic images using generative adversarial neural networks applied to biological anthropology, medical imaging, and evolutionary biology
Application summary
'It is vital to remember that information is not knowledge, that knowledge is not wisdom, and that wisdom is not foresight. But information is the first essential step to all of these', Sir Arthur C. Clarke.Large-volume versus high-resolution information is an inherent conundrum in all domains of research that rely on imaging; you can achieve either one, or the other, but never both. The incentives to increase resolution while keeping large context are many, from comprehensive analysis of basic biological phenomena, to maintaining radiation safety standards in imaging, to preserving the integrity of rare specimens of cultural value. However, extrapolating nonexisting spatial information has always been an ill-posed problem because its solution is not unique ' until now with the application of deep learning using neural networks. Although there is ample literature on upscaling single images using super-resolution convolutional neural networks (CNN), recurrent neural networks (RNN) or generative adversarial networks (GAN), most algorithms are designed for (and validated on) synthetically downsampled 2D images. Here we merge expertise in deep learning, tomographic X-ray imaging, biomineralization, biological anthropology, and dental radiology to design and validate an open-ended upsampling algorithm using 3D images of historical (ancient mummies), clinical (dental cone-beam computed tomography) and basic science (bird eggshell) samples. Unique to our project is the availability of original multi-scale 3D image sets that span multiple resolution/volume scales and which can be accurately superimposed (registered) in 3D, and can be expertly segmented (having meaningful features identified and assigned to the target class on a voxel basis). Images are hierarchical, meaning that the neighborhood of local features is as important as the features themselves; indeed, that is the basis of the CNN operation which identifies and labels features based on their context (and the context of context). GANs include a generator algorithm that constructs artificial features, and a discriminator algorithm that compares artificial and true features: iteration of the two leads to convergence, and to construction of realistic artificial spatial information. By applying GAN-CNN to a low-resolution/large-volume image using a high-resolution/low-volume image as ground truth, we will achieve 3D image upsampling ×n. To circumvent the inevitable increase of the data size (×n3) we will implement a parallel segmentation algorithm that reduces voxel depth, because the ultimate objective is the segmentation of upsampled images. Finally, we will explore the limit of image upsampling in 3D using registered series 3D images of ancient mummies, bird eggshells and human craniofacial complex, acquired at multiple magnifications. This open-ended study will bring biological anthropology, zoology and clinical radiology towards the new tier in bioimaging and 3D image analysis.