Grant: Discovery Grants Program - Individual

My research program focuses on the technological and algorithmic underpinnings that allow digital cameras to sense the physical world for photographic and scientific purposes. I am specifically interested in the processing steps - collectively termed the camera pipeline - that are applied by onboard digital cameras to convert the raw unprocessed sensor's response to scene radiance to the final output image. The current camera pipeline architecture has changed little in the last 20 years and uses a photo-centric design with the primary goal of producing visually pleasing photographs. However, the basic CMOS imaging sensor used in virtually all cameras, from DSLRs to smartphones, can capture accurate information about the physical environment suitable for scientific and medical applications. One of the high-level goals of my research program is to investigate novel camera pipeline designs that seamlessly enable dual-purpose functionality for both scientific and photographic applications. Towards this goal, my focus is primarily on smartphone camera pipelines, as smartphone platforms now represent the most common and accessible imaging means. Another core component of my research program is determining how best to leverage AI-based algorithms and deep neural networks to realize the "next-generation" neural camera pipeline design. The processing steps used in conventional photo-centric camera pipelines are primarily based on signal processing algorithms. AI-based methods, particularly neural networks, have shown exceptional performance for many conventional camera pipeline routines, such as image denoising, digital zoom, and multi-frame fusion. The shift from signal-processing pipelines to a neural camera pipeline offers new opportunities to rethink all aspects of the camera experience, from improved image quality for both photos and scientific applications to better image encoding formats and improved image compression. The long-term prospect of this research program is to shape the future of camera-based imaging technology for a wide range of applications.