Grant: Discovery Grants Program - Individual

Scientists are gaining access to massive and growing data sets from many different sources, including: social media, scientific instruments, experiments, computer modelling, and cyber-physical systems. Data is growing from megabytes and gigabytes to terabytes, petabytes, and beyond. Managing and analyzing such vast quantities of data poses numerous and complex challenges including collecting, storing, and accessing data; reproducing and sharing experimental approaches; scaling computational resources to ensure timely processing; identifying and managing important meta-data; and integrating a broad range of analytic and visualization techniques. An urgent and pressing concern is the ability of software systems to handle and continuously respond to the ever-increasing amount and diversity of data and take advantage of new analytic and big data technologies in an area of rapid change. The overall goal of my research is to better understand and address the challenges of software evolution so that software developers can better respond to changing and emerging user needs and more readily exploit technological advances. The proposed research will explore software designs and services that support data intensive discovery; analysis techniques that allow us to better understand large, complex software systems and how they best evolve to support data intensive discovery; and, techniques for migrating software to take advantage of big data technologies and meet data intensive discovery needs. My short-term objectives are to (1) develop software designs and services to track and manage data and processing steps in big data environments; (2) create a reference software architecture and framework to guide software development for data intensive discovery; (3) create analysis and visualization tools and techniques to better understand workflow and its use in large, complex software systems; (4) create analysis and visualization techniques and tools to understand data use and component composition in large, complex software systems; (5) produce tools and techniques for migrating software to access new big data ready software libraries and services; and (6) produce tools and techniques for modularizing software to support adaptation for data intensive discovery. The Government of Canada estimates that there are over 39,000 Information and Communications Technology (ICT) firms in Canada with over $181 billion in annual revenues. The ICT sector is expanding at a rate of nearly twice that of the total Canadian economy and the software and computer sub-sector is experiencing the fastest GDP growth (+6.4% in 2016). The results of this research program are intended to improve the competitiveness of Canadian software firms in delivering innovative, quality, and usable software that readily evolves to meet emerging information processing advances and user needs. HQP trained are in high demand, not only as software experts but also as big data experts.