Grant: Strategic Projects - Group

We propose to create new software technology for the exploration of large high-dimensional datasets, where the number of data points ranges from tens of thousands to millions. Our tools will help people develop insight into complex problems that depend on the relationships between many variables, ranging from tens to hundreds. Previous approaches to handling these problems assumed that exploration would only occur after an entire, finite dataset could be loaded into a software system. We will also address problem domains where the full dataset cannot simply be loaded; sampling must occur first, before interactive visualization or machine learning can be used to interpret it. This situation can occur for several reasons. Some datasets are inherently continuous and must be sampled to discretize them. Other datasets are finite but simply too large to handle in their entirety. In some application domains, the cost of taking a sample is so computationally expensive that only a minimal number of them should be taken. Our approach is based on a tight coupling of approaches from signal processing and statistical analysis, visualization, machine learning, and human-computer interaction. The collaborative interplay between these four approaches, where each is informed by the results to date on the others, and in turn informs the next stage of investigating the other three, will be critical to the success of our ambitious goals. We will i) develop new sampling strategies for high-dimensional spaces; ii) develop new dimensionality reduction techniques for fast interactive exploration; iii) develop new machine learning approaches for categorizing datasets; and iv) develop novel interaction techniques and user interfaces for these three approaches. We will also v) conduct both informal and formal evaluations to judge whether our targeted domain problems are being solved; and vi) demonstrate an integrated system as a platform for our experiments and a means for technology transfer to our partner companies.