Grant: Discovery Grants Program - Individual

This research concentrates on the foundations of reliable modeling and evaluation in learning from data. Many data or system structures are complex, ranging from modern energy infrastructures to social systems and biomedical signals. Investigating on accurate and efficient information extraction methods from available data and addressing the critical issue of data--driven model evaluation, validation and reliability are currently challenging tasks. Recent machine learning advances, such as deep learning, have provided impressive results in learning from large data sets. However, currently these algorithms are becoming more complex in both structure and computation and the growth is faster than being able to thoroughly validate the underlying performance. The reliability and convergence of these algorithms concerns the artificial intelligence community. What differentiates Machine Learning from optimization is in the concept of generalization and ability to satisfy the same minimization behavior of loss function over the test data and the unseen data. To satisfy the generalization property the richness of training data set, richness of the competing hypothesis classes and proper formulation of the loss function are essential. On the other hand, due to the advances in data storage and computing, computational complexity and efficiency issues are now marginalized and partially ignored. If not addressed properly and promptly, this problem will soon be a major issue of curse of dimensionality. The proper choice of hyperparameters is a critically relevant issue in this context. Here adaptive learning is to automate a layer of involved hyperparameters. Generalization approaches such as cross validation are heuristic and computationally extensive. On the other hand, dimensionality evaluation methods such as regularization approaches are disconnected from validation procedure with variety of the additional penalty terms in the regularization cost function. Throughout the modeling process many hyperparameters are still tuned manually or obtained through heuristic and computationally extensive grid search methods. The focus of this work is on proposing reliable and efficient methods for evaluation of the learning procedure that simultaneously provides quantified measures for the generalization and the hyperparameter (dimensionality) selection. The method consequently provides an efficient automatic hyperparameter tuning approach. Our primary results in this direction for linear regression show that model complexity in this form of data-driven learning does not have an absolute definition. Rather complexity here is a relative term that is a function of data characteristics such as its length or level of corruption, feature size and the hypothesis parameters. Our work will continue providing measures and evaluation of the relative complexity in the considered data-driven modeling procedure.