Grant: Discovery Grants Program - Individual

Ancient manuscripts constitute a primary carrier of cultural heritage globally, and they are currently being intensively digitized all over the world to ensure their preservation, and, ultimately, wide accessibility to their content. Critical to this research process are the legibility of the documents in image form and access to live texts. Several state-of-the-art methods and approaches have been proposed and developed to address the challenges associated with processing these manuscripts. However, there is a huge amount of data involved, and also the high cost and scarcity of human expert feedback and reference data call for the development of fundamental approaches that encompass all these aspects in an objective and tractable manner.*In this research, we propose one such approach, which is a novel framework for the computational pattern analysis of ancient manuscripts that is data-driven, multilevel, self-sustaining and learning-based, and takes advantage of the large quantities of unprocessed data available. Unlike many approaches, which fast-forward to the analysis of feature vectors, our proposed framework represents a new perspective on the task, which starts from ground zero of the problem, which is the definition of objects. In addition, it leverages the data-driven mining of relations among objects to discover hidden but persistent links between them. The problem is addressed at three main levels. At the lowest level, that of images, we tackle the automatic, data-driven enhancement and restoration of document images using spatial, spectral, sparse and graph-based representations of visual objects with a focus on spatial graphs of patch-based representations empowered with spatial proximity and local similarities. In terms of degradation modeling, our approach is to perform a cluster analysis on the data point manifolds. Transfer learning approaches will be also used to discover, model and correct unseen degradation holistically. At the second level, which is transliteration, we use directed graphical models, HMMs, Undirected Random Fields and spatial relations models of patch-based representations in an active learning framework to recognize the live text in manuscript images, in an effort to drastically reduce dependency on human experts and on reference data that are rarely available. In addition, cross-lingual approaches to adaptation in spoken language translations, such as transform mapping at state level, are also considered, in order to allow adaptation across writing styles and even across written languages. Finally, at the highest level, that of network analysis of the relations among objects (from patches and words to manuscripts and writers), we search for `social networks' linking manuscripts. Considering this data-driven approach under the heading of Visual Language Processing (VLP), we hope that it will pave the way for new research in Canada's upcoming data stewardship plan.*This research program will lead to novel paradigms for processing and understanding ancient manuscripts using coherent, data-driven frameworks with tractable solutions. The multi-level structure of the proposed approach enables researchers to collaboratively mine, model and interpret digitized manuscripts, all of which can be achieved thanks to data-driven approaches, which have been largely absent from the field up to now. Empowered by the concept of life-long learning, our research makes the methods and models developed transferable across collections. It will advance the paradigms of image processing, pattern recognition, machine learning and network science, with the potential to impact huge collections of manuscripts and leverage different representation spaces and metrics, and their associated relationships and links.