Grant: Discovery Grants Program - Individual

Computers understand very little of the meaning of human language. However, language data permeates almost all aspects of our daily life, e.g., news, emails, social media, telephone conversations, medical records, books, just to name a few. Search engines like Google are only scratching the surface of human language, and yet the impact on society and the economy is already immense. Developing better semantic modelling technologies will have extensive impact on real-life applications. In general, machine understanding of natural language with human-level performance remains one of the grand challenges of artificial intelligence (AI), where modelling meaning is a major obstacle to achieving the goal. * The long-term objective of this program is to devise state-of-the-art computational models that learn better representations for text. Specifically, we focus on distributed representations, including those learned with neural network models, in which text is represented as real-valued vectors. If properly learned, such representations have been shown to be very effective, resulting in cutting-edge performance on a wide range of natural language processing (NLP) problems.* Building on our past contributions, this research program comprises three specific sets of short-term objectives for advancing distributed representations and neural network models for semantic representation. The first set aims to contribute novel algorithms for semantic composition. The principle of compositionality, positing that the meaning of a whole is a (complicated) function of the meaning of the parts, is a fundamental approach to learning representation for natural language. Compositionality is in general regarded by many as a basic ingredient of human intelligence. Based on our recent progress, this proposal aims to explore better neural-network-based semantic composition by further investigating external knowledge. In this set of objectives, we will also further explore non-compositionality, a basic phenomenon in language, in neural composition models. In the second set of objectives, we will delve into several specific aspects of semantics to deepen our understanding of distributed representation and neural-network-based modelling. While distributed representation has been shown to be very effective for modelling similarity and relatedness, its effectiveness for representing other key aspects of semantics, such as contrasting meaning and entailment, requires more investigation. The third set of objectives attempt to learn better distributed representation by further exploring syntactic structures. Language is rich in structures. It still remains a widely open question as to how to effectively consider such structures with the demonstrated modelling ability of distributed representation and neural networks.