Canada Spends Logo

Explorez notre analyse du budget 2026

Grant: Discovery Grants Program - Individual

École Polytechnique de Montréal

$36,000.00

Knowledge assessment from artifacts

EF 2024-25

Résumé
This research program aims to develop data-driven models and algorithms to represent and assess the expertise level of authors of documents within a set of topics, as well as authors of program code in different technical disciplines. Conversely, this task amounts to assessing the disciplinary depth of a document or program code. Considering an expert can readily tell if a given text or program is written by a novice or an expert, and that this task is generally out of the reach of algorithms, we believe there is a solid opportunity to apply machine learning approaches towards efficient disciplinary depth assessment and its potential applications. The research program will rely in part on supervised algorithms that require labeled data. The first objective of the program is to generate such labeled data with generative AI techniques. Large Language Models (LLM) are now capable of generating realistic text and program code that can be tailored on demand to correspond to a given level of expertise over a single, or multiple disciplines. This capability offers an unprecedented opportunity to generate labeled data over a space of expertise and disciplines that compensates for the lack of labeled datasets which hindered research efforts to create models of disciplinary depth. Efforts will be conducted to assess the validity of such synthetic data. Another objective of the program is to extract from data what are the discriminant keywords and expressions that characterize levels of disciplinary depth. We will leverage known and effective features such as TF-IDF and entropy to infer the discriminant keywords for that purpose, and investigate how these features can be relevant to LLM for disciplinary depth assessment. For example, while existing word embeddings are very useful for revealing semantic similarity, they lack the representation of how discriminant a word is, and if that word represents highly disciplinary jargon that requires prerequisite knowledge to correctly grasp it meaning. A related objective of the program is the inference of the prerequisite structure of concepts within a discipline. We obtained encouraging results using a combination of transformer architecture and graph convolution network that can be further investigated. It can benefit from the work on building labeled synthetic data and can, in turn, benefit the objective of providing disciplinary depth characteristics for the previous objective. In addition to labeled approaches, we also plan to pursue work my team has done on unsupervised learning of disciplinary depth with the Latent Dirichlet Allocation extension that incorporates a specific parameter to represent disciplinary depth. The outcome of this research program is expected to have impacts in a wide array of applications such as programming bug assignments to qualified developers, didactic content sequencing in intelligent learning environments, technical project staffing, just to name a few examples.
Programme
Discovery Grants Program - Individual
Emplacement
Quebec, Canada
Type de bénéficiaire
Academia
Type
Grant
Source
NSERC's Awards Data

Tel que publié

Application id
RGPIN-2024-06617
Name nom
Desmarais, Michel
Department département
Génie informatique et génie logiciel
Organization id
64
Institution établissement
École Polytechnique de Montréal
Province en
Québec
Province fr
Québec
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2024
Competition year année de concours
2024
Award amount
36000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
1507
Committee name en
Computer Science
Committee name fr
Sciences informatiques
Area of application code
1103
Area of application group en
Health, education and social services
Area of application group fr
Santé publique, éducation et services sociaux
Area of application en
Learning and education
Area of application fr
Apprentissage et éducation
Research subject code
2803
Research subject group en
Artificial intelligence
Research subject group fr
Intelligence artificielle
Research subject en
Knowledge representation
Research subject fr
Répresentation des connaissances
Field of research list names en
Not available
Field of research list names fr
Non disponible
Socioeconomic objective list names en
Not available
Socioeconomic objective list names fr
Non disponible
Application title
Knowledge assessment from artifacts
Keywords
knowledge assessment; cognitive modeling; intelligent and adaptive interfaces; topic modeling; keyword extraction; learning analytics; natural langage processing; intelligent tutoring systems; statistical learning
Application summary
This research program aims to develop data-driven models and algorithms to represent and assess the expertise level of authors of documents within a set of topics, as well as authors of program code in different technical disciplines. Conversely, this task amounts to assessing the disciplinary depth of a document or program code. Considering an expert can readily tell if a given text or program is written by a novice or an expert, and that this task is generally out of the reach of algorithms, we believe there is a solid opportunity to apply machine learning approaches towards efficient disciplinary depth assessment and its potential applications. The research program will rely in part on supervised algorithms that require labeled data. The first objective of the program is to generate such labeled data with generative AI techniques. Large Language Models (LLM) are now capable of generating realistic text and program code that can be tailored on demand to correspond to a given level of expertise over a single, or multiple disciplines. This capability offers an unprecedented opportunity to generate labeled data over a space of expertise and disciplines that compensates for the lack of labeled datasets which hindered research efforts to create models of disciplinary depth. Efforts will be conducted to assess the validity of such synthetic data. Another objective of the program is to extract from data what are the discriminant keywords and expressions that characterize levels of disciplinary depth. We will leverage known and effective features such as TF-IDF and entropy to infer the discriminant keywords for that purpose, and investigate how these features can be relevant to LLM for disciplinary depth assessment. For example, while existing word embeddings are very useful for revealing semantic similarity, they lack the representation of how discriminant a word is, and if that word represents highly disciplinary jargon that requires prerequisite knowledge to correctly grasp it meaning. A related objective of the program is the inference of the prerequisite structure of concepts within a discipline. We obtained encouraging results using a combination of transformer architecture and graph convolution network that can be further investigated. It can benefit from the work on building labeled synthetic data and can, in turn, benefit the objective of providing disciplinary depth characteristics for the previous objective. In addition to labeled approaches, we also plan to pursue work my team has done on unsupervised learning of disciplinary depth with the Latent Dirichlet Allocation extension that incorporates a specific parameter to represent disciplinary depth. The outcome of this research program is expected to have impacts in a wide array of applications such as programming bug assignments to qualified developers, didactic content sequencing in intelligent learning environments, technical project staffing, just to name a few examples.