Canada Spends Logo

Explore our Budget 2026 analysis

Grant: Discovery Grants Program - Individual

University of Alberta

$35,000.00

Big-Data Visual Code Completion Leveraging the Naturalness of Visual Source Code

FY 2024-25

Summary
We seek to innovate code completion for visual code. Code completion tools suggest what code comes next. While textual code completion has received extensive attention, visual code completion has not. To enable visual code completion, we will leverage language and graph models trained on big-data collections of source code. This research is important as it targets end-user programmers who are more numerous than professional programmers, and under served by software engineering (SE). Code completion uses the current code and context to show programmers what they can do next, enabling exploration and experimentation. Recently, Github Co-Pilot revolutionized textual code completion with language models trained by mining the millions of software repositories on Github. Yet not all programs are text, visual programming languages often represent source code visually as graphs of nodes with input and output ports, that you connect together graphically with arrows, connectors, or edges. These languages are promoted by the No-Code movement, supported by companies like Shopify, as they enable end-user programmers to automate workflows and program software without textual code. There are many visual programming languages in use today across many domains, with many example programs available online: * Code in game levels (Unreal Engine Blueprints); * Audio/Music data-flow based languages (pure-data, Max/MSP); * Workflows for image processing (Blender, Photoshop); * Model Driven Engineering (Eclipse Modelling Framework); * Teaching programming (Scratch); and * Web app design / Workflow automation with No-code (Zapier, Shopify Mesa). Many companies in Canada such as Electronic Arts (EA), Bioware, Epic, and Shopify use and develop visual programming languages. EA's and Epic's games rely on visual code to describe effects and events within games, often making up more than 60% of the code-base, yet visual code receives far less tool support and software quality support than textual code. Classical SE models that measure software quality and support code completion are missing from visual code environments. End-user programmers who produce visual code are often unaware of their options: what the common parameters are, what comes next, and what quality code looks like. Visual code completion helps end-user programmers by suggesting visual code solutions drawn from past successful programs. Thus we propose to improve the state of visual programming languages by adapting and inventing methods for visual code completion, to suggest what nodes, edges, or parameters come next, and visual code improvement, to warn programmers about style, modularity, and error-prone code. Applying what we learned about visual programs and the naturalness of software, we will introduce SE ideas missing in visual code environments. Visual code completion will leverage the success of the past visual programs to enable end-user programmers to build successful programs in the future.
Program
Discovery Grants Program - Individual
Location
Alberta, Canada
Recipient Type
Academia
Type
Grant
Source
NSERC's Awards Data

As published

Application id
RGPIN-2022-03464
Name nom
Hindle, Abram
Department département
Computing Science
Organization id
9
Institution établissement
University of Alberta
Province en
Alberta
Province fr
Alberta
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2024
Competition year année de concours
2022
Award amount
35000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
1507
Committee name en
Computer Science
Committee name fr
Sciences informatiques
Area of application code
801
Area of application group en
Information and communication services
Area of application group fr
Services d'information et de télécommunications
Area of application en
Computer software
Area of application fr
Logiciel
Research subject code
2706
Research subject group en
Information technology
Research subject group fr
Technologie de l'information
Research subject en
Software engineering
Research subject fr
Génie logiciel
Field of research list names en
Not available
Field of research list names fr
Non disponible
Socioeconomic objective list names en
Not available
Socioeconomic objective list names fr
Non disponible
Application title
Big-Data Visual Code Completion Leveraging the Naturalness of Visual Source Code
Keywords
visual source code; visual programming languages; code completion; mining software repositories; naturalness of software; end-user programming; commit level software quality checking; software analytics; software quality; software metrics
Application summary
We seek to innovate code completion for visual code. Code completion tools suggest what code comes next. While textual code completion has received extensive attention, visual code completion has not. To enable visual code completion, we will leverage language and graph models trained on big-data collections of source code. This research is important as it targets end-user programmers who are more numerous than professional programmers, and under served by software engineering (SE). Code completion uses the current code and context to show programmers what they can do next, enabling exploration and experimentation. Recently, Github Co-Pilot revolutionized textual code completion with language models trained by mining the millions of software repositories on Github. Yet not all programs are text, visual programming languages often represent source code visually as graphs of nodes with input and output ports, that you connect together graphically with arrows, connectors, or edges. These languages are promoted by the No-Code movement, supported by companies like Shopify, as they enable end-user programmers to automate workflows and program software without textual code. There are many visual programming languages in use today across many domains, with many example programs available online: * Code in game levels (Unreal Engine Blueprints); * Audio/Music data-flow based languages (pure-data, Max/MSP); * Workflows for image processing (Blender, Photoshop); * Model Driven Engineering (Eclipse Modelling Framework); * Teaching programming (Scratch); and * Web app design / Workflow automation with No-code (Zapier, Shopify Mesa). Many companies in Canada such as Electronic Arts (EA), Bioware, Epic, and Shopify use and develop visual programming languages. EA's and Epic's games rely on visual code to describe effects and events within games, often making up more than 60% of the code-base, yet visual code receives far less tool support and software quality support than textual code. Classical SE models that measure software quality and support code completion are missing from visual code environments. End-user programmers who produce visual code are often unaware of their options: what the common parameters are, what comes next, and what quality code looks like. Visual code completion helps end-user programmers by suggesting visual code solutions drawn from past successful programs. Thus we propose to improve the state of visual programming languages by adapting and inventing methods for visual code completion, to suggest what nodes, edges, or parameters come next, and visual code improvement, to warn programmers about style, modularity, and error-prone code. Applying what we learned about visual programs and the naturalness of software, we will introduce SE ideas missing in visual code environments. Visual code completion will leverage the success of the past visual programs to enable end-user programmers to build successful programs in the future.