Grant: Discovery Grants Program - Individual
University of Alberta
$35,000.00
Big-Data Visual Code Completion Leveraging the Naturalness of Visual Source Code
FY 2023-24
Summary
We seek to innovate code completion for visual code. Code completion tools suggest what code comes next. While textual code completion has received extensive attention, visual code completion has not. To enable visual code completion, we will leverage language and graph models trained on big-data collections of source code. This research is important as it targets end-user programmers who are more numerous than professional programmers, and under served by software engineering (SE). Code completion uses the current code and context to show programmers what they can do next, enabling exploration and experimentation. Recently, Github Co-Pilot revolutionized textual code completion with language models trained by mining the millions of software repositories on Github. Yet not all programs are text, visual programming languages often represent source code visually as graphs of nodes with input and output ports, that you connect together graphically with arrows, connectors, or edges. These languages are promoted by the No-Code movement, supported by companies like Shopify, as they enable end-user programmers to automate workflows and program software without textual code. There are many visual programming languages in use today across many domains, with many example programs available online: * Code in game levels (Unreal Engine Blueprints); * Audio/Music data-flow based languages (pure-data, Max/MSP); * Workflows for image processing (Blender, Photoshop); * Model Driven Engineering (Eclipse Modelling Framework); * Teaching programming (Scratch); and * Web app design / Workflow automation with No-code (Zapier, Shopify Mesa). Many companies in Canada such as Electronic Arts (EA), Bioware, Epic, and Shopify use and develop visual programming languages. EA's and Epic's games rely on visual code to describe effects and events within games, often making up more than 60% of the code-base, yet visual code receives far less tool support and software quality support than textual code. Classical SE models that measure software quality and support code completion are missing from visual code environments. End-user programmers who produce visual code are often unaware of their options: what the common parameters are, what comes next, and what quality code looks like. Visual code completion helps end-user programmers by suggesting visual code solutions drawn from past successful programs. Thus we propose to improve the state of visual programming languages by adapting and inventing methods for visual code completion, to suggest what nodes, edges, or parameters come next, and visual code improvement, to warn programmers about style, modularity, and error-prone code. Applying what we learned about visual programs and the naturalness of software, we will introduce SE ideas missing in visual code environments. Visual code completion will leverage the success of the past visual programs to enable end-user programmers to build successful programs in the future.
Program
Discovery Grants Program - Individual
Recipient
Location
Alberta, Canada
Recipient Type
Academia
Type
Grant
Source
NSERC's Awards Data
As published
Application id
RGPIN-2022-03464
Name nom
Hindle, Abram
Department département
Computing Science
Organization id
9
Institution établissement
University of Alberta
Province en
Alberta
Province fr
Alberta
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2023
Competition year année de concours
2022
Award amount
35000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
1507
Committee name en
Computer Science
Committee name fr
Sciences informatiques
Area of application code
801
Area of application group en
Information and communication services
Area of application group fr
Services d'information et de télécommunications
Area of application en
Computer software
Area of application fr
Logiciel
Research subject code
2705
Research subject group en
Information technology
Research subject group fr
Software and development
Research subject en
Technologie de l'information
Research subject fr
Logiciels et développement
Application title
Big-Data Visual Code Completion Leveraging the Naturalness of Visual Source Code
Keyword
Code completion; Commit Level Software Quality Checking; End-user programming; Mining Software Repositories; Naturalness of Software; Software Analytics; Software Metrics; Software Quality; Visual Programming Languages; Visual Source Code
Application summary
We seek to innovate code completion for visual code. Code completion tools suggest what code comes next. While textual code completion has received extensive attention, visual code completion has not. To enable visual code completion, we will leverage language and graph models trained on big-data collections of source code. This research is important as it targets end-user programmers who are more numerous than professional programmers, and under served by software engineering (SE). Code completion uses the current code and context to show programmers what they can do next, enabling exploration and experimentation. Recently, Github Co-Pilot revolutionized textual code completion with language models trained by mining the millions of software repositories on Github. Yet not all programs are text, visual programming languages often represent source code visually as graphs of nodes with input and output ports, that you connect together graphically with arrows, connectors, or edges. These languages are promoted by the No-Code movement, supported by companies like Shopify, as they enable end-user programmers to automate workflows and program software without textual code. There are many visual programming languages in use today across many domains, with many example programs available online: * Code in game levels (Unreal Engine Blueprints); * Audio/Music data-flow based languages (pure-data, Max/MSP); * Workflows for image processing (Blender, Photoshop); * Model Driven Engineering (Eclipse Modelling Framework); * Teaching programming (Scratch); and * Web app design / Workflow automation with No-code (Zapier, Shopify Mesa). Many companies in Canada such as Electronic Arts (EA), Bioware, Epic, and Shopify use and develop visual programming languages. EA's and Epic's games rely on visual code to describe effects and events within games, often making up more than 60% of the code-base, yet visual code receives far less tool support and software quality support than textual code. Classical SE models that measure software quality and support code completion are missing from visual code environments. End-user programmers who produce visual code are often unaware of their options: what the common parameters are, what comes next, and what quality code looks like. Visual code completion helps end-user programmers by suggesting visual code solutions drawn from past successful programs. Thus we propose to improve the state of visual programming languages by adapting and inventing methods for visual code completion, to suggest what nodes, edges, or parameters come next, and visual code improvement, to warn programmers about style, modularity, and error-prone code. Applying what we learned about visual programs and the naturalness of software, we will introduce SE ideas missing in visual code environments. Visual code completion will leverage the success of the past visual programs to enable end-user programmers to build successful programs in the future.