Grant: Discovery Grants Program - Individual
University of Toronto
$28,000.00
Enhancing Online User-Facing Systems through Dynamic Experimentation
EF 2020-21
Résumé
People are continually interacting with online software systems, for a wide range of reasons. For example, people browse websites to gain information or make decisions, and apps to communicate or improve their health. Even when designers put careful thought into how to create such software, it often fails to meet user needs - any user of a website or app can attest to frustrations in software not helping them achieve their goals. A constant challenge in human-computer interaction is that it is difficult to know how effective a piece of technology is until it is deployed with real users, and data is being collected. I aim to understand: How can we build computational systems so that they can be perpetually changed and enhanced, even after deployment to users? For example, if a non-profit organization created a website describing its work and inviting donations, how could it deploy the website to the public, and then keep enhancing the information and appeals it makes to users?
My research program will explore methods for better using randomized A/B experiments in improving the design of user-facing software, in scalable and efficient ways. I will explore how to draw on the human intelligence of users and designers to generate new versions of technology components that can be experimentally evaluated (e.g. different messages to get users to donate to a charity). I will investigate how to help people conduct randomized A/B experiments to create alternative versions of a product/interface/experience, and measure their relative efficacy. I will also investigate how to pool human intelligence in encouraging collaborative design of experiments between designers, between designers and users, and for crowdsourcing ideas and experimental conditions from users. To make such experimentation an engine for continual improvement, I will explore how best to apply algorithms that automatically discover which conditions are effective and dynamically provide better versions to future users (such as multi-armed bandits and other algorithms for trading off exploration and exploitation from reinforcement learning). This research can explore how to use experimentation to bring together work on human computation and reinforcement learning, identifying computational principles for perpetually improving real-world systems that help people.
Programme
Discovery Grants Program - Individual
Bénéficiaire
Emplacement
Ontario, Canada
Type de bénéficiaire
Academia
Type
Grant
Source
NSERC's Awards Data
Tel que publié
Application id
RGPIN-2019-06968
Name nom
Williams, Joseph
Department département
Computer Science
Organization id
31
Institution établissement
University of Toronto
Province en
Ontario
Province fr
Ontario
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2020
Competition year année de concours
2019
Award amount
28000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
Discovery grants programs
Group fr
Programmes de subventions à la découverte
Committee code
1507
Committee name en
Computer Science
Committee name fr
Sciences informatiques
Area of application code
802
Area of application group en
Information and communication services
Area of application group fr
Services d'information et de télécommunications
Area of application en
Information systems and technology
Area of application fr
Systèmes d'information et technologie de l'information
Research subject code
2710
Research subject group en
Information technology
Research subject group fr
Technologie de l'information
Research subject en
Information systems design
Research subject fr
Conception de systèmes d'information
Installment
inst:2-5
Partie
4
Nb partie
5
Application title
Enhancing Online User-Facing Systems through Dynamic Experimentation
Keyword
NSERC
Application summary
People are continually interacting with online software systems, for a wide range of reasons. For example, people browse websites to gain information or make decisions, and apps to communicate or improve their health. Even when designers put careful thought into how to create such software, it often fails to meet user needs - any user of a website or app can attest to frustrations in software not helping them achieve their goals. A constant challenge in human-computer interaction is that it is difficult to know how effective a piece of technology is until it is deployed with real users, and data is being collected. I aim to understand: How can we build computational systems so that they can be perpetually changed and enhanced, even after deployment to users? For example, if a non-profit organization created a website describing its work and inviting donations, how could it deploy the website to the public, and then keep enhancing the information and appeals it makes to users?
My research program will explore methods for better using randomized A/B experiments in improving the design of user-facing software, in scalable and efficient ways. I will explore how to draw on the human intelligence of users and designers to generate new versions of technology components that can be experimentally evaluated (e.g. different messages to get users to donate to a charity). I will investigate how to help people conduct randomized A/B experiments to create alternative versions of a product/interface/experience, and measure their relative efficacy. I will also investigate how to pool human intelligence in encouraging collaborative design of experiments between designers, between designers and users, and for crowdsourcing ideas and experimental conditions from users. To make such experimentation an engine for continual improvement, I will explore how best to apply algorithms that automatically discover which conditions are effective and dynamically provide better versions to future users (such as multi-armed bandits and other algorithms for trading off exploration and exploitation from reinforcement learning). This research can explore how to use experimentation to bring together work on human computation and reinforcement learning, identifying computational principles for perpetually improving real-world systems that help people.