Jaylin Herskovitz selected for NSF Graduate Research Fellowship

PhD student Jaylin Herskovitz has been awarded a prestigious NSF Graduate Research Fellowship to continue her studies in computer science and engineering at the University of Michigan.

The program recognizes and supports outstanding graduate students in NSF-supported science, technology, engineering, and mathematics disciplines who are pursuing research-based masters and doctoral degrees at accredited United States institutions.

Jaylin’s research is centered around creating augmented reality (AR) tools for collaborative work. Her goal is to develop methods to improve the accessibility of emerging technologies for people with disabilities, and to improve upon the involvement of underrepresented populations in the research community overall. 

Augmented reality enhances human capabilities and experiences by overlaying virtual information onto the world around us. Using AR, people can be guided through difficult tasks, have life-like meetings with people on different continents, and learn about objects as if they were physical. In order to create AR experiences that can be maximally beneficial, putting these superpowers into the hands of diverse user populations is crucial. 

This vision has not yet been fully realized, as AR applications are designed largely for able-bodied users and are thus inaccessible to users with a variety of impairments. AR is often primarily visual, and thus not accessible to people with visual impairments; yet, this population should not be left out of the growing use of AR in areas including education, retail, and social media.

Because of the functionality of AR to situate information within a user’s environment, there is an opportunity to pair AR with other technologies that also augment human capabilities, such as machine learning or computer vision, to further create assistive AR applications. For example, assistive AR applications have helped visually impaired users navigate their environments and access inaccessible physical interfaces. By developing methods to make AR accessible, such applications can become more practical and commonplace.

Jaylin is first author on the paper Making Mobile Augmented Reality Applications Accessible, which was presented at the 2020 ACM SIGACCESS Conference on Computers and Accessibility. In this paper, the authors identify common tasks in augmented reality applications and characterize the design space of tasks that require accessible alternatives for the visually impaired. For each of the major task categories, they created prototype accessible alternatives that they evaluate in a study with 10 blind participants to explore their perceptions of accessible AR. The study demonstrates that these prototypes make AR possible to use for blind users and reveals a number of insights to move forward. The researchers believe their work sets forth not only examples for developers to create accessible AR applications, but also a roadmap for future research to make AR comprehensively accessible.

In her first year participating in research, Jaylin worked on a series of projects that aimed to develop new ways of providing programming support for software developers. The largest of these projects was EdCode, a collaborative programming support tool for providing on demand, expert support. The research team began researching this topic in an educational context, and she designed and implemented features for a web-based support system that addressed needs such as student privacy and learning outcomes. The team’s paper on this work, entitled EdCode: Towards Personalized Support at Scale for Remote Assistance in CS Education, received a Best Short Paper Award at the IEEE Symposium on Visual Languages and Human-Centered Computing (VL/HCC 2020).

Jaylin is currently a graduate student instructor for EECS 493, User Interface Development, and secretary for the U-M Computer Science and Engineering Graduate  Student Organization (CSEG). In the summer and fall of 2020, Jaylin was a volunteer mentor and team lead for the Bold Idea Website Development course for 4th – 12th grade students, and received an Outstanding Mentor Award for her contributions. She previously served as an M-STEM Academics academic facilitator. She has completed internships at Apple and Microsoft.

Jaylin is advised in her studies by Profs. Anhong Guo, Alanson Sample, and Michael Nebeling.