April 23, 2023
Faculty and students from the University of Washington’s Department of Human Centered Design & Engineering had a strong presence at the 2023 Conference on Human Factors in Computing Systems (CHI), the premier international conference on Human-Computer Interaction.
HCDE researchers are contributing 17 papers to the 2023 CHI conference, including two selected for the Best Paper Award recognition (given to the top 1% of submissions) and one selected for Best Paper Honorable Mention recognition (given to the top 5% of submissions).
Researchers from the UW community overall contributed to 62 papers. These publications draw from 7 different UW departments and programs, demonstrating the exceptional power of interdisciplinary research that is at the heart of the UW DUB community. Read the full list of UW contributions on the DUB website.
CHI 2023 is held from April 23-28 in Hamburg, Germany. Find information about contributions of HCDE researchers below. Names of HCDE students and faculty are in bold.
CHI BEST PAPER AWARD
Deceptive Design Patterns in Safety Technologies: A Case Study of the Citizen App
Ishita Chordia, UW Information School; Lena-Phuong Tran, UW Human Centered Design & Engineering; Tala June Tayebi, UW Information School; Emily Parrish, UW Information School; Sheena Erete, University of Maryland, College Park; Jason Yip, UW Information School; Alexis Hiniker, UW Information School
Deceptive design patterns (known as dark patterns) are interface characteristics which modify users’ choice architecture to gain users’ attention, data, and money. Deceptive design patterns have yet to be documented in safety technologies despite evidence that designers of safety technologies make decisions that can powerfully influence user behavior. To address this gap, we conduct a case study of the Citizen app, a commercially available technology which notifies users about local safety incidents. We bound our study to Atlanta and triangulate interview data with an analysis of the user interface. Our results indicate that Citizen heightens users’ anxiety about safety while encouraging the use of profit-generating features which offer security. These findings contribute to an emerging conversation about how deceptive design patterns interact with sociocultural factors to produce deceptive infrastructure. We propose the need to expand an existing taxonomy of harm to include emotional load and social injustice and offer recommendations for designers interested in dismantling the deceptive infrastructure of safety technologies.
CHI BEST PAPER AWARD
Probing a Community-Based Conversational Storytelling Agent to Document Digital Stories of Housing Insecurity
Brett A Halperin, UW Human Centered Design & Engineering; Gary Hsieh, UW Human Centered Design & Engineering; Erin McElroy, The University of Texas at Austin; James Pierce, UW Art + Art History + Design; Daniela Rosner, UW Human Centered Design & Engineering
Despite the central role that stories play in social movement-building, they are difficult to sustainably document for many reasons. To explore this challenge, this paper describes the design of a community-based conversational storytelling agent (CSA) to document digital stories of housing insecurity. Building on insights from an ongoing grassroots project, the Anti-Eviction Mapping Project, we share how a study initially focused on CSA-support opened an investigation of the role that artificial intelligence may play in housing justice movements. Drawing from 17 interviews with narrators of housing insecurity experiences and collectors of such stories, we find that collectors perceive opportunities to expand means of documentation with multimedia and multi-language support. Meanwhile, some narrators perceive potential for a CSA to offer therapeutic storytelling experiences and document otherwise unrecorded stories. Yet, CSA encounters also surface perils of machine bias, as well as reduced possibilities of human connections and relations.
CHI BEST PAPER HONORABLE MENTION
SwitchTube: A Proof-of-Concept System Introducing "Adaptable Commitment Interfaces" as a Tool for Digital Wellbeing
Kai Lukoff, Santa Clara University; Ulrik Lyngs, University of Oxford; Karina Shirokova, UW Human Centered Design & Engineering; Raveena Rao, UW Information School; Larry Tian, UW Information School; Himanshu Zade, UW Human Centered Design & Engineering; Sean A Munson, UW Human Centered Design & Engineering; Alexis Hiniker, UW Information School
YouTube has many features, such as homepage recommendations, that encourage users to explore its vast library of videos. However, when users visit YouTube with a specific intention, e.g., learning how to program in Python, these features to encourage exploration are often distracting. Prior work has innovated ‘commitment interfaces’ that restrict social media but finds that they often indiscriminately block needed content. In this paper, we describe the design, development, and evaluation of an ‘adaptable commitment interface,’ the SwitchTube mobile app, in which users can toggle between two interfaces when watching YouTube videos: Focus Mode (search-first) and Explore Mode (recommendations-first). In a three-week field deployment with 46 US participants, we evaluate how the ability to switch between interfaces affects user experience, finding that it provides users with a greater sense of agency, satisfaction, and goal alignment. We conclude with design implications for how adaptable commitment interfaces can support digital wellbeing.
Advancing Human-AI Complementarity: The Impact of User Expertise and Algorithmic Tuning on Joint Decision Making
Kori Inkpen, Microsoft; Shreya Chappidi, University of Virginia; Keri Mallari, UW Human Centered Design & Engineering; Besmira Nushi, Microsoft; Divya Ramesh, University of Michigan; Pietro Michelucci, Human Computation Institute; Vani Mandava, UW eScience Institute; Libuše Hannah Vepřek, Ludwig-Maximilians-Universität München; Gabrielle Quinn, Western Washington University
Human-AI collaboration for decision-making strives to achieve team performance that exceeds the performance of humans or AI alone. However, many factors can impact success of Human-AI teams, including a user’s domain expertise, mental models of an AI system, trust in recommendations, and more. This paper reports on a study that examines users’ interactions with three simulated algorithmic models, all with equivalent accuracy rates but each tuned differently in terms of true positive and true negative rates. Our study examined user performance in a non-trivial blood vessel labeling task where participants indicated whether a given blood vessel was flowing or stalled. Users completed 150 trials across multiple stages, first without an AI and then with recommendations from an AI-Assistant. Although all users had prior experience with the task, their levels of proficiency varied widely. Our results demonstrated that while recommendations from an AI-Assistant can aid in users’ decision making, several underlying factors, including user base expertise and complementary human-AI tuning, significantly impact the overall team performance. First, users’ base performance matters, particularly in comparison to the performance level of the AI. Novice users improved, but not to the accuracy level of the AI. Highly proficient users were generally able to discern when they should follow the AI recommendation and typically maintained or improved their performance. Mid-performers, who had a similar level of accuracy to the AI, were most variable in terms of whether the AI recommendations helped or hurt their performance. Second, tuning an AI algorithm to complement users’ strengths and weaknesses also significantly impacted users’ performance. For example, users in our study were better at detecting flowing blood vessels, so when the AI was tuned to reduce false negatives (at the expense of increasing false positives), users were able to reject those recommendations more easily and improve in accuracy. Finally, users’ perception of the AI’s performance relative to their own performance had an impact on whether users’ accuracy improved when given recommendations from the AI. Overall, this work reveals important insights on the complex interplay of factors influencing Human-AI collaboration and provides recommendations on how to design and tune AI algorithms to complement users in decision-making tasks.
Autospeculation: Reflecting on the Intimate and Imaginative Capacities of Data Analysis
Brian Kinnee, UW Human Centered Design & Engineering; Audrey Desjardins, UW Art + Art History + Design; Daniela Rosner, UW Human Centered Design & Engineering
Given decades of Human computer interaction (HCI) research focused on scientific empiricism, it can be hard for the field to acknowledge that data analysis is both an emotional and speculative process. But what does it mean for this process of data analysis to embrace its situated and speculative nature? In this paper, we explore this possibility by building on decades of HCI mixed methods that root data analysis in design. Drawing on an autoethnographic design inquiry, we examine how data analysis can work as an implicating process, one that is not only critically grounded in a designer’s own situation but also offers modes of imagining the world otherwise. In this analysis, we find that autobiographical design can help HCI scholars to respond to current critiques of speculative design by grounding and rendering more personal certain kinds of speculation, opening a space for diverse voices to emerge.
Disability-First Design and Creation of a Dataset Showing Private Visual Information Collected With People Who Are Blind
Tanusree Sharma, University of Illinois Urbana-Champaign; Abigale Stangl, UW Human Centered Design & Engineering; Lotus Zhang, UW Human Centered Design & Engineering; Yu-Yun Tseng, University of Colorado; Inan Xu, University of California, Santa Cruz; Leah Findlater, UW Human Centered Design & Engineering; Danna Gurari, University of Colorado; Yang Wang, University of Illinois Urbana-Champaign
We present the design and creation of a disability-first dataset, “BIV-Priv,” which contains 728 images and 728 videos of 14 private categories captured by 26 blind participants to support downstream development of artificial intelligence (AI) models. While best practices in dataset creation typically attempt to eliminate private content, some applications require such content for model development. We describe our approach in creating this dataset with private content in an ethical way, including using props rather than participants’ own private objects and balancing multi-disciplinary perspectives (e.g., accessibility, privacy, computer vision) to meet the tangible metrics (e.g., diversity, category, amount of content) to support AI innovations. We observed challenges that our participants encountered during the data collection, including accessibility issues (e.g., understanding foreground vs. background object placement) and issues due to the sensitive nature of the content (e.g., discomfort in capturing some props such as condoms around family members).
Doufu, Rice Wine, and 面饼: Supporting the Connections Between Precision and Cultural Knowledge in Cooking
Danli Luo, UW Human Centered Design & Engineering; Daniela Rosner, UW Human Centered Design & Engineering; Nadya Peek, UW Human Centered Design & Engineering
The digital codification and measurement of food preparation has made strong contributions to HCI food research, whether through ingredient manipulation, workflow management, or recipe interaction. But prior work has shown that technical developments that emphasize precise gourmet practices tend to overlook the importance of cultural knowledge. Drawing on an integrative autobiographical design approach, we describe an open-source hardware toolkit that we developed to examine the process of integrating precision techniques with ritual cooking practices across three recipes: flour skin, rice wine, and doufu. Our work points to the importance of understanding precision as a cultural process with roots in personal and familial experience. We end with a reflection on the particular knowledge-forms that come from cultivating cultural relationships to fabrication processes and their implications for reading digital fabrication processes as meaningfully relational.
"Easier or Harder, Depending on Who the Hearing Person Is": Codesigning Videoconferencing Tools for Small Groups With Mixed Hearing Status
Emma J McDonnell, UW Human Centered Design & Engineering; Soo Hyun Moon, UW Human Centered Design & Engineering; Lucy Jiang, UW Computer Science & Engineering; Steven M Goodman, UW Human Centered Design & Engineering; Raja Kushalnagar, Gallaudet University; Jon E Froehlich, UW Computer Science & Engineering; Leah Findlater, UW Human Centered Design & Engineering
With improvements in automated speech recognition and increased use of videoconferencing, real-time captioning has changed significantly. This shift toward broadly available but less accurate captioning invites exploration of the role hearing conversation partners play in shaping the accessibility of a conversation to d/Deaf and hard of hearing (DHH) captioning users. While recent work has explored DHH individuals’ videoconferencing experiences with captioning, we focus on established groups’ current practices and priorities for future tools to support more accessible online conversations. Our study consists of three codesign sessions, conducted with four groups (17 participants total, 10 DHH, 7 hearing). We found that established groups crafted social accessibility norms that met their relational contexts. We also identify promising directions for future captioning design, including the need to standardize speaker identification and customization, opportunities to provide behavioral feedback during a conversation, and ways that videoconferencing platforms could enable groups to set and share norms.
Envisioning Narrative Intelligence: A Creative Visual Storytelling Anthology
Brett A Halperin, UW Human Centered Design & Engineering; Stephanie M Lukin, US Army Research Laboratory
In this paper, we collect an anthology of 100 visual stories from authors who participated in our systematic creative process of improvised story-building based on image sequences. Following close reading and thematic analysis of our anthology, we present five themes that characterize the variations found in this creative visual storytelling process: (1) Narrating What is in Vision vs. Envisioning; (2) Dynamically Characterizing Entities/Objects; (3) Sensing Experiential Information About the Scenery; (4) Modulating the Mood; (5) Encoding Narrative Biases. In understanding the varied ways that people derive stories from images, we offer considerations for collecting story-driven training data to inform automatic story generation. In correspondence with each theme, we envision narrative intelligence criteria for computational visual storytelling as: creative, reliable, expressive, grounded, and responsible. From these criteria, we discuss how to foreground creative expression, account for biases, and operate in the bounds of visual storyworlds.
From User Perceptions to Technical Improvement: Enabling People Who Stutter to Better Use Speech Recognition
Colin Lea, Apple; Zifang Huang, Apple; Jaya Narain, Apple; Lauren Tooley, Apple; Dianna Yee, Apple; Dung Tien Tran, Apple; Panayiotis Georgiou, Apple; Jeffrey P Bigham, Carnegie Mellon University / Apple; Leah Findlater, UW Human Centered Design & Engineering / Apple
Consumer speech recognition systems do not work as well for many people with speech differences, such as stuttering, relative to the rest of the general population. However, what is not clear is the degree to which these systems do not work, how they can be improved, or how much people want to use them. In this paper, we first address these questions using results from a 61-person survey from people who stutter and find participants want to use speech recognition but are frequently cut off, misunderstood, or speech predictions do not represent intent. In a second study, where 91 people who stutter recorded voice assistant commands and dictation, we quantify how dysfluencies impede performance in a consumer-grade speech recognition system. Through three technical investigations, we demonstrate how many common errors can be prevented, resulting in a system that cuts utterances off 79.1% less often and improves word error rate from 25.4% to 9.9%.
How Older Adults Use Online Videos for Learning
Seoyoung Kim, KAIST; Donghoon Shin, UW Human Centered Design & Engineering; Jeongyeon Kim, University of California, San Diego; Soonwoo Kwon, Riiid; Juho Kim, KAIST
Online videos are a promising medium for older adults to learn. Yet, few studies have investigated what, how, and why they learn through online videos. In this study, we investigated older adults’ motivation, watching patterns, and difficulties in using online videos for learning by (1) running interviews with 13 older adults and (2) analyzing large-scale video event logs (N=41.8M) from a Korean Massive Online Open Course (MOOC) platform. Our results show that older adults (1) are motivated to learn practical topics, leading to less consumption of STEM domains than non-older adults, (2) watch videos with less interaction and watch a larger portion of a single video compared to non-older adults, and (3) face various difficulties (e.g., inconvenience arisen due to their unfamiliarity with technologies) that limit their learning through online videos. Based on the findings, we propose design guidelines for online videos and platforms targeted to support older adults’ learning.
Imprimer: Computational Notebooks for CNC Milling
Jasper Tran O'Leary, UW Computer Science & Engineering; Gabrielle Benabdallah, UW Human Centered Design & Engineering; Nadya Peek, UW Human Centered Design & Engineering
Digital fabrication in industrial contexts involves standardized procedures that prioritize precision and repeatability. However, fabrication machines are now available for practitioners who focus instead on experimentation. In this paper, we reframe hobbyist CNC milling as writing literate programs which interleave documentation, interactive graphics, and source code for machine control. To test this approach, we present Imprimer, a machine infrastructure for a CNC mill and an associated library for a computational notebook. Imprimer lets makers learn experimentally, prototype new interactions for making, and understand physical processes by writing and debugging code. We demonstrate three experimental milling workflows as computational notebooks, conduct a user study with practitioners with a range of backgrounds, and discuss literate programming as a future vision for digital fabrication altogether.
IntroBot: Exploring the Use of Chatbot-Assisted Familiarization in Online Collaborative Groups
Donghoon Shin, UW Human Centered Design & Engineering; Soomin Kim, Seoul National University; Ruoxi Shang, UW Human Centered Design & Engineering; Joonhwan Lee, Seoul National University; Gary Hsieh, UW Human Centered Design & Engineering
Many people gather online and form teams with strangers to collaborate on tasks. However, while intrateam trust and cohesion are critical for team performance, such characteristics take time to establish and are harder to build up through computer-mediated communication. Building on prior research that has shown that enhancing familiarity between members can help, we hypothesized that the use of a chatbot to support the familiarization of ad hoc teammates can help their collaboration. As such, we designed IntroBot, a chatbot that builds on an online discussion facilitator framework and leverages the social media data of users to assist their familiarization process. Through a between-subjects study (N=60), we found that participants who used IntroBot reported higher levels of trust, cohesion, and interaction quality, as well as generated more ideas in a collaborative brainstorming task. We discuss insights gained from our study, and present opportunities for the future of chatbot-assisted collaboration.
On the Grounds of Solutionism: Ontologies of Blackness and HCI
Jay L Cunningham, UW Human Centered Design & Engineering; Gabrielle Benabdallah, UW Human Centered Design & Engineering; Daniela Rosner, UW Human Centered Design & Engineering; Alex S Taylor, City, University of London
Why is the solution the end point to a problem? While many in HCI and design have examined the impulse to solve problems–the solutionist or techno-solutionist mindset–we examine the logic that binds the solution and the problem together as a pair. Focusing on the timely and consequential problem of systemic racial injustice, we think through the paradoxical possibility that the pairing of the problem and solution (so often treated as the default in design and HCI) perpetuates the very conditions we seek to improve. With Calvin Warren’s profound Afro-pessimism, we recognize how the tools used to solve structural inequities around Black life are constructed with inequities themselves. The problem-solution, therefore, is a dead end. We use this paradox as an invitation to rethink ongoing efforts to seek equity and justice more broadly, setting out a fragile but hopeful path for HCI and design.
Understanding Collaborative Practices and Tools of Professional UX Practitioners in Software Organizations
K J Kevin Feng, UW Human Centered Design & Engineering; Tony W Li, UW Human Centered Design & Engineering; Amy X Zhang, UW Computer Science & Engineering
User experience (UX) has undergone a revolution in collaborative practices, due to tools that enable quick feedback and continuous collaboration with a varied team across a design’s lifecycle. However, it is unclear how this shift in collaboration has been received in professional UX practice, and whether new pain points have arisen. To this end, we conducted a survey (N = 114) with UX practitioners at software organizations based in the U.S. to better understand their collaborative practices and tools used throughout the design process. We found that while an increase in collaborative activity enhanced many aspects of UX work, some long-standing challenges—such as handing off designs to developers—still persist. Moreover, we observed new challenges emerging from activities enabled by collaborative tools such as design system management. Based on our findings, we discuss how UX practices can improve collaboration moving forward and provide concrete design implications for collaborative UX tools.
Understanding Visual Arts Experiences of Blind People
Franklin Mingzhe Li, Carnegie Mellon University; Lotus Zhang, UW Human Centered Design & Engineering; Maryam Bandukda, University College London; Abigale Stangl, UW Human Centered Design & Engineering; Kristen Shinohara, Rochester Institute of Technology; Leah Findlater, UW Human Centered Design & Engineering; Patrick Carrington, Carnegie Mellon University
Visual arts play an important role in cultural life and provide access to social heritage and self-enrichment, but most visual arts are inaccessible to blind people. Researchers have explored different ways to enhance blind people’s access to visual arts (e.g., audio descriptions, tactile graphics). However, how blind people adopt these methods remains unknown. We conducted semi-structured interviews with 15 blind visual arts patrons to understand how they engage with visual artwork and the factors that influence their adoption of visual arts access methods. We further examined interview insights in a follow-up survey (N=220). We present: 1) current practices and challenges of accessing visual artwork in-person and online (e.g., Zoom tour), 2) motivation and cognition of perceiving visual arts (e.g., imagination), and 3) implications for designing visual arts access methods. Overall, our findings provide a roadmap for technology-based support for blind people’s visual arts experiences.
What Is in the Cards: Exploring Uses, Patterns, and Trends in Design Cards
Gary Hsieh, UW Human Centered Design & Engineering; Brett A Halperin, UW Human Centered Design & Engineering; Evan Schmitz, UW Human Centered Design & Engineering; Yen Nee Chew, National Tsing Hua University; Yuan-Chi Tseng, National Tsing Hua University
Card-based design tools–design cards–increasingly present opportunities to support practitioners. However, the breadth and depth of the design card landscape remain underexplored. In this work, we surveyed 103 design practitioners to assess current usages and associated barriers. Additionally, we analyzed and classified 161 decks of design cards from 1952-2020. We held a workshop with four experienced practitioners to generate initial categories, and then coded the remaining decks. We found that the cards contain seven different types of design knowledge: Creative Inspiration; Human Insights; Material & Domain; Methods & Tooling; Problem Definition; Team Building; and Values in Practice. The content of these cards can support designers across design stages; however, most are intended to support the early stages of design (e.g., research and ideation) rather than later design stages (e.g., prototyping and implementation). We share additional patterns uncovered and provide recommendations to support the future development and adoption of these tools.