About

James Fogarty is a Professor of Computer Science & Engineering at the University of Washington. He is a core member of the DUB Group, a cross-campus initiative that advances research and education in Human-Computer Interaction and Design. His broad research interests encompass Human-Computer Interaction, User Interface Software and Technology, and Ubiquitous Computing. Professor Fogarty focuses on developing, deploying, and evaluating new approaches to personal data and human-AI interaction, with particular applications in health and accessibility. His research is conducted collaboratively with a group of colleagues and current advisees. His work has received direct support from the National Science Foundation, the National Library of Medicine, and the Agency for Healthcare Research and Quality, as well as generous support from industry partners including Adobe, FXPAL, Google, Intel, Microsoft, and Nokia.

Research topics

• Computer Science
• Medicine
• Psychology
• Artificial Intelligence
• Internet privacy
• Developmental psychology
• Psychiatry
• Environmental health
• Data science
• Engineering
• Clinical psychology
• Applied psychology
• World Wide Web
• Pedagogy
• Human–computer interaction
• Physical medicine and rehabilitation
• Nursing

Selected publications

• A Framework for Adapting In-Car Touchscreen Interfaces to Driver Behaviors, Perception, and Cognition

  2026-04-13 · 1 citations

  article
  PublisherDOI

• TaskAudit: Detecting Functiona11ity Errors in Mobile Apps via Agentic Task Execution

  2026-04-13 · 1 citations

  articleOpen access

  Accessibility checkers are tools in support of accessible app development, and their use is encouraged by accessibility best practices. However, most current checkers evaluate static or mechanically-generated contexts, failing to capture common accessibility errors impacting mobile app functionality. In this work, we define functiona11ity errors as accessibility barriers that only manifest through interaction (i.e., named according to a blend of "functionality" and "accessibility"). We introduce TaskAudit, which comprises three components: a Task Generator that constructs interactive tasks from app screens, a Task Executor that uses agents with a screen reader proxy to perform these tasks, and an Accessibility Analyzer that detects and reports accessibility errors by examining interaction traces. Our evaluation on real-world apps shows that TaskAudit detects 48 functiona11ity errors from 54 app screens, compared to between 4 and 20 with existing checkers. Our analysis demonstrates common error patterns that TaskAudit can detect in addition to those from prior work, including label-functionality mismatch, cluttered navigation, and inappropriate feedback.

  PublisherDOI

• Modeling Accessibility: Characterizing What We Mean by “Accessible”

  2025-10-22 · 4 citations

  articleOpen access

  Accessibility research has a broad mandate: use technology to make the world more accessible to disabled people. Yet, as a field, accessibility research lacks a clear characterization of what "accessibility" is. Furthermore, it has been historically limited in who is designed for, focusing on specific types of disability and often failing to consider how disability intersects with other identities. We set out to explicate what it means to make something accessible, grounded in the lived experiences of a diverse group of 25 disabled people. From our empirical findings, we develop a process for modeling accessibility. First, an individual assesses their experience of inaccess, specifically, the type of barrier they face, the technology repertoire they possess, and the contextual factors that shape how they address accessibility barriers. Then, having assessed an access barrier, they perform consequence calculus, weighing all available options to achieve access and deciding upon the option that best matches their priorities. We highlight the situated nature of access; people's identities, contextual factors, repertoires, and priorities all dictate their experience of accessibility.

  PublisherDOI

• “I Want to Think Like an SLP”: A Design Exploration of AI-Supported Home Practice in Speech Therapy

  2025-04-24 · 7 citations

  article
  PublisherDOI

• Exploring AI-Based Support in Speech-Language Pathology for Culturally and Linguistically Diverse Children

  2025-04-24 · 9 citations

  articleOpen access
  PublisherDOI

• Inaccessible and Deceptive: Examining Experiences of Deceptive Design with People Who Use Visual Accessibility Technology

  2025-04-25 · 3 citations

  articleOpen accessSenior author

  Deceptive design patterns manipulate people into actions to which they would otherwise object. Despite growing research on deceptive design patterns, limited research examines their interplay with accessibility and visual accessibility technology (e.g., screen readers, screen magnification, braille displays). We present an interview and diary study with 16 people who use visual accessibility technology to better understand experiences with accessibility and deceptive design. We report participant experiences with six deceptive design patterns, including designs that are intentionally deceptive and designs where participants describe accessibility barriers unintentionally manifesting as deceptive, together with direct and indirect consequences of deceptive patterns. We discuss intent versus impact in accessibility and deceptive design, how access barriers exacerbate harms of deceptive design patterns, and impacts of deceptive design from a perspective of consequence-based accessibility. We propose that accessibility tools could help address deceptive design patterns by offering higher-level feedback to well-intentioned designers.

  PublisherOA PDFDOI

• Examining Researcher Experiences and Tensions Around Participant Engagement in Health HCI Research

  2025-04-25 · 3 citations

  articleOpen access
  PublisherOA PDFDOI

• ScreenAudit: Detecting Screen Reader Accessibility Errors in Mobile Apps Using Large Language Models

  2025-04-24 · 12 citations

  preprintOpen access

  Many mobile apps are inaccessible, thereby excluding people from their potential benefits. Existing rule-based accessibility checkers aim to mitigate these failures by identifying errors early during development but are constrained in the types of errors they can detect. We present ScreenAudit, an LLM-powered system designed to traverse mobile app screens, extract metadata and transcripts, and identify screen reader accessibility errors overlooked by existing checkers. We recruited six accessibility experts including one screen reader user to evaluate ScreenAudit's reports across 14 unique app screens. Our findings indicate that ScreenAudit achieves an average coverage of 69.2%, compared to only 31.3% with a widely-used accessibility checker. Expert feedback indicated that ScreenAudit delivered higher-quality feedback and addressed more aspects of screen reader accessibility compared to existing checkers, and that ScreenAudit would benefit app developers in real-world settings.

  PublisherDOI

• Deploying and Examining Beacon for At-Home Patient Self-Monitoring with Critical Flicker Frequency

  2025-04-24

  articleOpen accessSenior author

  Chronic liver disease can lead to neurological conditions that result in coma or death. Although early detection can allow for intervention, testing is infrequent and unstandardized. Beacon is a device for at-home patient self-measurement of cognitive function via critical flicker frequency, which is the frequency at which a flickering light appears steady to an observer. This paper presents our efforts in iterating on Beacon's hardware and software to enable at-home use, then reports on an at-home deployment with 21 patients taking measurements over 6 weeks. We found that measurements were stable despite being taken at different times and in different environments. Finally, through interviews with 15 patients and 5 hepatologists, we report on participant experiences with Beacon, preferences around how CFF data should be presented, and the role of caregivers in helping patients manage their condition. Informed by our experiences with Beacon, we further discuss design implications for home health devices.

  PublisherDOI

• Engagements with Generative AI and Personal Health Informatics: Opportunities for Planning, Tracking, Reflecting, and Acting around Personal Health Data

  Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · 2025-09-03 · 5 citations

  articleOpen access

  Personal informatics processes require navigating distinct challenges across stages of tracking, but the range of data, goals, expertise, and context that individuals bring to self-tracking often presents barriers that undermine those processes. We investigate the potential of Generative AI (GAI) to support people across stages of pursuing self-tracking for health. We conducted interview and observation sessions with 19 participants from the United States who self-track for health, examining how they interact with GAI around their personal health data. Participants formulated and refined queries, reflected on recommendations, and abandoned queries that did not meet their needs and health goals. They further identified opportunities for GAI support across stages of self-tracking, including in deciding what data to track and how, in defining and modifying tracking plans, and in interpreting data-driven insights. We discuss GAI opportunities in accounting for a range of health goals, in providing support for self-tracking processes across planning, reflection, and action, and in consideration of limitations of embedding GAI in health self-tracking tools.

  PublisherOA PDFDOI

Recent grants

• CAREER: Pixel-Based Interpretation and Modification of Graphical User Interfaces

  NSF · $589k · 2011–2017

• HCC-Small: Investigating and Supporting the Iterative and Exploratory Process of Applying Statistical Machine Learning

  NSF · $506k · 2008–2012

• Open Tools for Self-Tracking, Self Experimentation, and Patient-Provider Collaboration in Symptom Self-Management and Clinical Care

  NIH · $1.3M · 2018–2024

• CHS: Medium: Improving the Accessibility of Mobile Applications by Enabling Third-Party Assessment, Repair, and Enhancement

  NSF · $1.6M · 2017–2023

• CHS: Small: Support for Self-Tracking and Patient-Provider Collaboration Using Data for Multiple and Evolving Goals in People with Migraine

  NSF · $524k · 2018–2023

Frequent coauthors

• Sean A. Munson

  35 shared

• Scott E. Hudson

  Carnegie Mellon University

  18 shared

• Jacob O. Wobbrock

  Defense Information School

  16 shared

• Jessica Schroeder

  University of Washington

  15 shared

• Julie A. Kientz

  University of Washington

  14 shared

• Ravi Karkar

  14 shared

• Daniel A. Epstein

  University of California, Irvine

  13 shared

• Jasmine Zia

  Swedish Medical Center

  12 shared

Labs

• James Fogarty's LabPI

  Developing, deploying, and evaluating new approaches to personal data and human-AI interaction, often with applications in health and accessibility.