About

Jaime Ruiz is a professor whose research focuses on human-computer interaction, particularly in the areas of virtual reality, prosthetic control, natural language interaction, and augmented reality. His work explores the design and evaluation of interactive systems that enhance human capabilities and improve user experiences.

Research topics

• Computer Science
• Psychology
• Artificial Intelligence
• Human–computer interaction
• Engineering
• Business
• Communication
• Knowledge management
• Multimedia
• Process management

Selected publications

• ‘Over Time Everyone’s Gonna Be Open to It’: User Attitudes Toward Security and Privacy in Continuous Authentication for Smart Homes

  IEEE Access · 2025-01-01 · 1 citations

  articleOpen access

  Continuous authentication (CA), a user authentication approach that continuously verifies a person’s identity without requiring explicit input, is increasingly being deployed in smart homes to maintain security posture throughout user sessions. However, prior research has overlooked user attitudes toward the increased data collection and surveillance associated with CA in smart homes. To bridge this gap, we conducted a focus group study with 33 participants, using a design probe video to simulate various CA implementation scenarios in smart homes. We explored participants’ current authentication methods (e.g., passwords and physiological biometrics) and examined their perceptions of CA. Through affinity diagramming, we found that participants perceive smart-home CA as presenting privacy and security challenges yet possessing great potential for enhanced usability. Participants also envisioned CA systems that offer more granular permission controls over personal data. Our findings indicate the contextual dependencies in balancing usability with privacy and security concerns. Our contributions include a comprehensive empirical dataset featuring the design probe video, participant transcripts, and a conceptual model of users’ nuanced understanding. We provide design recommendations for smart-home CA systems, emphasizing transparency as a crucial factor in building user trust and improving adoption rates.

  PublisherDOI

• When is Self-Gaze Helpful? Examining Uni- vs Bi-Directional Gaze Visualization in Collocated AR Tasks

  2025-10-08

  articleSenior author

  Shared-gaze visualizations (SGVs) in augmented reality enable collaborators to share focus and intentions through gaze interactions. Most prior research has examined bi-directional visualizations, where both users see their own and their partner's gaze, to provide feedback on how their gaze is communicated to their partner. However, bi-directional SGV approaches are largely based on research for remote collaboration. In collocated settings, bi-directional SGVs can obstruct views and cause distractions. Additionally, collocated applications differ from remote ones. We propose that if eye-tracking is well-calibrated, bi-directional visualizations may be unnecessary in collocated settings. To explore this, we conducted a user study comparing perceptions of uni- and bi-directional gaze visualizations in a virtual collaborative sorting task. Our results suggest that self-gaze may not always be necessary for users; however, there are cases in which self-gaze helps them feel more confident in the task. We offer a deeper understanding for future collaborative gaze interaction systems.

  PublisherDOI

• May The Force be With You: Cloning Distant Objects to Improve Medium-Field Interactions in Augmented Reality

  2025-03-08 · 1 citations

  articleSenior author

  Augmented Reality (AR) interactions feature users interacting with virtual objects registered in the physical world. With contemporary AR experiences increasingly featuring interactions at distances, we conceptualized The Force, a technique that allows users to clone distant objects and manipulate their replicas. An empirical evaluation was conducted, comparing it against two well-established techniques including controller-based ray-casting and a gaze-based pinching technique in a pick-and-place task. We employed a within-subjects design, collecting data on both objective performance and subjective user experience. Results suggest that The Force allows for higher levels of accuracy and efficiency in medium-field tasks that require precision and fine motor control. Furthermore, we discovered avenues towards iteratively refining this technique. We go on to discuss the implications of our findings in an effort to facilitate better interactions in augmented reality.

  PublisherDOI

• MuCHEx: A Multimodal Conversational Debugging Tool for Interactive Visual Exploration of Hierarchical Object Classification

  IEEE Computer Graphics and Applications · 2025-08-12

  article

  Object recognition is a fundamental challenge in computer vision, particularly for fine-grained object classification, where classes differ in minor features. Improved fine-grained object classification requires a teaching system with numerous classes and instances of data. As the number of hierarchical levels and instances grows, debugging these models becomes increasingly complex. Moreover, different types of debugging tasks require varying approaches, explanations, and levels of detail. We present MuCHEx, a multimodal conversational system that blends natural language and visual interaction for interactive debugging of hierarchical object classification. Natural language allows users to flexibly express high-level questions or debugging goals without needing to navigate complex interfaces, while adaptive explanations surface only the most relevant visual or textual details based on the user's current task. This multimodal approach combines the expressiveness of language with the precision of direct manipulation, enabling context-aware exploration during model debugging.

  PublisherDOI

• Adaptive vs Monthly Support for Weight-Loss Maintenance

  JAMA Network Open · 2025-09-22 · 1 citations

  articleOpen access

  Importance: Weight regain is common after the end of initial weight-loss treatment. Existing extended care programs (which typically provide sessions once per month) improve long-term weight-loss outcomes, but with only modest effect vs control. Objective: To determine whether weight regain is reduced via provision of telephone-based extended care on an adaptive (triggered by a study algorithm that estimates when individuals are at high risk for weight regain) vs static (provided once per month) schedule. Design, Setting, and Participants: A randomized clinical trial conducted between October 2019 and November 2024 in areas within driving distance of Gainesville, Florida. Adults with obesity enrolled in a 16-week weight-loss program; those who lost 5% or more of their baseline weight were eligible for randomization to 1 of 2 maintenance care conditions. Intervention: Participants received 20 months of telephone-based extended care support, delivered individually by a trained interventionist, either on an adaptive or static schedule. Main Outcomes and Measures: The primary outcome was weight change from month 4 (end of initial intervention) to month 24. Results: The 255 participants (mean [SD] age, 50.6 [11.3] years; 209 [82.0%] women; 51 [20.0%] Black, 25 [9.8%] Hispanic or Latino, and 170 [66.7%] non-Hispanic White) were randomized to an adaptive (128 participants) or static (127 participants) group. Weight regain from month 4 to 24 was 1.27 (95% CI, 0.07 to 2.47) kg in the adaptive vs 1.75 (95% CI, 0.43 to 3.06) kg in the static group, with no difference by condition. At month 24, adaptive participants maintained a mean (SD) weight loss of 8.1% (7.8%) from initial intervention baseline while static participants maintained 7.9% (8.5%); there was not a significant difference in the proportion of participants maintaining weight loss of 5% or more (adaptive, 59.5%; static, 59.8%). Conclusions and relevance: In this randomized clinical trial, participants in both conditions were successful at maintaining initial weight loss 20 months after the end of a weight-loss program, but providing extended care on an adaptive schedule did not confer additional benefit vs the once-per-month static schedule. Future research should investigate whether more precise algorithms of high-risk periods for weight regain can be developed and whether these models can improve weight maintenance outcomes. Moreover, given success of participants in both conditions at maintaining initial weight loss, future research should also investigate methods of improving the implementation and dissemination potential of telephone-based extended care interventions. Trial Registration: ClinicalTrials.gov Identifier: NCT04116853.

  PublisherOA PDFDOI

• CReLeRI: Explainable, Concept-centric, Representation, Learning, Reasoning, and Interaction Video Analysis System

  2025-10-25

  article

  Existing video analysis models often lack explainability, perform poorly on long videos, and frequently hallucinate. Commercial solutions are closed-source and costly. We introduce CReLeRI, an open-source system for action detection in untrimmed videos. CReLeRI segments videos using scene and action transitions, detects actions and their arguments and grounds them in 3D space to improve interpretability and reduce hallucinations. The system promotes transparency and trust in AI-driven analysis of complex, real-world videos. A demonstration video is also available.

  PublisherDOI

• Deep Learning Forecast of Cognitive Workload Using fNIRS Data

  2024-05-15 · 6 citations

  article

  Introduction: In the domain of helicopter piloting, the pilot’s performance is driven by many cognitive processes, demanding substantial cognitive resources. The pilot must maintain situation awareness and perform rapid decision-making. An objective of integrated helicopter technologies is to predict and effectively manage pilot cognitive workload to ensure safety and efficiency throughout flight. Methods: In this study, we collected data on seven participants, including three experienced pilots, using a UH-60V cockpit simulator to perform 46 distinct trials under various flight conditions. fNIRS neuroimaging was used to collect high-resolution neurophysiological data for exploring and forecasting cognitive workload using a collection of deep learning models. Model implementation: Three deep learning architectures are detailed in this work: a stacked LSTM model, a CNN-LSTM hybrid, and a transformer model. Results: An evaluation of three Seq2Seq models, each with two distinct forecasting lengths (10s and 30s), revealed LSTM-based architectures as superior performers for 10s forecasting tasks. Discussion: The LSTM-based models’ superior performance suggested potential limitations with the transformer’s self-attention mechanisms for our specific application. Surprisingly, the CNN-LSTM architecture did not surpass the stacked LSTM model’s performance during forecasting tasks. Conclusion: Future research directions include exploring diverse time-series Seq2Seq methods and forecasting cognitive workload as ordinal measures, offering insights into shifting cognitive demands.

  PublisherDOI

• "I used to live in Florida": Exploring the Impact of Spam Call Warning Accuracy on Callee Decision-Making

  2024-01-01 · 1 citations

  articleOpen access
  PublisherDOI

• A low‐burden, self‐weighing intervention to prevent weight gain in adults with obesity who do not enroll in comprehensive treatment

  Obesity Science & Practice · 2024-03-20 · 2 citations

  articleOpen access

  Background: For individuals who are eligible but unlikely to join comprehensive weight loss programs, a low burden self-weighing intervention may be a more acceptable approach to weight management. Methods: who reported lack of interest in joining a comprehensive weight loss program, or did not enroll in a comprehensive program after being provided program information. In the self-weighing intervention, participants were asked to weigh themselves daily on a cellular connected scale and were sent text messages every other week with tailored weight change feedback, including messages encouraging use of comprehensive programs if weight gain occurred. Results: Of 86 eligible patients, 39 enrolled (45.3%) in the self-weighing intervention. Self-weighing occurred on average 4.6 days/week (SD = 1.4). At 12 months, 12 participants (30.8%) lost ≥3% baseline weight, 11 (28.2%) experienced weight stability (±3% baseline), 6 (15.4%) gained ≥3% of baseline weight, and 10 (25.6%) did not have available weight data to evaluate. Three participants reported joining a weight loss program during the intervention (7.7%). Participants reported high intervention satisfaction in quantitative ratings (4.1 of 5), and qualitative interviews identified areas of satisfaction (e.g., timing and content of text messages) and areas for improvement (e.g., increasing personalization of text messages). Conclusion: A low-burden self-weighing intervention can reach adults with overweight/obesity who would be unlikely to engage in comprehensive weight loss programs; the efficacy of this intervention for preventing weight gain should be further evaluated in a randomized trial.

  PublisherOA PDFDOI

• Human-Centered Evaluation of EMG-Based Upper-Limb Prosthetic Control Modes

  UNC Libraries · 2024-10-26

  articleOpen access

  The aim of this study was to experimentally test the effects of different electromyographic-based prosthetic control modes on user task performance, cognitive workload, and perceived usability to inform further human-centered design and application of these prosthetic control interfaces. We recruited 30 able-bodied participants for a between-subjects comparison of three control modes: direct control (DC), pattern recognition (PR), and continuous control (CC). Multiple human-centered evaluations were used, including task performance, cognitive workload, and usability assessments. To ensure that the results were not task-dependent, this study used two different test tasks, including the clothespin relocation task and Southampton hand assessment procedure-door handle task. Results revealed performance with each control mode to vary among tasks. When the task had high-angle adjustment accuracy requirements, the PR control outperformed DC. For cognitive workload, the CC mode was superior to DC in reducing user load across tasks. Both CC and PR control appear to be effective alternatives to DC in terms of task performance and cognitive load. Furthermore, we observed that, when comparing control modes, multitask testing and multifaceted evaluations are critical to avoid task-induced or method-induced evaluation bias. Hence, future studies with larger samples and different designs will be needed to expand the understanding of prosthetic device features and workload relationships.

  PublisherDOI

Recent grants

• CAREER: Next Generation Mulitmodal Interfaces

  NSF · $504k · 2018–2025

Frequent coauthors

• Lisa Anthony

  University of Florida

  18 shared

• Isaac Wang

  James Madison University

  17 shared

• Julia Woodward

  University of South Florida

  14 shared

• Sarah Morrison-Smith

  Hamilton College

  12 shared

• Edward Lank

  University of Waterloo

  12 shared

• He Huang

  Chongqing Technology and Business University

  11 shared

• Noelle Noyes

  University of Minnesota

  10 shared

• David Kaber

  9 shared

Labs

• Ruiz HCI LabPI

Awards & honors

• NSF CAREER Award (2018 – 2023)