About

Margrit Betke is a Professor in the Department of Computer Science at Boston University, where she is part of the Vision and Graphics group. Her research focuses on various aspects of computer vision, computer graphics, machine learning, and human-computer interaction. She contributes to advancing understanding and development in these fields through her academic work and leadership within the department.

Research topics

• Computer Science
• Artificial Intelligence
• Natural Language Processing
• Social psychology
• Theoretical computer science
• Psychology
• Linguistics
• Multimedia
• Human–computer interaction
• Data science
• Mathematics education

Selected publications

• Machine Learning Driven ‘Therapy Calculator’ for Self-Managed Digital Speech-Language Therapy for Individuals with Post-stroke Aphasia

  medRxiv · 2026-01-24

  articleOpen access

  ABSTRACT Individuals with post-stroke aphasia live with long-term disabilities, yet they do not know whether they will improve their communication and cognitive skills over time. We propose a “Therapy Calculator” to provide patients with a better understanding of likely recovery as they engage with therapy. Using a large dataset of rehabilitation outcomes from a digital therapeutic called Constant Therapy (3.5 million therapy sessions of 18,000+ users), we developed a machine learning algorithm that estimates the probability of improvement from one functional landmark (i.e., a given skill level) to the next in a functional domain (e.g., reading) while accounting for age, etiology, starting performance, and frequency and duration of therapy. This logistic regression model performed a binary classification task, i.e., whether patients can improve to the next landmark, with an average F1 score of all models at 0.84, suggesting reliable prediction of moving to the next landmark. Then, we created an online “Therapy Calculator” to assess a new user’s current functional level and demographic information, and make predictions by passing these features into models trained on relevant subsets of historical data. The findings indicate that our model can provide reliable predictions for patients beginning self-managed SLT, and therapy calculator is publicly available.

  PublisherOA PDFDOI

• Gen-AFFECT: Generation of Avatar Fine-grained Facial Expressions with Consistent identiTy

  2026-03-06

  article

  Different forms of customized 2D avatars are widely used in gaming applications, virtual communication, education, and content creation. However, existing approaches often fail to capture fine-grained facial expressions and struggle to preserve identity across different expressions. We propose Gen-AFFECT, a novel framework for personalized avatar generation that generates expressive and identity-consistent avatars with a diverse set of facial expressions. Our framework proposes conditioning a multimodal diffusion transformer on an extracted identity-expression representation. This enables identity preservation and representation of a wide range of facial expressions. Gen-AFFECT additionally employs consistent attention at inference for information sharing across the set of generated expressions, enabling the generation process to maintain identity consistency over the array of generated fine-grained expressions. Gen-AFFECT demonstrates superior performance compared to previous state-of-the-art methods on the basis of the accuracy of the generated expressions, the preservation of the identity and the consistency of the target identity across an array of fine-grained facial expressions.

  PublisherDOI

• Enhancing Text Entry for Users with Motor Impairments: Design and Evaluation of KeyGlide

  2025-06-25

  articleOpen accessSenior author
  PublisherDOI

• DebiasPI: Inference-time Debiasing by Prompt Iteration of a Text-to-Image Generative Model

  OpenBU (Boston University) · 2025-01-28

  preprintOpen accessSenior author

  Ethical intervention prompting has emerged as a tool to counter demographic biases of text-to-image generative AI models. Existing solutions either require to retrain the model or struggle to generate images that reflect desired distributions on gender and race. We propose an inference-time process called DebiasPI for Debiasing-by-Prompt-Iteration that provides prompt intervention by enabling the user to control the distributions of individuals' demographic attributes in image generation. DebiasPI keeps track of which attributes have been generated either by probing the internal state of the model or by using external attribute classifiers. Its control loop guides the text-to-image model to select not yet sufficiently represented attributes, With DebiasPI, we were able to create images with equal representations of race and gender that visualize challenging concepts of news headlines. We also experimented with the attributes age, body type, profession, and skin tone, and measured how attributes change when our intervention prompt targets the distribution of an unrelated attribute type. We found, for example, if the text-to-image model is asked to balance racial representation, gender representation improves but the skin tone becomes less diverse. Attempts to cover a wide range of skin colors with various intervention prompts showed that the model struggles to generate the palest skin tones. We conducted various ablation studies, in which we removed DebiasPI's attribute control, that reveal the model's propensity to generate young, male characters. It sometimes visualized career success by generating two-panel images with a pre-success dark-skinned person becoming light-skinned with success, or switching gender from pre-success female to post-success male, thus further motivating ethical intervention prompting with DebiasPI.

  PublisherOA PDFDOI

• Insights into Climate Change Narratives: Emotional Alignment and Engagement Analysis on TikTok

  2025-01-01

  articleOpen access

  Ge Gao, Zhengyang Shan, James Crissman, Ekaterina Novozhilova, YuCheng Huang, Arti Ramanathan, Margrit Betke, Derry Wijaya. Proceedings of the Fourth Workshop on NLP for Positive Impact (NLP4PI). 2025.

  PublisherOA PDFDOI

• GenEAva: Generating Cartoon Avatars with Fine-Grained Facial Expressions from Realistic Diffusion-based Faces

  ArXiv.org · 2025-04-10

  preprintOpen access

  Cartoon avatars have been widely used in various applications, including social media, online tutoring, and gaming. However, existing cartoon avatar datasets and generation methods struggle to present highly expressive avatars with fine-grained facial expressions and are often inspired from real-world identities, raising privacy concerns. To address these challenges, we propose a novel framework, GenEAva, for generating high-quality cartoon avatars with fine-grained facial expressions. Our approach fine-tunes a state-of-the-art text-to-image diffusion model to synthesize highly detailed and expressive facial expressions. We then incorporate a stylization model that transforms these realistic faces into cartoon avatars while preserving both identity and expression. Leveraging this framework, we introduce the first expressive cartoon avatar dataset, GenEAva 1.0, specifically designed to capture 135 fine-grained facial expressions, featuring 13,230 expressive cartoon avatars with a balanced distribution across genders, racial groups, and age ranges. We demonstrate that our fine-tuned model generates more expressive faces than the state-of-the-art text-to-image diffusion model SDXL. We also verify that the cartoon avatars generated by our framework do not include memorized identities from fine-tuning data. The proposed framework and dataset provide a diverse and expressive benchmark for future research in cartoon avatar generation.

  PublisherOA PDFDOI

• Aphasia severity prediction using a multi-modal machine learning approach

  NeuroImage · 2025-06-17 · 6 citations

  articleOpen access

  The present study examined an integrated multiple neuroimaging modality (T1 structural, Diffusion Tensor Imaging (DTI), and resting-state FMRI (rsFMRI)) to predict aphasia severity using Western Aphasia Battery-Revised Aphasia Quotient (WAB-R AQ) in 76 individuals with post-stroke aphasia. We employed Support Vector Regression (SVR) and Random Forest (RF) models with supervised feature selection and a stacked feature prediction approach. The SVR model outperformed RF, achieving an average root mean square error (RMSE) of 16.38±5.57, Pearson's correlation coefficient (r) of 0.70±0.13, and mean absolute error (MAE) of 12.67±3.27, compared to RF's RMSE of 18.41±4.34, r of 0.66±0.15, and MAE of 14.64±3.04. Resting-state neural activity and structural integrity emerged as crucial predictors of aphasia severity, appearing in the top 20% of predictor combinations for both SVR and RF. Finally, the feature selection method revealed that functional connectivity in both hemispheres and between homologous language areas is critical for predicting language outcomes in patients with aphasia. The statistically significant difference in performance between the model using only single modality and the optimal multi-modal SVR/RF model (which included both resting-state connectivity and structural information) underscores that aphasia severity is influenced by factors beyond lesion location and volume. These findings suggest that integrating multiple neuroimaging modalities enhances the prediction of language outcomes in aphasia beyond lesion characteristics alone, offering insights that could inform personalized rehabilitation strategies.

  PublisherDOI

• DebiasPI: Inference-Time Debiasing by Prompt Iteration of a Text-to-Image Generative Model

  Lecture notes in computer science · 2025-01-01 · 2 citations

  book-chapterSenior author
  PublisherDOI

• Walk and Read Less: Improving the Efficiency of Vision-and-Language Navigation via Tuning-Free Multimodal Token Pruning

  2025-01-01

  articleOpen accessSenior author
  PublisherOA PDFDOI

• CLARUS: Contrastive Learning and Anomaly Detection for Respiratory Ultrasound Screening

  2025-09-15

  articleSenior author

  In-Time and accurate identification of pulmonary consolidations in pediatric lung ultrasound (LUS) is critical for early intervention and treatment. While deep learning (DL) based methods have shown potential to support clinical evaluation, their development is often hindered by the inherent variability in image acquisition across operators and the limited availability of annotated LUS datasets. These challenges complicate the training of robust and generalizable DL models for real-world deployment. In this study, we present CLARUS, a compact and interpretable framework that uses contrastive self-supervised learning (SSL) and unsupervised anomaly detection using latent features. CLARUS integrates three backbone CNN architectures (ResNet18, DenseNet121 and EfficientNetB0) to learn meaningful latent representations. For the proof of concept, the framework is used to classify individual ultrasound frames for the presence or absence of pulmonary consolidations associated with pediatric pneumonia. Results show that ResNet18 outperforms the other backbone architectures in a self-supervised environment (AUC: 0.80), while EfficientNetB0 offers stronger performance for distance-based anomaly detection (AUC: 0.63). In particular, model predictions are sensitive to user-defined thresholds, enabling clinicians to tailor the system’s sensitivity to specific diagnostic needs. These findings highlight the adaptability of CLARUS and its potential to assist in automated and interpretable LUS assessment in resource-constrained or high-variability environments while not being dependent on large labeled datasets.

  PublisherDOI

Recent grants

• HCC: Large: Intelligent Tracking Systems that Reason about Group Behavior

  NSF · $2.9M · 2009–2015

• MRI Collaborative: Development of iRehab, an Intelligent Closed-Loop Instrument for Adaptive Rehabilitation

  NSF · $200k · 2013–2017

• HCC: Intelligent Interfaces to Empower People with Disabilities to Participate in the Information Society

  NSF · $386k · 2007–2011

• BIGDATA: IA: Multiplatform, Multilingual, and Multimodal Tools for Analyzing Public Communication in over 100 Languages

  NSF · $1.0M · 2018–2023

• INT: Collaborative Research: Detecting, Predicting and Remediating Student Affect and Grit Using Computer Vision

  NSF · $615k · 2016–2022

Frequent coauthors

• Stan Sclaroff

  Boston University

  31 shared

• John Magee

  27 shared

• Zheng Wu

  25 shared

• Lauren Etter

  Boston University

  22 shared

• Christopher Gill

  20 shared

• Libertario Demi

  University of Trento

  19 shared

• Bindu N. Setty

  Boston Medical Center

  19 shared

• Umair Khan

  University of Trento

  19 shared

Labs

• Data Mining & Data ManagementPI

Education

• Ph.D., Computer Science and Electrical Engineering

  Massachusetts Institute of Technology

  1995

Awards & honors

• National Science Foundation Faculty Early Career Development…