David M. Blei

Causal Inference for Recommender Systems

2020 · 129 citations

The task of recommender systems is classically framed as a prediction of users’ preferences and users’ ratings. However, its spirit is to answer a counterfactual question: “What would the rating be if we ‘forced’ the user to watch the movie?” This is a question about an intervention, that is a causal inference question. The key challenge of this causal inference is unobserved confounders, variables that affect both which items the users decide to interact with and how they rate them. To this end…

Joint representation and visualization of derailed cell states with Decipher

Genome biology · 2025-07-23 · 5 citations

Biological insights often depend on comparing conditions such as disease and health. Yet, we lack effective computational tools for integrating single-cell genomics data across conditions or characterizing transitions from normal to deviant cell states. Here, we present Decipher, a deep generative model that characterizes derailed cell-state trajectories. Decipher jointly models and visualizes gene expression and cell state from normal and perturbed single-cell RNA-seq data, revealing shared and…

Estimating the Hallucination Rate of Generative AI

arXiv (Cornell University) · 2024-06-11 · 4 citations

This paper presents a method for estimating the hallucination rate for in-context learning (ICL) with generative AI. In ICL, a conditional generative model (CGM) is prompted with a dataset and a prediction question and asked to generate a response. One interpretation of ICL assumes that the CGM computes the posterior predictive of an unknown Bayesian model, which implicitly defines a joint distribution over observable datasets and latent mechanisms. This joint distribution factorizes into two co…

Estimating wage disparities using foundation models

Proceedings of the National Academy of Sciences · 2025-05-30 · 3 citations

The rise of foundation models marks a paradigm shift in machine learning: instead of training specialized models from scratch, foundation models are trained on massive datasets before being adjusted or fine-tuned to make predictions on smaller datasets. Initially developed for text, foundation models have also excelled at making predictions about social science data. However, while many estimation problems in the social sciences use prediction as an intermediate step, they ultimately require dif…

A Bayesian model of underreporting for sexual assault on college campuses

The Annals of Applied Statistics · 2024-10-31 · 2 citations