David Donoho

About

I have studied the exploitation of sparse signals in signal recovery, including for denoising, superresolution, and solution of underdetermined equations. This research with collaborators showed that ell-1 penalization was an effective and even optimal way to exploit sparsity of the object to be recovered. Compressed sensing has impacted scientific and technical fields, including magnetic resonance imaging in medicine, where it has been implemented in FDA-approved medical imaging protocols already used for millions of patient MRIs. In recent years, my postdocs and students have been studying large-scale covariance matrix estimation, large-scale matrix denoising, detection of rare and weak signals among many pure noise non-signals, compressed sensing and related scientific imaging problems, and most recently, empirical deep learning.

Research topics

• Mathematics
• Computer science
• Algorithm
• Artificial intelligence
• Combinatorics

Selected publications

• Prevalence of neural collapse during the terminal phase of deep learning training

  Proceedings of the National Academy of Sciences · 2020-09-21 · 328 citations

  articleOpen accessSenior authorCorresponding

  (NC), involving four deeply interconnected phenomena. (NC1) Cross-example within-class variability of last-layer training activations collapses to zero, as the individual activations themselves collapse to their class means. (NC2) The class means collapse to the vertices of a simplex equiangular tight frame (ETF). (NC3) Up to rescaling, the last-layer classifiers collapse to the class means or in other words, to the simplex ETF (i.e., to a self-dual configuration). (NC4) For a given activation,…

  PublisherOA PDFDOI

• The science of deep learning

  Proceedings of the National Academy of Sciences · 2020-11-23 · 51 citations

  articleOpen accessCorresponding

  Scientists today have completely different ideas of what machines can learn to do than we had only 10 y ago. In image processing, speech and video processing, machine vision, natural language processing, and classic two-player games, in particular, the state-of-the-art has been rapidly pushed forward over the last decade, as a series of machine-learning performance records were achieved for publicly organized challenge problems. In many of these challenges, the records now meet or exceed human p…

  PublisherOA PDFDOI

• Data Science at the Singularity

  Harvard Data Science Review · 2024-01-29 · 42 citations

  articleOpen access1st authorCorresponding

  Something fundamental to computation-based research has really changed in the last ten years. In certain fields, progress is simply dramatically more rapid than previously. Researchers in affected fields are living through a period of profound transformation, as the fields undergo a transition to frictionless reproducibility (FR). This transition markedly changes the rate of spread of ideas and practices, affects scientific mindsets and the goals of science, and erases memories of much that came…

  PublisherOA PDFDOI

• ScreeNOT: Exact MSE-optimal singular value thresholding in correlated noise

  The Annals of Statistics · 2023-02-01 · 24 citations

  articleOpen access1st authorCorresponding

  We derive a formula for optimal hard thresholding of the singular value decomposition in the presence of correlated additive noise; although it nominally involves unobservables, we show how to apply it even where the noise covariance structure is not a priori known or is not independently estimable. The proposed method, which we call ScreeNOT, is a mathematically solid alternative to Cattell’s ever-popular but vague scree plot heuristic from 1966. ScreeNOT has a surprising oracle property: it ty…

  PublisherOA PDFDOI

• Principled and interpretable alignability testing and integration of single-cell data

  Proceedings of the National Academy of Sciences · 2024-02-28 · 22 citations

  articleOpen access

  Single-cell data integration can provide a comprehensive molecular view of cells, and many algorithms have been developed to remove unwanted technical or biological variations and integrate heterogeneous single-cell datasets. Despite their wide usage, existing methods suffer from several fundamental limitations. In particular, we lack a rigorous statistical test for whether two high-dimensional single-cell datasets are alignable (and therefore should even be aligned). Moreover, popular methods c…

  PublisherOA PDFDOI

Recent grants

• "Big-Data" Asymptotics: Theory and Large-Scale Experiments

  NSF · $701k · 2014–2018

Frequent coauthors

• Jean‐Luc Starck

  CEA Cadarache

  62 shared

• Iain M. Johnstone

  42 shared

• Xiaoming Huo

  27 shared

• Andrea Montanari

  23 shared

• Jared Tanner

  21 shared

• Emmanuel J. Candès

  21 shared

• Ery Arias-Castro

  University of California, San Diego

  20 shared

• Matan Gavish

  Hebrew University of Jerusalem

  18 shared

Education

• Ph.D., Statistics

  Harvard University

  1984

• AB, Statistics

  Princeton University

  1978

Awards & honors

• 2022 IEEE Jack S. Kilby Signal Processing Medal

Similar researchers at Stanford University

• Bradley Efronh-106
• Jerome H. Friedmanh-103