About

David H Brainard, PhD, is a Professor of Psychology in the Department of Psychology at the University of Pennsylvania's Perelman School of Medicine. His primary research interests focus on human vision, machine vision, and the computational modeling of visual processing. He investigates how the visual system estimates object properties from the light signal incident at the eye, conducting psychophysical experiments to explore how object color appearance relates to surface properties under various illumination conditions and how color is used for object identification. Additionally, he is interested in developing machine visual systems that mimic human performance and in understanding the neural mechanisms underlying vision. His expertise encompasses visual perception and its neural mechanisms, digital image processing, psychophysics, computational modeling, and fMRI techniques.

Research topics

• Computer Science
• Sociology
• Artificial Intelligence
• Linguistics
• Anthropology
• Psychology

Selected publications

• Author response: Comprehensive characterization of human color discrimination thresholds

  2026-05-20

  peer-reviewOpen accessSenior author

  Color discrimination thresholds—the smallest detectable color differences—provide a benchmark for models of color vision, enable quantitative evaluation of eye diseases, and inform the design of display technologies. Despite their importance, a comprehensive characterization of these thresholds has long been considered intractable due to the psychophysical curse of dimensionality. Here, we address this challenge using a novel semi-parametric Wishart Process Psychophysical Model (WPPM), which leverages the feature that the internal noise limiting color discrimination varies smoothly across stimulus space. The model was fit to data collected with a non-parametric adaptive trial-placement procedure, enabling efficient stimulus selection. Together, through the combination of adaptive trial placement and post hoc WPPM fitting, we achieved a comprehensive characterization of color discrimination in the isoluminant plane with only ∼6,000 trials per participant (N = 8). Once fit, the WPPM allows readouts of discrimination performance for any stimulus pair. We validated these readouts against 25 probe psychometric functions, measured with an additional 6,000 trials per participant held out from model fitting. In conclusion, our study provides a foundational dataset for color vision, and our approach generalizes beyond color to any domain in which the internal noise limiting performance varies smoothly across stimulus space, offering a powerful and efficient method for comprehensively characterizing various perceptual discrimination thresholds.

  PublisherDOI

• Deep neural networks trained for estimating reflectance and illumination achieve lightness constancy differently than human observers

  Journal of Vision · 2026-02-18

  articleOpen access

  Lightness constancy, the ability to create perceptual representations that are strongly correlated with surface reflectance despite variations in lighting and context, is a challenging computational problem. Indeed, it has proven difficult to develop image-computable models of how human vision achieves a substantial degree of lightness constancy in complex scenes. Recently, convolutional neural networks have been developed that are proficient at estimating reflectance, but little is known about how they achieve this, or whether they are good models of human vision. We examined this question by training a convolutional neural network to estimate reflectance and illumination in a computer-rendered virtual world, and evaluating both the convolutional neural network and human observers in a lightness matching task. In several conditions, we eliminated cues potentially supporting lightness constancy: local contrast, shading, shadows, and all contextual cues. We found that the network achieved a high degree of lightness constancy, outperforming human observers. However, we also found that eliminating cues affected the convolutional neural network and humans very differently. Humans were most affected when local contrast cues were made uninformative, whereas the convolutional neural network mostly relied on shading and shadows. In a follow-up experiment, we found that the convolutional neural network could learn to exploit noise artifacts typically associated with ray tracing and correlated with illuminance, with potential implications for the many studies relying on ray-traced images. We conclude that convolutional neural networks can learn an effective, global strategy of estimating lightness, which is closer to an optimal strategy for the ensemble of scenes we studied than the computation used by human vision.

  PublisherOA PDFDOI

• A deep convolutional neural network trained for lightness constancy is susceptible to lightness illusions

  Journal of Vision · 2026-03-25

  articleOpen access

  Human viewers are able to perform tasks that depend on accurate estimates of surface reflectance, even across large changes in illumination and context. This is a remarkable ability, and successful image-computable models of how the visual system achieves this have remained elusive. Recently, deep convolutional neural networks (CNNs) have been developed that are adept at estimating surface reflectance. Here we evaluated one such network as a starting point for a new model of human lightness perception by testing whether it was susceptible to a range of classic lightness illusions. We implemented a CNN and trained it via supervised learning to estimate surface reflectance at each pixel in grayscale, rendered images of geometric objects. We examined the network's output on several illusions, including the argyle, Koffka, snake, simultaneous contrast, White's, and checkerboard illusions, as well as control figures. We included variants where low-luminance regions important to the illusions were generated either by low reflectance or by cast shadows. For comparison, we carried out a lightness matching experiment with human observers using the same stimuli, and also examined the outputs of three classic lightness and brightness models. The CNN largely removed lighting effects such as shading and shadows, and produced good reflectance estimates on a test set. It also qualitatively predicted the illusions perceived by humans in most cases, the exceptions being White's and checkerboard illusions. The CNN outperformed classical models, both at estimating reflectance and at tracking human lightness matches. These findings support a normative view of lightness perception and highlight the promise of deep learning models in this area.

  PublisherDOI

• An image-computable spatio-chromatic receptive field model of the midget retinal ganglion cell mosaic across the retina

  Journal of Computational Neuroscience · 2026-04-01

  articleOpen accessSenior author

  Image-computable models of primate retinal ganglion cell (RGC) mosaics that are synthesized and constrained jointly by optical, anatomical and physiological properties, and which operate on images defined by their spatial-spectral radiance, do not currently exist. Here, we deploy a novel computational framework which synthesizes mosaics of linear spatio-chromatic receptive fields (RFs) of ON midget RGCs (mRGCs) by integrating published anatomical, physiological, and optical quality measurements, all varying with eccentricity. We use the synthesized mRGC mosaics to simulate both in vivo and in vitro physiological experiments and demonstrate the model’s consistency with published data. The model enables computation of how visual performance is shaped by the representation of visual information provided by the linear spatiochromatic processing stage of midget RGCs. The developed computational framework carefully accounts for the effect of physiological optics on mRGC responses, enables comparison of in vivo and in vitro data, and allows exploration of how different assumptions about RF organization, such as selectivity for the type of cones pooled by the RF center mechanism, affect physiological responses and psychophysical performance. The open-source and freely available implementation provides a platform for understanding how the linear spatiochromatic receptive field representation of the mRGCs shapes visual performance, as well as a foundation for future work that incorporates response nonlinearities, temporal filtering, and extends to additional RGC mosaics.

  PublisherDOI

• Transformation-tolerant object recognition in tree shrews despite lacking a fovea

  bioRxiv (Cold Spring Harbor Laboratory) · 2026-04-14

  articleOpen access

  Abstract Object recognition depends on the ability to extract stable representations across changes in how they are viewed, yet it remains unclear how this capacity depends on visual acuity and cortical hierarchy. We combined behavioral testing and computational modeling to determine whether tree shrews, close relatives of primates with lower spatial acuity, can perform transformation-tolerant object recognition. Front-end modeling incorporating species-specific optics and photoreceptor sampling showed that, when scaled for acuity, tree shrew retinal filtering preserves the similarity structure of natural image categories relevant for object recognition. Behaviorally, tree shrews reliably discriminated complex objects across variations in position, scale, and viewpoint, including when embedded within natural scenes, and generalized to novel exemplars. Their recognition behavior was best explained by visual features emphasizing differences in global shape and size between objects and by representations from intermediate and deep layers of hierarchical neural network models. These results demonstrate that visual processing supporting object-level generalization can arise within visual systems lacking high-acuity front-end optics and establish the tree shrew as a key model for understanding the computational and evolutionary origins of high-level vision.

  PublisherDOI

• A Wishart Process model combined with adaptive sampling for efficiently capturing discrimination thresholds in high-dimensional stimulus spaces

  Journal of Vision · 2025-07-15

  articleOpen accessSenior author

  Traditional methods for trial selection do not support exhaustive characterization of perceptual thresholds in high-dimensional settings; they require exponentially more data with increasing stimulus dimensionality. Here, we study the efficacy of adaptive sampling for trial selection combined with a Wishart Process model (WPM) for dense characterization of thresholds. Specifically, we simulated color-discrimination performance using the CIELAB color space, which is intended to be perceptually uniform and which coarsely approximates human color sensitivity. We restricted attention to a plane in color space. We generated CIELAB-based 'ground-truth' responses for a forced-choice task in which an observer identifies the odd-colored stimulus (comparison) from two other stimuli that are identical to each other (reference). The reference (R & B calibrated monitor channel intensities) and comparison stimuli (R+ΔR & B+ΔB) were chosen using AEPsych, a non-parametric method that adaptively selects informative trials (Owens et al., 2021). The monitor G intensity was held fixed. We fit the generated responses with a semi-parametric WPM which expresses smoothness of performance over the stimulus space, on the assumption of multivariate-Gaussian performance-limiting internal noise. This approach accurately recovered the ‘ground-truth’ thresholds with 2,800 trials, ~10x fewer trials than required by conventional methods (~30,000 assuming 25 reference stimuli x 8 comparison directions/reference x 150 trials/direction). Evaluation using the Bures-Wasserstein distance showed close agreement between ‘ground-truth’ threshold ellipses and model predictions (mean = 0.009, SD = 0.003; compare to a baseline set by distance between the ‘ground-truth’ ellipses and their inscribed circles: mean = 0.040, SD = 0.010). Crucially, our approach densely characterizes discrimination, interpolating to predict thresholds for any comparison direction around any reference. The approach generalizes to higher stimulus dimensions (full color space) and to any modality where smoothly varying multivariate-Gaussian noise limits performance, for example auditory localization and motor reaching.

  PublisherDOI

• Poster Session: The effect of fixational eye-movements on the temporal summation at detection threshold: A simulation study

  Journal of Vision · 2025-04-01

  articleOpen access

  We explored how fixational eye movements (FEMs) affect threshold temporal summation of increment pulses using realistic simulations of early visual processing. Using the Image Systems Engineering Toolbox for Biology, we assessed performance in a spatial 2AFC increment detection task, where the observer identified whether a stimulus appeared on the left or right. The signal-known-exactly ideal observer was trained on the noise-free photocurrent output of the cone mosaic for both stimulus alternatives, with performance calculated using noisy instances of photocurrents, given FEMs knowledge. The stimuli, modelled as 0.24x2.2 arcmin increments of 543 nm light presented via an AOSLO, included both a single 2 ms flash and pairs of flashes separated by interstimulus intervals (ISI) of 17 ms, 33 ms, 100 ms, or 300 ms. Detection thresholds, defined as the stimulus contrast corresponding to 75% correct, were assessed with and without FEMs. Without FEMs, thresholds for detecting two flashes separated by 17-100 ms slightly increased with ISI but remained lower than those for a single flash. With FEMs, the modelled differences between single- and two-flash thresholds were less pronounced, suggesting that, at the level of photocurrent signals, FEMs reduce the benefits of temporal summation for detection. Future work will quantify this reduction by simulating FEMs with varying velocities and explore if adding a temporal adaptation stage improves effect of FEMs' on performance.

  PublisherDOI

• Comprehensive characterization of human color discrimination thresholds

  eLife · 2025-11-25 · 2 citations

  articleOpen accessSenior author

  Color discrimination thresholds—the smallest detectable color differences—provide a benchmark for models of color vision, enable quantitative evaluation of eye diseases, and inform the design of display technologies. Despite their importance, a comprehensive characterization of these thresholds has long been considered intractable due to the psychophysical curse of dimensionality. Here, we address this challenge using a novel semi-parametric Wishart Process Psychophysical Model (WPPM), which leverages the feature that the internal noise limiting color discrimination varies smoothly across stimulus space. The model was fit to data collected with a non-parametric adaptive trial-placement procedure, enabling efficient stimulus selection. Together, through the combination of adaptive trial placement and post hoc WPPM fitting, we achieved comprehensive characterization of color discrimination in the isoluminant plane with only ~6,000 trials per participant (N = 8). Once fit, the WPPM allows readouts of discrimination performance for any stimulus pair. We validated these readouts against 25 probe psychometric functions, measured with an additional 6,000 trials per participant held out from model fitting. In conclusion, our study provides a foundational dataset for color vision, and our approach generalizes beyond color to any domain in which the internal noise limiting performance varies smoothly across stimulus space, offering a powerful and efficient method for comprehensively characterizing various perceptual discrimination thresholds.

  PublisherDOI

• A deep convolutional neural network trained for lightness constancy is susceptible to lightness illusions

  bioRxiv (Cold Spring Harbor Laboratory) · 2025-11-12 · 1 citations

  preprintOpen access

  Abstract Human viewers are able to perform tasks that depend on accurate estimates of surface reflectance, even across large changes in illumination and context. This is a remarkable ability, and successful image-computable models of how the visual system achieves this have remained elusive. Recently, deep convolutional neural networks (CNNs) have been developed that are adept at estimating surface reflectance. Here we evaluated one such network as a starting point for a new model of human lightness perception by testing whether it was susceptible to a range of classic lightness illusions. We implemented a CNN and trained it via supervised learning to estimate surface reflectance at each pixel in grayscale, rendered images of geometric objects. We examined the network’s output on several illusions, including the argyle, Koffka, snake, simultaneous contrast, White’s, and checkerboard illusions, as well as control figures. We included variants where low-luminance regions important to the illusions were generated either by low reflectance or by cast shadows. For comparison, we carried out a lightness matching experiment with human observers using the same stimuli, and also examined the outputs of three classic lightness and brightness models. The CNN largely removed lighting effects such as shading and shadows, and produced good reflectance estimates on a test set. It also qualitatively predicted the illusions perceived by humans in most cases, the exceptions being White’s and checkerboard illusions. The CNN outperformed classical models, both at estimating reflectance and at tracking human lightness matches. These findings support a normative view of lightness perception and highlight the promise of deep learning models in this area.

  PublisherDOI

• Modeling tree shrew high-level visual behaviors

  Journal of Vision · 2025-07-15

  articleOpen access

  A hallmark of primate vision is the ability to quickly recognize objects despite considerable variations in how an object is projected onto the retina. However, the evolutionary origins of this behavior remain poorly understood. Among the closest relatives to primates, tree shrews (Tupaia belangeri) offer unique insights into the evolution of visual processing. Their extensive extrastriate cortex and visually guided behaviors represent key adaptations that may have supported advanced object recognition in primates. We trained three adult tree shrews on a match-to-sample task using stimuli previously used to demonstrate complex object recognition in humans, macaques, and marmosets. Like primates, tree shrews successfully identified objects across variations in position, size, and orientation, and when embedded in complex scenes. Moreover, behavioral performance was correlated across shrews, suggesting they utilize a common shape representation. To gain deeper insight into the representations driving their behavior, we compared tree shrew performance with predictions from visual processing models. We accounted for tree shrew optics using a front-end visual system model, ISETBio, then employed deep convolutional neural networks (DCNN) to probe the visual representations emerging from core features of the primate visual system—hierarchical connectivity and convolutional processing. We analyzed the correspondence between DCNN layer representations and tree shrew behavioral performance, finding that layers best predicting tree shrew performance varied with task complexity. While this provides insights into the depth of processing, it does not reveal which specific stimulus features drive tree shrew behavior. Moreover, the most diagnostic stimulus features for tree shrew behavior may not be captured by DCNNs. Therefore, we are testing models representing specific aspects of processing, including local texture (Gabor-jet model), structural shape (skeletal model), and visual saliency (SALICON). These findings help establish tree shrews as a model for high-level processing and offer insights not just about whether, but how they discriminate complex objects.

  PublisherDOI

Recent grants

• NIH Grant R01EY024681

  NIH · $1.6M · 2021

• Color Constancy

  NIH · $5.8M · 1992–2019

• NIH Grant R29EY010016

  NIH · $461k · 1998

• P30 Core Grant For Vision Research

  NIH · $15.6M · 1997–2029

• NIH Grant F32EY006278

  NIH · $26k

Frequent coauthors

• Gustavo D. Aguirre

  65 shared

• Nicolas P. Cottaris

  California University of Pennsylvania

  57 shared

• Brian A. Wandell

  Stanford University

  37 shared

• Ana Radonjić

  University of Pennsylvania

  36 shared

• Manuel Spitschan

  Institute for Advanced Study

  32 shared

• Robert F. Cooper

  Cranfield University

  27 shared

• David R. Williams

  21 shared

• Jessica I. W. Morgan

  Penn Presbyterian Medical Center

  21 shared