About

Ennio Mingolla is an affiliated faculty member in the Electrical and Computer Engineering department at Northeastern University College of Engineering, where he also serves as the Chair of Speech Language Pathology & Audiology. His research focuses on the development and empirical testing of neural network models of visual perception, specifically the segmentation, grouping, and contour formation processes involved in early and middle vision in primates. He is interested in how these models transition to technological applications. Dr. Mingolla holds a PhD from the University of Connecticut, an M.Ed. from Boston University, and has completed Peace Corps Training in Kakata, Liberia, as well as earning an A.B. from Harvard College. His contributions to the field include delivering the Ninth Annual Kanizsa Lecture in Trieste, Italy, and receiving the Helmholtz Award in 2007 from the International Neural Network Society for his achievements in research related to sensation and perception.

Research topics

• Artificial Intelligence
• Computer Science
• Computer vision
• Psychology
• Engineering
• Machine Learning
• Cognitive psychology
• Communication
• Cognitive science
• Human–computer interaction

Selected publications

• Psychophysics of neon color spreading: Chromatic and temporal factors are not limiting

  Vision Research · 2024-08-01 · 1 citations

  articleOpen access

  Neon color spreading (NCS) is an illusory color phenomenon that provides a dramatic example of surface completion and filling-in. Numerous studies have varied both spatial and temporal aspects of the neon-generating stimulus to explore variations in the strength of the effect. Here, we take a novel, parametric, low-level psychophysical approach to studying NCS in two experiments. In Experiment 1, we test the ability of both cone-isolating and equiluminant stimuli to generate neon color spreading for both increments and decrements in cone modulations. As expected, sensitivity was low to S(hort-wavelength) cone stimuli due to their poor spatial resolution, but sensitivity was similar for the other color directions. We show that when these differences in detection sensitivity are accounted for, the particular cone type, and the polarity (increment or decrement), make little difference in generating neon color spreading, with NCS visible at about twice detection threshold level in all cases. In Experiment 2, we use L-cone flicker modulations (reddish and greenish excursions around grey) to study sensitivity to NCS as a function of temporal frequency from 0.5 to 8 Hz. After accounting for detectability, the temporal contrast sensitivity functions for NCS are approximately constant or even increase over the studied frequency range. Therefore there is no evidence in this study that the processes underlying NCS are slower than the low-level processes of simple flicker detection. These results point to relatively fast mechanisms, not slow diffusion processes, as the substrate for NCS.

  PublisherDOI

• Tracking objects that change in appearance with phase synchrony

  arXiv (Cornell University) · 2024-10-02

  preprintOpen access

  International audience

  PublisherOA PDF

• Extreme Image Transformations Facilitate Robust Latent Object Representations

  arXiv (Cornell University) · 2023-09-19

  preprintOpen accessSenior author

  Adversarial attacks can affect the object recognition capabilities of machines in wild. These can often result from spurious correlations between input and class labels, and are prone to memorization in large networks. While networks are expected to do automated feature selection, it is not effective at the scale of the object. Humans, however, are able to select the minimum set of features required to form a robust representation of an object. In this work, we show that finetuning any pretrained off-the-shelf network with Extreme Image Transformations (EIT) not only helps in learning a robust latent representation, it also improves the performance of these networks against common adversarial attacks of various intensities. Our EIT trained networks show strong activations in the object regions even when tested with more intense noise, showing promising generalizations across different kinds of adversarial attacks.

  PublisherOA PDFDOI

• Extreme image transformations affect humans and machines differently

  Biological Cybernetics · 2023 · 1 citations

  Senior authorCorresponding
  • Computer Science
  • Artificial Intelligence
  • Computer Science

  Some recent artificial neural networks (ANNs) claim to model aspects of primate neural and human performance data. Their success in object recognition is, however, dependent on exploiting low-level features for solving visual tasks in a way that humans do not. As a result, out-of-distribution or adversarial input is often challenging for ANNs. Humans instead learn abstract patterns and are mostly unaffected by many extreme image distortions. We introduce a set of novel image transforms inspired by neurophysiological findings and evaluate humans and ANNs on an object recognition task. We show that machines perform better than humans for certain transforms and struggle to perform at par with humans on others that are easy for humans. We quantify the differences in accuracy for humans and machines and find a ranking of difficulty for our transforms for human data. We also suggest how certain characteristics of human visual processing can be adapted to improve the performance of ANNs for our difficult-for-machines transforms.

  PublisherOA PDFDOI

• Extreme Image Transformations Affect Humans and Machines Differently

  arXiv (Cornell University) · 2022-11-30

  preprintOpen accessSenior author

  Some recent artificial neural networks (ANNs) claim to model aspects of primate neural and human performance data. Their success in object recognition is, however, dependent on exploiting low-level features for solving visual tasks in a way that humans do not. As a result, out-of-distribution or adversarial input is often challenging for ANNs. Humans instead learn abstract patterns and are mostly unaffected by many extreme image distortions. We introduce a set of novel image transforms inspired by neurophysiological findings and evaluate humans and ANNs on an object recognition task. We show that machines perform better than humans for certain transforms and struggle to perform at par with humans on others that are easy for humans. We quantify the differences in accuracy for humans and machines and find a ranking of difficulty for our transforms for human data. We also suggest how certain characteristics of human visual processing can be adapted to improve the performance of ANNs for our difficult-for-machines transforms.

  PublisherOA PDFDOI

• Figure-Ground Segregation, Computational Neural Models of

  Encyclopedia of Computational Neuroscience · 2022-01-01

  book-chapterSenior authorCorresponding
  PublisherDOI

• Tracking Without Re-recognition in Humans and Machines.

  HAL (Le Centre pour la Communication Scientifique Directe) · 2021-12-06 · 3 citations

  articleOpen access

  Imagine trying to track one particular fruitfly in a swarm of hundreds. Higher biological visual systems have evolved to track moving objects by relying on both appearance and motion features. We investigate if state-of-the-art deep neural networks for visual tracking are capable of the same. For this, we introduce PathTracker, a synthetic visual challenge that asks human observers and machines to track a target object in the midst of identical-looking "distractor" objects. While humans effortlessly learn PathTracker and generalize to systematic variations in task design, state-of-the-art deep networks struggle. To address this limitation, we identify and model circuit mechanisms in biological brains that are implicated in tracking objects based on motion cues. When instantiated as a recurrent network, our circuit model learns to solve PathTracker with a robust visual strategy that rivals human performance and explains a significant proportion of their decisionmaking on the challenge. We also show that the success of this circuit model extends to object tracking in natural videos. Adding it to a transformer-based architecture for object tracking builds tolerance to visual nuisances that affect object appearance, resulting in a new state-of-the-art performance on the largescale TrackingNet object tracking challenge. Our work highlights the importance of building artificial vision models that can help us better understand human vision and improve computer vision.

  Publisher

• Tracking Without Re-recognition in Humans and Machines

  arXiv (Cornell University) · 2021 · 3 citations

  • Computer Science
  • Artificial Intelligence
  • Computer Science

  Imagine trying to track one particular fruitfly in a swarm of hundreds. Higher biological visual systems have evolved to track moving objects by relying on both appearance and motion features. We investigate if state-of-the-art deep neural networks for visual tracking are capable of the same. For this, we introduce PathTracker, a synthetic visual challenge that asks human observers and machines to track a target object in the midst of identical-looking "distractor" objects. While humans effortlessly learn PathTracker and generalize to systematic variations in task design, state-of-the-art deep networks struggle. To address this limitation, we identify and model circuit mechanisms in biological brains that are implicated in tracking objects based on motion cues. When instantiated as a recurrent network, our circuit model learns to solve PathTracker with a robust visual strategy that rivals human performance and explains a significant proportion of their decision-making on the challenge. We also show that the success of this circuit model extends to object tracking in natural videos. Adding it to a transformer-based architecture for object tracking builds tolerance to visual nuisances that affect object appearance, resulting in a new state-of-the-art performance on the large-scale TrackingNet object tracking challenge. Our work highlights the importance of building artificial vision models that can help us better understand human vision and improve computer vision.

  PublisherOA PDFDOI

• The Challenge of Appearance-Free Object Tracking with Feedforward Neural Networks

  arXiv (Cornell University) · 2021-09-30 · 2 citations

  preprintOpen accessSenior author

  Nearly all models for object tracking with artificial neural networks depend on appearance features extracted from a "backbone" architecture, designed for object recognition. Indeed, significant progress on object tracking has been spurred by introducing backbones that are better able to discriminate objects by their appearance. However, extensive neurophysiology and psychophysics evidence suggests that biological visual systems track objects using both appearance and motion features. Here, we introduce $\textit{PathTracker}$, a visual challenge inspired by cognitive psychology, which tests the ability of observers to learn to track objects solely by their motion. We find that standard 3D-convolutional deep network models struggle to solve this task when clutter is introduced into the generated scenes, or when objects travel long distances. This challenge reveals that tracing the path of object motion is a blind spot of feedforward neural networks. We expect that strategies for appearance-free object tracking from biological vision can inspire solutions these failures of deep neural networks.

  PublisherOA PDFDOI

• The Challenge of Appearance-Free Object Tracking with Feedforward Neural\n Networks

  arXiv (Cornell University) · 2021 · 1 citations

  Senior authorCorresponding
  • Artificial Intelligence
  • Computer Science
  • Artificial Intelligence

  Nearly all models for object tracking with artificial neural networks depend\non appearance features extracted from a "backbone" architecture, designed for\nobject recognition. Indeed, significant progress on object tracking has been\nspurred by introducing backbones that are better able to discriminate objects\nby their appearance. However, extensive neurophysiology and psychophysics\nevidence suggests that biological visual systems track objects using both\nappearance and motion features. Here, we introduce $\\textit{PathTracker}$, a\nvisual challenge inspired by cognitive psychology, which tests the ability of\nobservers to learn to track objects solely by their motion. We find that\nstandard 3D-convolutional deep network models struggle to solve this task when\nclutter is introduced into the generated scenes, or when objects travel long\ndistances. This challenge reveals that tracing the path of object motion is a\nblind spot of feedforward neural networks. We expect that strategies for\nappearance-free object tracking from biological vision can inspire solutions\nthese failures of deep neural networks.\n

  PublisherOA PDFDOI

Frequent coauthors

• Stephen Grossberg

  Boston University

  100 shared

• A. Yazdanbakhsh

  18 shared

• Gail A. Carpenter

  Iowa State University

  18 shared

• S. Grossberg

  Boston University

  17 shared

• Gennady Livitz

  Connect

  17 shared

• R Linsker

  San Francisco State University

  16 shared

• James L. Beck

  16 shared

• Federico Faggin

  16 shared

Education

• PhD, Psychology

  University of Connecticut

  1983

Awards & honors

• Helmholtz Award, 2007, from the International Neural Network…
• Delivered the Ninth Annual Kanizsa Lecture, Trieste, Italy,…