About

Professor Alexander Huth leads research that employs quantitative and computational methods to understand how the human brain processes the natural world, with a particular focus on how the meaning of language is represented in the brain. His lab uses functional magnetic resonance imaging (fMRI) to record brain responses while participants listen to speech in the form of stories or podcasts. The research involves building encoding models that predict these brain responses based on the audio and transcript of the stories. These models leverage neural network language models to extract meaningful information from the stories, enabling detailed mapping of language representation across the brain. Professor Huth's work investigates the effectiveness of neural network language models in this context and demonstrates the ability to decode language directly from fMRI data. The datasets collected by his lab are shared freely, along with code and tutorials to facilitate the use of encoding models for language neuroscience. His research contributes to advancing the understanding of semantic language processing and the neural basis of language comprehension.

Research topics

• Computer Science
• Artificial Intelligence
• Psychology
• Neuroscience
• Natural Language Processing
• Cognitive psychology
• Algorithm
• Computer vision
• Speech recognition

Selected publications

• Fine-tuning language encoding models on slow fMRI improves prediction for fast ECoG

  ArXiv.org · 2026-05-19

  articleOpen accessSenior author

  Neuroscientists have recently turned to intracranial brain recording methods, like electrocorticography (ECoG), for human experiments because of the fine spatial and temporal resolution that they afford. Models trained on this data, however, are fundamentally restricted by the patient populations that can receive the implants necessary for recording. We propose using non-invasive fMRI to bridge the gap in training data. Using spoken language representations fine-tuned on fMRI, we build encoding models of ECoG. These representations showed improved prediction performance in ECoG, even though the temporal resolution of fMRI is two orders of magnitude worse. Prediction improved in frequency bands well beyond what is directly measured in fMRI. Next, to test the procedure's generalization ability, we fine-tuned models on fMRI responses that were temporally downsampled by a factor of 2. Despite the loss in resolution, these models were able to predict fMRI and ECoG responses at levels comparable to the original fMRI-tuned models. Finally, we showed that ECoG performance steadily scales with the amount of fMRI-tuning data. Our results show that "slow" data like fMRI can be a valuable resource for building better models of "fast" brain data like ECoG. In the future, integrating across multiple recording methods may further improve performance in other applications, like decoding.

  PublisherOA PDF

• Rapid cortical mapping with cross-participant encoding models

  bioRxiv (Cold Spring Harbor Laboratory) · 2026-03-27 · 1 citations

  articleOpen accessSenior author

  Voxelwise encoding models trained on functional MRI data can produce detailed maps of cortical organization. However, voxelwise encoding models must be trained on many hours of brain responses from each participant, limiting clinical applications. In this study, we introduce a cross-participant modeling framework for rapid cortical mapping. In this framework, voxelwise encoding models are trained on many hours of brain responses from previously scanned reference participants, and then transferred to a new participant by aligning brain responses using a small set of stimuli. We evaluated cross-participant encoding models on linguistic semantic mapping, non-linguistic semantic mapping, and auditory mapping. In each case we found that cross-participant encoding models had more accurate selectivity estimates and prediction performance than within-participant encoding models trained on the same amount of data from the new participant. We also found that cross-participant encoding models improved with the amount of data from each reference participant and the number of reference participants. These results demonstrate that cross-participant modeling can substantially reduce the amount of data required for accurate cortical mapping, which may facilitate new clinical applications of functional neuroimaging.

  PublisherOA PDFDOI

• Fine-tuning language encoding models on slow fMRI improves prediction for fast ECoG

  arXiv (Cornell University) · 2026-05-19

  preprintOpen accessSenior author

  Neuroscientists have recently turned to intracranial brain recording methods, like electrocorticography (ECoG), for human experiments because of the fine spatial and temporal resolution that they afford. Models trained on this data, however, are fundamentally restricted by the patient populations that can receive the implants necessary for recording. We propose using non-invasive fMRI to bridge the gap in training data. Using spoken language representations fine-tuned on fMRI, we build encoding models of ECoG. These representations showed improved prediction performance in ECoG, even though the temporal resolution of fMRI is two orders of magnitude worse. Prediction improved in frequency bands well beyond what is directly measured in fMRI. Next, to test the procedure's generalization ability, we fine-tuned models on fMRI responses that were temporally downsampled by a factor of 2. Despite the loss in resolution, these models were able to predict fMRI and ECoG responses at levels comparable to the original fMRI-tuned models. Finally, we showed that ECoG performance steadily scales with the amount of fMRI-tuning data. Our results show that "slow" data like fMRI can be a valuable resource for building better models of "fast" brain data like ECoG. In the future, integrating across multiple recording methods may further improve performance in other applications, like decoding.

  PublisherDOI

• Decoding concept representations in aphasia after stroke

  bioRxiv (Cold Spring Harbor Laboratory) · 2026-04-08

  articleOpen access

  Many stroke survivors with aphasia struggle to map their thoughts into words or motor plans. Neuroprostheses that decode concept representations could help these individuals communicate by predicting the words, phrases, or sentences that they are struggling to produce. Here we decoded concept representations measured using functional magnetic resonance imaging (fMRI) from participants with different aphasia profiles. The decoders generated continuous word sequences that could describe the concepts that the participants were hearing about, seeing, or imagining. To forecast how this approach would generalize across the heterogeneity of aphasia profiles, we characterized how stroke affects the anatomical organization and information capacity of conceptual processing. Mapping how concepts are organized across the brain, we found that conceptual tuning during non-linguistic processing was largely consistent between the participants with aphasia and neurologically healthy participants. Comparing information processing between the participants with aphasia and neurologically healthy participants, we found that both groups processed similar amounts of non-linguistic information. Our findings indicate that concept representations can be largely spared in individuals with aphasia and demonstrate how these representations can be decoded to support communication.

  PublisherOA PDFDOI

• Naturalistic auditory semantic mapping using high-density diffuse optical tomography

  2026-03-05

  article
  PublisherDOI

• Efficient uniform sampling explains non-uniform memory of narrative stories

  bioRxiv (Cold Spring Harbor Laboratory) · 2025-08-01

  preprintOpen accessSenior author

  Abstract Humans do not remember all experiences uniformly. We remember certain moments better than others, and central gist better than detail. Current theories focus exclusively on surprise to explain why some moments are better remembered, and do not explain gist memory. We propose that humans uniformly sample incoming information in time, which explains both non-uniform memory and gist. Rather than surprise, this model predicts that the mutual information between a given moment and the rest of the experience drives memory. To test this model, participants listened to narrative stories and recalled them immediately afterward. Using large language models to quantify the information structure of narrative stories and participants’ recall, we found that our parsimonious uniform sampling model explained memory better than earlier theories. These findings suggest an alternative, simpler account of human memory that does not rely on costly feedback mechanisms for prioritizing encoding of specific information.

  PublisherOA PDFDOI

• Semantic mapping of visual object categories in movies using very high-density diffuse optical tomography

  2025-03-19

  article

  Visual semantic mapping connects object category labels to language processing across extensive cortical regions. While fMRI is effective for semantic encoding (mapping) and decoding, it is costly and unsuitable for applications in natural environments. High-density diffuse optical tomography (HD-DOT) offers a cost-effective alternative for imaging in naturalistic settings. We developed a semantic encoding model using very high-density DOT (VHD-DOT). Participants (N=3) underwent three 89-minute VHD-DOT imaging sessions, including naturalistic movie clips (120 minutes training and 90 minutes testing data). Semantic maps were derived for single object categories using a GLM and a full set of categories using a voxelwise encoding model. VHD-DOT demonstrated high repeatability and sufficient signal quality across imaging sessions, as well as reproducible single-category and rich multiple-category semantic maps. This study confirms the feasibility of VHD-DOT for semantic mapping, building towards the development of brain-computer interfaces for individuals with language difficulties, like stroke-induced aphasia, in natural environments.

  PublisherDOI

• Semantic language decoding across participants and stimulus modalities

  Current Biology · 2025-02-06 · 10 citations

  articleOpen accessSenior author
  PublisherOA PDFDOI

• Visual Semantic Encoding and Identification of Naturalistic Movies via High-Density Diffuse Optical Tomography

  bioRxiv (Cold Spring Harbor Laboratory) · 2025-12-08

  articleOpen access

  Understanding how the brain represents meaning in real-world contexts is essential for both fundamental neuroscience and clinical applications. Brain encoding and decoding models from naturalistic stimuli provide a powerful window into semantic representations. Yet, existing approaches rely on a constrained scanning environment, or on conventional fNIRS, which has been limited to sparse sampling and/or block-design paradigms. Here, we tested whether high-density diffuse optical tomography (HD-DOT), an advanced high-density tomographic optical imaging method, can support semantic encoding and decoding using naturalistic movies. We collected 3.5 hours of naturalistic movie viewing data from six participants using stimuli labeled with 1,708 categories. Encoding models robustly predicted voxel-level responses, yielding single semantic category maps consistent with prior fMRI studies. In complementary decoding analyses, we showed that DOT responses captured sufficient semantic content to identify which clips participants viewed. To assess organization across individuals, we identified a shared low-dimensional semantic space that captures common semantic dimensions. Finally, clustering analyses revealed interpretable higher-order semantic dimensions like social and animate agents, objects vs natural organisms, and textural scenes, consistently mapped across the cortex. These findings demonstrate that DOT can recover distributed, high-dimensional semantic representations from naturalistic movies, bridging fMRI-level semantic mapping with the accessibility of optical imaging.

  PublisherDOI

• Evaluating scientific theories as predictive models in language neuroscience

  bioRxiv (Cold Spring Harbor Laboratory) · 2025-08-12

  preprintOpen accessSenior author

  the underlying phenomena, i.e. what features of the stimulus drive the response? We present Question Answering encoding models, a method for converting qualitative theories of language selectivity into highly accurate, interpretable models of brain responses. QA encoding models annotate a language stimulus by using a large language model to answer yes-no questions corresponding to qualitative theories. A compact QA encoding model that uses only 35 questions outperforms existing baselines at predicting brain responses in both fMRI and ECoG data. The model weights also provide easily interpretable maps of language selectivity across cortex; these maps show quantitative agreement with meta-analyses of the existing literature and selectivity maps identified in a follow-up fMRI experiment. These results demonstrate that LLMs can bridge the widening gap between qualitative scientific theories and data-driven models.

  PublisherOA PDFDOI

Frequent coauthors

• Paul S. Scotti

  Princeton University

  37 shared

• Eduard T. Klapwijk

  Erasmus University Rotterdam

  37 shared

• Arman P. Kulkarni

  University of Wisconsin–Madison

  37 shared

• Matan Mazor

  University of Oxford

  37 shared

• Javier S. Turek

  29 shared

• Jack L. Gallant

  University of California, Berkeley

  28 shared

• Shailee Jain

  18 shared

• Richard Antonello

  17 shared