About

Bret Payseur is a Professor in the Department of Genetics and Medical Genetics at the University of Wisconsin–Madison. He earned his Ph.D. from the University of Arizona in 2003 and completed postdoctoral research at Cornell University. His research focuses on understanding mechanisms of evolution through genetics and genomics. Payseur and his students investigate how organisms adapt to new environments, the evolution of meiotic recombination, and the process of speciation from a genetic perspective. His work primarily utilizes natural variation in the house mouse as a powerful genetic model organism to explore these evolutionary processes.

Research topics

• Biology
• Computer Science
• Genetics
• Artificial Intelligence
• Zoology
• Demography
• Computational biology
• Data science
• Ecology
• Evolutionary biology

Selected publications

• Population Genomics of Giant Mice from the Faroe Islands: Hybridization, Colonization, and a Novel Challenge to Identifying Genomic Targets of Selection

  bioRxiv (Cold Spring Harbor Laboratory) · 2025-01-22

  preprintOpen access1st authorCorresponding

  Abstract Populations that colonize islands provide unique insights into demography, adaptation, and the spread of invasive species. House mice on the Faroe Islands evolved exceptionally large bodies after colonization, generating interest from biologists since Darwin. To reconstruct the evolutionary history of these mice, we sequenced genomes of population samples from three Faroe Islands (Sandoy, Nólsoy, and Mykines) and Norway as a mainland comparison. Mice from the Faroe Islands are hybrids between the subspecies Mus musculus domesticus and M. m. musculus , with ancestry alternating along the genome. Analyses based on the site frequency spectrum of single nucleotide polymorphisms and the ancestral recombination graph (ARG) indicate that mice arrived on the Faroe Islands on a timescale consistent with transport by Norwegian Vikings, with colonization of Sandoy likely preceding colonization of Nólsoy. Substantial reductions in nucleotide diversity and effective population size associated with colonization suggest that mice on the Faroe Islands evolved large body size during periods of heightened genetic drift. Genomic scans for positive selection uncover windows with unusual site frequency spectra, but this pattern is mostly generated by clusters of singletons in individual mice. Variants showing evidence of selection in both Nólsoy and Sandoy based on the ARG are enriched for genes with neurological functions. Our findings reveal a dynamic evolutionary history for the enigmatic mice from Faroe Island and emphasize the challenges that accompany population genomic inferences in island populations. Significance Statement Populations that colonize islands are expected to have unusual histories compared to their mainland counterparts. Using population genomic data, we conclude that giant mice living on the Faroe Islands originated from hybrids, invaded the islands on a timescale consistent with transport by Vikings, and persisted despite drastic reductions in population size. We also uncover a novel challenge to scanning genomes for genes involved in adaptation.

  PublisherOA PDFDOI

• The Genomic Imprint of Chromosomal Inversions and Demographic History in Island Populations of Deer Mice

  Molecular Biology and Evolution · 2025-10-04

  articleOpen accessSenior author

  Populations that colonize islands experience novel selective pressures, fluctuations in size, and changes to their connectivity. Owing to their unique geographic setting, islands can function as natural laboratories in which to examine the interactions between demographic history and natural selection replicated across isolated populations. We used whole genome sequences of wild-caught deer mice (Peromyscus maniculatus) from two islands (Saturna and Pender) and one mainland location (Maple Ridge) in the Gulf Islands region of coastal British Columbia to investigate two primary determinants of genome-wide diversity: chromosomal inversions and non-equilibrium demographic history. We found that segregating inversions produce characteristic, large-scale distortions in allele frequencies and linkage disequilibrium that make it possible to identify and characterize them from short-read sequence data. Patterns of variation within and between karyotypes indicate that six inversion polymorphisms have been maintained by a shared history of balancing selection in both island and mainland populations. Whereas the estimated timing of contemporary population splits is consistent with the isolation of island populations from each other following the Last Glacial Maximum, ancestral island and mainland lineages are inferred to have diverged much earlier. These aspects of demographic history suggest that shared inversions existed long ago in a common ancestor or spread via limited gene flow between ancestral island and mainland lineages. Our results raise the possibility that inversions segregating among Gulf Islands populations are on similar evolutionary trajectories, providing a contrast to previous findings in mainland P. maniculatus and contributing to the emerging portrait of inversion evolution in this species.

  PublisherOA PDFDOI

• Phenotypic and Developmental Dissection of an Instance of the Island Rule

  bioRxiv (Cold Spring Harbor Laboratory) · 2025-01-24

  preprintOpen accessSenior author

  Organismal body weight correlates with morphology, life history, physiology, and behavior, making it perhaps the most telling single indicator of an organism's evolutionary and ecological profile. Island populations provide an exceptional opportunity to study body weight evolution. In accord with the "island rule," insular small-bodied vertebrates often evolve larger sizes, whereas insular large-bodied vertebrates evolve smaller sizes. To understand how island populations evolve extreme sizes, we adopted a developmental perspective and compared a suite of traits with established connections to body size in the world's largest wild house mice from Gough Island and mice from a smaller-bodied mainland strain. We pinpoint 24-hour periods during the third and fifth week of age in which Gough mice gain exceptionally more weight than mainland mice. We show that Gough mice accumulate more visceral fat beginning early in postnatal development. During a burst of weight gain, Gough mice shift toward carbohydrates and away from fat as fuel, despite being more active than and consuming equivalent amounts of food as mainland mice. Our findings showcase the value of developmental phenotypic characterization for discovering how body weight evolves in the context of broader patterns of trait evolution.

  PublisherOA PDFDOI

• Elevated mutation near crossovers inhibits the evolution of recombination

  bioRxiv (Cold Spring Harbor Laboratory) · 2025-11-01

  preprintOpen access1st authorCorresponding

  Recombination diversifies offspring genomes and helps ensure chromosome segregation during meiosis. Mutation rates are elevated near crossovers due to the induction of double-strand breaks and their imperfect repair, a byproduct of recombination typically ignored by theory designed to explain its evolution. To examine the evolutionary role of mutagenic recombination, we analyze a population genetic model in which a modifier locus controls both the rate of recombination between two loci experiencing viability selection and the rate of mutation at those loci. Analytical and numerical results demonstrate that the advantage of recombination conferred by its capacity to remove epistatic, deleterious variants is overcome by the selective cost of even small increases in the mutation rate. By incorporating the mutagenic effects of recombination, our analysis extends a rich body of theory to acknowledge a molecular feature inherent in the formation of crossovers. Our findings suggest that higher recombination rate evolves by altering steps in the crossover pathway that are less likely to inflict mutational damage.

  PublisherDOI

• Population Genomics of Giant Mice from the Faroe Islands: Hybridization, Colonization, and a Novel Challenge to Identifying Genomic Targets of Selection

  Genome Biology and Evolution · 2025-07-12

  articleOpen access1st authorCorresponding

  Populations that colonize islands provide unique insights into demography, adaptation, and the spread of invasive species. House mice on the Faroe Islands evolved exceptionally large bodies after colonization, generating longstanding interest from biologists. To reconstruct the evolutionary history of these mice, we sequenced genomes of population samples from three Faroe Islands (Sandoy, Nólsoy, and Mykines) and Norway as a mainland comparison. Mice from the Faroe Islands are hybrids between the subspecies Mus musculus domesticus and M. m. musculus, with ancestry alternating along the genome. Analyses based on the site frequency spectrum of single nucleotide polymorphisms and the ancestral recombination graph (ARG) indicate that mice arrived on the Faroe Islands on a timescale consistent with transport by Norwegian Vikings, with colonization of Sandoy likely preceding colonization of Nólsoy. Substantial reductions in nucleotide diversity and effective population size associated with colonization suggest that mice on the Faroe Islands evolved large body size during periods of heightened genetic drift. Genomic scans for positive selection uncover windows with unusual site frequency spectra, but this pattern is mostly generated by clusters of singletons in individual mice. Three genomic regions show evidence for selection on islands based on the ARG, including variants located in transcription factor binding sites. Our findings reveal a dynamic evolutionary history for the enigmatic mice from Faroe Island and emphasize the challenges that accompany population genomic inferences in island populations.

  PublisherOA PDFDOI

• Comparative Patterns of Variation on the X Chromosome and Autosomes: The Role of the Breeding Sex Ratio

  bioRxiv (Cold Spring Harbor Laboratory) · 2025-01-22 · 2 citations

  preprintOpen accessSenior author

  Abstract In many populations, unequal numbers of females and males reproduce each generation. This imbalance in the breeding sex ratio (BSR) shapes patterns of genetic variation on the sex chromosomes and the autosomes in distinct ways. Despite recognition of this phenomenon, effects of the BSR on some aspects of variation remain unclear, especially for populations with non-equilibrium demographic histories. To address this gap in the field, we used coalescent simulations to examine relative patterns of variation at X-linked loci and autosomal loci in populations spanning the range of BSR with historical changes in population size. Shifts in BSR away from 1:1 reduce nucleotide diversity and the number of unique haplotypes and increase linkage disequilibrium and the frequency of the most common haplotype, with contrasting effects on X-linked loci and autosomal loci. Strong population bottlenecks transform relationships between the BSR, the site frequency spectrum, and linkage disequilibrium while relationships between the BSR, nucleotide diversity, and haplotype characteristics are broadly conserved. Our findings indicate that evolutionary interpretations of variation on the X chromosome should consider the combined effects of the BSR and demographic history. The genomic signatures we report could be used to reconstruct these fundamental population parameters from genomic data in natural populations. Significance Statement The breeding sex ratio is a fundamental evolutionary parameter, but genomic analyses routinely assume it is 1:1. Our research characterizes the relationships between the breeding sex ratio and multiple facets of genomic variation and shows how these relationships change in the context of dynamic demographic histories. In doing so, we provide increasingly realistic expectations for patterns of X-linked and autosomal variation in population genomic datasets collected from natural populations.

  PublisherDOI

• The Breeding Sex Ratio Interacts With Demographic History to Shape Comparative Patterns of Variation on the X Chromosome and the Autosomes

  Genome Biology and Evolution · 2025-02-28

  articleOpen accessSenior author

  In many populations, unequal numbers of females and males reproduce each generation. This imbalance in the breeding sex ratio shapes patterns of genetic variation on the sex chromosomes and the autosomes in distinct ways. Despite recognition of this phenomenon, effects of the breeding sex ratio on some aspects of variation remain unclear, especially for populations with nonequilibrium demographic histories. To address this gap in the field, we used coalescent simulations to examine relative patterns of variation at X-linked loci and autosomal loci in populations spanning the range of breeding sex ratio with historical changes in population size. Shifts in breeding sex ratio away from 1:1 reduce nucleotide diversity and the number of unique haplotypes and increase linkage disequilibrium and the frequency of the most common haplotype, with contrasting effects on X-linked loci and autosomal loci. Strong population bottlenecks transform relationships among the breeding sex ratio, the site frequency spectrum, and linkage disequilibrium, while relationships among the breeding sex ratio, nucleotide diversity, and haplotype characteristics are broadly conserved. Our findings indicate that evolutionary interpretations of variation on the X chromosome should consider the combined effects of the breeding sex ratio and demographic history. The genomic signatures we report could be used to reconstruct these fundamental population parameters from genomic data in natural populations.

  PublisherOA PDFDOI

• The genomic imprint of chromosomal inversions and demographic history in island populations of deer mice

  bioRxiv (Cold Spring Harbor Laboratory) · 2025-04-22

  preprintOpen accessSenior author

  ABSTRACT Populations that colonize islands experience novel selective pressures, fluctuations in size, and changes to their connectivity. Owing to their unique geographic setting, islands can function as natural laboratories in which to examine the interactions between demographic history and natural selection replicated across isolated populations. We used whole genome sequences of wild-caught deer mice ( Peromyscus maniculatus ) from two islands (Saturna and Pender) and one mainland location (Maple Ridge) in the Gulf Islands region of coastal British Columbia to investigate two primary determinants of genome-wide diversity: chromosomal inversions and non-equilibrium demographic history. We found that segregating inversions produce characteristic, large-scale distortions in allele frequencies and linkage disequilibrium that make it possible to identify and characterize them from short-read sequence data. Patterns of variation within and between karyotypes indicate that six inversion polymorphisms have been maintained by a shared history of balancing selection in both island and mainland populations. Whereas the estimated timing of contemporary population splits is consistent with the isolation of island populations from each other following the Last Glacial Maximum, ancestral island and mainland lineages are inferred to have diverged much earlier. These aspects of demographic history suggest that shared inversions existed long ago in a common ancestor or spread via limited gene flow between ancestral island and mainland lineages. Our results raise the possibility that inversions segregating among Gulf Islands populations are on similar evolutionary trajectories, providing a contrast to previous findings in mainland P. maniculatus and contributing to the emerging portrait of inversion evolution in this species.

  PublisherOA PDFDOI

• Population history across timescales in an urban archipelago

  bioRxiv (Cold Spring Harbor Laboratory) · 2025-01-25 · 1 citations

  preprintOpen accessSenior author

  Abstract Contemporary patterns of genetic variation reflect the cumulative history of a population. Population splitting, migration, and changes in population size leave genomic signals that enable their characterization. Existing methods aimed at reconstructing these features of demographic history are often restricted in their temporal resolution, leaving gaps about how basic evolutionary parameters change over time. To illustrate the prospects for extracting insights about dynamic population histories, we turn to a system that has undergone dramatic changes on both geological and contemporary timescales – an urbanized, near-shore archipelago. Using whole genome sequences, we employed both common and novel summaries of variation to infer the demographic history of three populations of endemic white-footed mice ( Peromyscus leucopus ) in Massachusetts’ Boston Harbor. We find informative contrasts among the inferences drawn from these distinct patterns of diversity. While demographic models that fit the joint site frequency spectrum (jSFS) coincide with the known geological history of the Boston Harbor, patterns of linkage disequilibrium reveal collapses in population size on contemporary timescales that are not recovered by our candidate models. Historical migration between populations is also absent from best-fitting models for the jSFS, but rare variants show unusual clustering along the genome within individual mice, a pattern that is reproduced by simulations of recent migration. Together, our findings indicate that these urban archipelago populations have been shaped by both ancient geological processes and recent human influence. More broadly, our study demonstrates that the temporal resolution of demographic history can be extended by examining multiple facets of genomic variation. Significance Statement Detailed information about a population’s history can be obtained by studying patterns of genomic variation, but these insights can be limited in their temporal scope. Our investigation of white-footed mice in the urban Boston Harbor archipelago demonstrates how combining multiple summaries of genomic variation enables a more complete reconstruction of population history over both the recent and distant past.

  PublisherDOI

• Phenotypic and developmental dissection of an instance of the island rule

  Evolution · 2025-03-12

  articleOpen accessSenior author

  Organismal body weight correlates with morphology, life history, physiology, and behavior, making it perhaps the most telling single indicator of an organism's evolutionary and ecological profile. Island populations provide an exceptional opportunity to study body weight evolution. In accord with the "island rule," insular small-bodied vertebrates often evolve larger sizes, whereas insular large-bodied vertebrates evolve smaller sizes. To understand how island populations evolve to extreme sizes, we adopted a developmental perspective and compared a suite of traits with established connections to body size in the world's largest wild house mice from Gough Island and mice from a smaller-bodied mainland strain. We pinpoint 24-h periods during the third and fifth week of age in which Gough mice gain exceptionally more weight than mainland mice. We show that Gough mice accumulate more visceral fat beginning early in postnatal development. During a burst of weight gain, Gough mice shift toward carbohydrates and away from fat as fuel, despite being more active than and consuming equivalent amounts of food as mainland mice. Our findings showcase the value of developmental phenotypic characterization for discovering how body weight evolves in the context of broader patterns of trait evolution.

  PublisherOA PDFDOI

Recent grants

• Identifying Genetic Determinants of Speciation from Genomic Maps of Ancestry in Hybrid Mice

  NSF · $900k · 2014–2020

• DISSERTATION IMPROVEMENT GRANT: Identifying Reproductive Isolation Loci from Clines in Hybrid Zones

  NSF · $20k · 2014–2016

• DISSERTATION RESEARCH: The Genetic Basis of Recombination Rate Variation in House Mice

  NSF · $15k · 2009–2011

• Evolution of Phenotypic Extremes and Mechanisms Governing Inheritance

  NIH · $4.9M · 2021–2026

• The Genetics and Evolution of Hybrid Male Sterility in House Mice

  NSF · $685k · 2009–2012

Frequent coauthors

• Michael A. White

  16 shared

• Christopher J. Vinyard

  Ohio University

  15 shared

• Richard J. Wang

  Indiana University Bloomington

  12 shared

• Mark J. Nolte

  12 shared

• Beth L. Dumont

  12 shared

• Jered A. Stratton

  11 shared

• Ryan J. Haasl

  University of Wisconsin–Platteville

  11 shared

• Michael W. Nachman

  Integra (United States)

  8 shared

Labs

• Laboratory of GeneticsPI

Education

• Ph.D., Genetics

  University of California, Berkeley

  2005

• B.S., Genetics

  University of California, Davis

  2000