Calibration of transmission-dynamic infectious disease models: a scoping review and reporting framework

medRxiv · 2025-03-10

Abstract Objective/Background Transmission-dynamic models are commonly used to study infectious disease epidemiology. Calibration involves identifying model parameter values that align model outputs with observed data or other evidence. Inaccurate calibration and inconsistent reporting produce inference errors and limit reproducibility, compromising confidence in modeled results. No standardized framework exists for reporting on calibration of infectious disease models, and an understanding of current calibration approaches is lacking. Methods We developed a 15-item framework for reporting calibration practices and applied it in a scoping review to assess calibration approaches and evaluate reporting comprehensiveness in transmission-dynamic models of tuberculosis, HIV and malaria published between January 1, 2018, and January 16, 2024. We searched relevant databases and websites to identify eligible publications, including peer-reviewed studies where these models were calibrated to empirical data or published estimates. Results We identified 411 eligible studies encompassing 419 models, with 74% (n=309) being compartmental models and 20% (n=82) individual-based models (IBMs). The predominant analytical purpose was to evaluate interventions (71% of models, n=298). Parameters were calibrated mainly because they were unknown or ambiguous (40%, n=168), or because determining their value was relevant to the scientific question beyond being necessary to run the model (20%, n=85). The choice of calibration method was significantly associated with model structure (p-value<0.001) and stochasticity (p-value=0.006), with approximate Bayesian computation more frequently used with IBMs and Markov-Chain Monte Carlo with compartmental models. Regarding reporting comprehensiveness, all 15 framework items in the framework were reported in 4% (n=18) of models; 11-14 items in 66% (n=277), and 10 or fewer items in 28% (n= 124). Implementation code was the least reported, available in only 20% (n=82) of models. Conclusions Reporting on calibration is heterogeneous in recent infectious disease modeling literature. Our proposed framework for reporting of calibration approaches could support improved reproducibility and credibility of modeled analyses. Author Summary Calibration, the identification of parameter values so that model outcomes are consistent with observed data or other evidence, is often employed in the process of obtaining model results to inform health decision making. Despite its importance, there has not been a standardized framework for reporting how calibration is conducted in infectious disease modeling studies. This has led to inconsistent reporting practices and challenges in reproducing model results, potentially compromising confidence in their validity. We developed a calibration reporting framework, based on best practices found in the literature and informed by our expertise in conducting calibration. To assess calibration practices and their reporting, we applied our framework in a scoping review of 419 infectious disease transmission models of HIV, TB and malaria published between 2018 and 2024. Most models reviewed were compartmental (74%) or individual-based (20%), and the choice of calibration methods was associated with model structure and stochasticity. Calibration was conducted predominantly in the context of models aimed at evaluating the impact of disease control interventions, highlighting the role of calibration in decision making. Parameters were calibrated mainly because they were unknown or ambiguous, or because reporting their value was relevant to the scientific question beyond just being necessary to run the model. The comprehensiveness of calibration reporting varied across models, with most models omitting 1 to 5 items in the framework. Accessible implementation code was the most underreported, with only 20% of models including it. Our proposed framework could serve as a tool to standardize calibration reporting, thereby enhancing the transparency and reproducibility of calibration processes in transmission-dynamic models.

Identifying malaria elimination strategies in the presence of human movement in Bangladesh

Communications Medicine · 2025-11-07

BACKGROUND: Malaria transmission in the Chittagong Hill Tracts (CHT) districts in Bangladesh is characterized by considerable heterogeneity in incidence and the frequent mixing and importation of parasites across districts. Thus, elimination efforts must account for human mobility between endemic and non-endemic locations, and the relative importance of local transmission and parasite importation domestically. METHODS: We construct a metapopulation malaria model, parameterized by human mobility data and fit to epidemiological data, to guide elimination efforts in the region. RESULTS: We find substantial heterogeneity in the transmission intensity across the CHT, with the estimated basic reproduction number varying greatly across places with similar levels of observed incidence. When vector control interventions are applied locally, the greatest impact in reducing overall incidence are in places with both high transmission intensity and high connectivity with more populated districts in the western part of the CHT. CONCLUSIONS: Local elimination in several areas with low or intermediate incidence has a moderate impact in reducing overall incidence, indicating that only focusing on high incidence areas is not sufficient for malaria elimination. More generally, our modeling framework can be used to prioritize resource allocation and identify the conditions necessary for malaria elimination in the region.

Responding to rising heat in workplaces and homes of low income workers

BMJ · 2025-11-04 · 3 citations

Adult-capture assays as a tool to measure insecticide resistance in Anopheles malaria vectors: a modeling comparison with larval-capture assays

Frontiers in Malaria · 2025-03-24

Systematic, long-term, and spatially representative monitoring of insecticide resistance in mosquito populations is urgently needed to quantify its impact on malaria transmission, and to combat failing interventions when resistance emerges. Resistance assays on wild-caught adult mosquitoes (known as adult-capture) offer an alternative to the current protocols, which recommend larval capture. Adult-capture assays can be done in a shorter time frame, in more locations, and in the absence of an insectary. However, unlike insectary-raised mosquitoes, a group of adults captured in the wild represents different ages and may have previous exposure to insecticides. Since age and prior exposure are critically important in determining the likelihood of death during the assay, taking these factors into account is important for assessing the relative utility of the assay. Currently such quantitative assessments are lacking. We developed a discrete-time deterministic model to simulate the mosquito life cycle, including insecticide exposure due to insecticide-treated bed nets. We incorporated non-lethal effects of insecticide exposure demonstrated in laboratory experiments and the impact of multiple exposure to insecticides on mosquito death rates during the assay. We then sampled from this population using both larval-captured and adult-captured mosquito collection and simulated insecticide resistance assays. To quantify possible biases in adult-capture assays, we compared the results of these assays to the true resistance allele frequency in the population. In simulated samples of 100 test mosquitoes, reflecting WHO-recommended sample sizes, we found that adult-capture samples had a 94% positive predictive value (PPV) for resistance at the WHO’s 10% resistance cutoff, and a 97% negative predictive value (NPV), compared to 98% PPV and 19% NPV for larval-captured samples. Bias in the adult-capture assays was primarily dependent on the level of insecticide resistance rather than coverage of bed nets or exposure heterogeneity. Using adult-captured mosquitoes for resistance assays may have advantages over larval-capture collection in many settings, and in our model does not appear to be significantly less accurate than larval-capture, especially when used to categorize resistance under the binary WHO criteria. These results suggest that adult-captured assays could be deployed for resistance monitoring programs at a more widespread scale.

Marked heterogeneity in malaria infection rate in a Malian longitudinal cohort

Nature Communications · 2025-07-15 · 2 citations

Variation in malaria infection risk, a product of disease exposure and immunity, is poorly understood. We genotypically profiled over 13,000 blood samples from a six-year longitudinal cohort in Mali to characterize malaria infection dynamics with detail. We generated Plasmodium falciparum amplicon sequencing data from 464 participants (aged 3 months – 25 years) across the six-month 2011 transmission season and profiled a subset of 120 participants across the subsequent five annual transmission seasons. We measured infection rate as the molecular force of infection (molFOI, number of genetically distinct parasites acquired over time). We found that molFOI varied extensively among individuals (0–55 in 2011) but was independent of age and consistent within individuals over multiple seasons. Reported bednet usage was nearly universal. The HbS allele was associated with lower molFOI, and functional antibody signatures for the CSP C-term and RH5 antigens were correlated with low molFOI participants, identifying candidate immune correlates of protection. The large inter-individual variability in molFOI and consistency of intra-individual infection rate over time exhibits much greater dynamic range than malaria case incidence, and is most likely due to heterogeneous exposure to infectious mosquito bites. This and other factors contributing to variable infection risk should be considered in future clinical trials and implementation of malaria interventions. Malaria infection risk is complex to quantify, partly because polyclonal as well as asymptomatic infections are common. Here, the authors use parasite genotyping data from a prospective cohort study in Mali to estimate infection incidence at the strain level and explore risk factors associated with heterogeneous infection rates.

Power Outages: Implications for California’s Medically Vulnerable Population

Disaster Medicine and Public Health Preparedness · 2025-01-01 · 3 citations

Natural disasters in the US have resulted in persistent morbidity and mortality due to disruptions in access to health care, loss of critical utilities, and displacement, disproportionately affecting disadvantaged communities.Among natural disasters, wildfires are frequently associated with unplanned power outages from infrastructure damage or planned outages aiming to de-energize powerlines in anticipation of wildfires.The planned outagespublic safety power shutoffs (PSPS)are deployed to protect life and property. 1Power outages particularly impact the medically vulnerable, precluding the use of electricity-dependent equipment like nebulizer machines and wound vacs, affecting refrigeration of medications, or simply shutting down fans and air conditioning. 2 The 2019 fire season was devastating, and its largest fire, the Kincade Fire, was caused by electrical transmission lines despite the extended, deliberate outages across the state.We calculated county-level cumulative exposure to power outages in 2019.More customers experienced PSPS events in 2019 alone than in 2020-2023 combined, and power outages caused extended disruptions in nearly every California county (Supplement A-C).More than half the events in October 2019 lasted more than 24 hours.Many Medicare beneficiaries were Durable Medical Equipment (DME) users in counties where disruptions lasted longer than 24 hours.Counties at the highest risk for significant power disruptions were home to many at-risk populations, including DME users and those with limited health care access (Supplement D, E).Since 2019, governmental, nongovernmental, and health care organizations have significantly invested in decreasing necessary de-energizing events and mitigating their negative impacts. 3 The California Department of Public Health and utility companies like PG&E have invested in microgridding efforts, portable battery programs, community resource centers, and increased public outreach and engagement, such as the Medical Baseline program for patients needing power to receive electricity services at lower rates.However, the results of a study of adults with access and functional needs in Mariposa County found significantly increased delays in medical care and health harms in those with more medical conditions or using more medical devices after the 2022 Oak Fire. 4 We continue to prepare for disasters in the US by mostly preparing for mass casualty events.Our analysis underscores the need to integrate in situ medical vulnerability (to power disruptions or other interruptions in health care access) into disaster planning and response.The US sees more frequent, cascading crises, such as the compound climate disaster resulting in severe power outages during Hurricane Beryl and extreme heat in Texas in the summer of 2024.Pre-emptive planning should require mapping and maintaining rosters of medically vulnerable populations that can be reached in anticipation of disasters in the context of local hazards.Effective alternative pathways to health care access must be integrated into disaster response planning.Local capacity to access and use such data in decision-making is limited and requires local or federal investment expansion.Integrating high-resolution socioeconomic and medical vulnerability data, especially regarding those critically dependent on electricity and other care services, in disaster planning and response is a prudent and essential step in caring more equitably for communities impacted by disasters.

Bias in mobility datasets drives divergence in modeled outbreak dynamics

Communications Medicine · 2025-01-07 · 3 citations

BACKGROUND: Digital data sources such as mobile phone call detail records (CDRs) are increasingly being used to estimate population mobility fluxes and to predict the spatiotemporal dynamics of infectious disease outbreaks. Differences in mobile phone operators' geographic coverage, however, may result in biased mobility estimates. METHODS: We leverage a unique dataset consisting of CDRs from three mobile phone operators in Bangladesh and digital trace data from Meta's Data for Good program to compare mobility patterns across these sources. We use a metapopulation model to compare the sources' effects on simulated outbreak trajectories, and compare results with a benchmark model with data from all three operators, representing around 100 million subscribers across the country. RESULTS: We show that mobility sources can vary significantly in their coverage of travel routes and geographic mobility patterns. Differences in projected outbreak dynamics are more pronounced at finer spatial scales, especially if the outbreak is seeded in smaller and/or geographically isolated regions. In some instances, a simple diffusion (gravity) model was better able to capture the timing and spatial spread of the outbreak compared to the sparser mobility sources. CONCLUSIONS: Our results highlight the potential biases in predicted outbreak dynamics from a metapopulation model parameterized with non-population representative data, and the limits to the generalizability of models built on these types of novel human behavioral data.

Calibration of transmission-dynamic infectious disease models: A scoping review and reporting framework

PLoS Computational Biology · 2025-11-04 · 1 citations

OBJECTIVE/BACKGROUND: Transmission-dynamic models are commonly used to study infectious disease epidemiology. Calibration involves identifying model parameter values that align model outputs with observed data or other evidence. Inaccurate calibration and inconsistent reporting produce inference errors and limit reproducibility, compromising confidence in the validity of modeled results. No standardized framework exists for reporting on calibration of infectious disease models, and an understanding of current calibration approaches is lacking. METHODS: We developed the Purpose-Inputs-Process-Outputs (PIPO) framework for reporting calibration practices and applied it in a scoping review to assess calibration approaches and evaluate reporting comprehensiveness in transmission-dynamic models of tuberculosis, HIV and malaria published between January 1, 2018, and January 16, 2024. We searched relevant databases and websites to identify eligible publications, including peer-reviewed studies where these models were calibrated to empirical data or published estimates. RESULTS: We identified 411 eligible studies encompassing 419 models, with 74% (n = 309) being compartmental models and 20% (n = 81) individual-based models (IBMs). The predominant analytical purpose was to evaluate interventions (71% of models, n = 298). Parameters were calibrated mainly because they were unknown or ambiguous (40%, n = 168), or because determining their value was relevant to the scientific question beyond being necessary to run the model (20%, n = 85). The choice of calibration method was significantly associated with model structure (p-value<0.001) and stochasticity (p-value = 0.006), with approximate Bayesian computation more frequently used with IBMs and Markov-Chain Monte Carlo with compartmental models. Regarding reporting comprehensiveness, all PIPO framework items were reported in 4% (n = 18) of models; 11-14 items in 66% (n = 277), and 10 or fewer items in 28% (n = 124). Implementation code was the least reported, available in only 20% (n = 82) of models. CONCLUSIONS: Reporting on calibration is heterogeneous in recent infectious disease modeling literature. Our proposed framework for reporting of calibration approaches could support improved reproducibility and credibility of modeled analyses.

Evidence-based Decision Making: Infectious Disease Modeling Training for Policymakers in East Africa

Annals of Global Health · 2024-03-22 · 7 citations

Background: Mathematical modeling of infectious diseases is an important decision-making tool for outbreak control. However, in Africa, limited expertise reduces the use and impact of these tools on policy. Therefore, there is a need to build capacity in Africa for the use of mathematical modeling to inform policy. Here we describe our experience implementing a mathematical modeling training program for public health professionals in East Africa. Methods: We used a deliverable-driven and learning-by-doing model to introduce trainees to the mathematical modeling of infectious diseases. The training comprised two two-week in-person sessions and a practicum where trainees received intensive mentorship. Trainees evaluated the content and structure of the course at the end of each week, and this feedback informed the strategy for subsequent weeks. Findings: Out of 875 applications from 38 countries, we selected ten trainees from three countries - Rwanda (6), Kenya (2), and Uganda (2) - with guidance from an advisory committee. Nine trainees were based at government institutions and one at an academic organization. Participants gained skills in developing models to answer questions of interest and critically appraising modeling studies. At the end of the training, trainees prepared policy briefs summarizing their modeling study findings. These were presented at a dissemination event to policymakers, researchers, and program managers. All trainees indicated they would recommend the course to colleagues and rated the quality of the training with a median score of 9/10. Conclusions: Mathematical modeling training programs for public health professionals in Africa can be an effective tool for research capacity building and policy support to mitigate infectious disease burden and forecast resources. Overall, the course was successful, owing to a combination of factors, including institutional support, trainees' commitment, intensive mentorship, a diverse trainee pool, and regular evaluations.

COVID‐19 vaccination strategies in Africa: A scoping review of the use of mathematical models to inform policy

Tropical Medicine & International Health · 2024-05-13 · 1 citations

OBJECTIVE: Mathematical models are vital tools to understand transmission dynamics and assess the impact of interventions to mitigate COVID-19. However, historically, their use in Africa has been limited. In this scoping review, we assess how mathematical models were used to study COVID-19 vaccination to potentially inform pandemic planning and response in Africa. METHODS: We searched six electronic databases: MEDLINE, Embase, Web of Science, Global Health, MathSciNet and Africa-Wide NiPAD, using keywords to identify articles focused on the use of mathematical modelling studies of COVID-19 vaccination in Africa that were published as of October 2022. We extracted the details on the country, author affiliation, characteristics of models, policy intent and heterogeneity factors. We assessed quality using 21-point scale criteria on model characteristics and content of the studies. RESULTS: The literature search yielded 462 articles, of which 32 were included based on the eligibility criteria. Nineteen (59%) studies had a first author affiliated with an African country. Of the 32 included studies, 30 (94%) were compartmental models. By country, most studies were about or included South Africa (n = 12, 37%), followed by Morocco (n = 6, 19%) and Ethiopia (n = 5, 16%). Most studies (n = 19, 59%) assessed the impact of increasing vaccination coverage on COVID-19 burden. Half (n = 16, 50%) had policy intent: prioritising or selecting interventions, pandemic planning and response, vaccine distribution and optimisation strategies and understanding transmission dynamics of COVID-19. Fourteen studies (44%) were of medium quality and eight (25%) were of high quality. CONCLUSIONS: While decision-makers could draw vital insights from the evidence generated from mathematical modelling to inform policy, we found that there was limited use of such models exploring vaccination impacts for COVID-19 in Africa. The disparity can be addressed by scaling up mathematical modelling training, increasing collaborative opportunities between modellers and policymakers, and increasing access to funding.