Predictive Modeling of Childhood Disease Severity at a Telemedicine Service: Evidence from the Improving Nighttime Access to Care and Treatment Studies in Haiti

The Journal of Pediatrics · 2026-02-17

OBJECTIVE: To develop models within pediatric telemedicine that identify potentially "sick" cases for additional safety checks and integrate those models into electronic clinical decision support tools. STUDY DESIGN: We conducted a secondary analysis of paired virtual and in-person examinations across 3 consecutive implementation studies conducted at a telemedicine and medication delivery service in Haiti. Artificial intelligence/machine learning (XGBoost) was applied to derive models focused on identifying "sick" patients (moderate or severe) and those requiring escalation. Given the limited sample size, we used an ensemble method based on gradient boosted decision trees. The area under the receiver operating characteristic curve (AUC) was the primary outcome measure. RESULTS: A total of 683 paired records were available for this secondary analysis from 2225 participants enrolled. The median age was 15 months and 47% were female. For prediction of a "sick" child, we found an AUC of 0.82 (95% confidence interval [CI]: 0.78-0.86) after 5-fold cross-validation; calibration slope and intercept were 1.09 (95% CI: 0.91-1.26) and 0.16 (95% CI: 0.03-0.35), respectively. For prediction of escalation, we found an AUC of 0.77 (95% CI: 0.73-0.81); calibration slope and intercept were 0.81 (95% CI: 0.66-0.96) and 0.08 (95% CI: 0.11-0.26), respectively. CONCLUSIONS: These methods and findings offer an innovative and important proof-of-concept for how to aid clinical decision-making within pediatric telemedicine environments. The models require external validation prior to electronic clinical decision support integration and deployment. Once validated, the models will provide a critical safety check for experienced providers and digitally convey expertise to new providers.

Evaluation of a Scalable Design for a Pediatric Telemedicine and Medication Delivery Service: A Prospective Cohort Study in Haiti

American Journal of Tropical Medicine and Hygiene · 2025-06-24 · 2 citations

Early access to health care is essential to avert morbidity and mortality. A telemedicine and medication delivery service (TMDS) is an innovative solution to address this need; however, pathways to scalability are unclear. We sought to evaluate a scalable pediatric TMDS. A TMDS in Haiti was configured for scalability by triaging severe cases to hospital-level care, nonsevere cases with higher clinical uncertainty to in-person examinations at households, and nonsevere cases with low clinical uncertainty to medication delivery alone. This design was evaluated in a prospective cohort study conducted among pediatric patients 10 years old or younger. Clinical and operational metrics were compared with a formative reference study in which all nonsevere patients received an in-person examination. The primary outcomes were rates of clinical improvement/recovery and in-person care seeking at 10 days. In total, 1,043 cases were enrolled in the scalable TMDS mode, and 19% (190) of nonsevere cases received an in-person examination; 382 cases were enrolled in the reference study, and 94% (338) of nonsevere cases received an in-person examination. At 10 days, rates of improvement were similar for the scalable and reference modes. Rates of participants who sought follow-up care were 15% in the scalable mode and 24% in the reference mode. In the context of a 5-fold reduction of in-person examinations, participants in the scalable mode had noninferior rates of improvement at 10 days. These findings highlight an innovative and now scalable solution to improve early access to health care without compromising safety.

Development of an AI-enabled predictive model to identify the ‘sick child’ at a pediatric telemedicine and medication delivery service in Haiti

medRxiv · 2025-06-28

ABSTRACT Background One of the most difficult challenges in pediatric telemedicine is to accurately discriminate between the ‘sick’ and ‘not sick’ child, especially in resource-limited settings. Models that flag potentially ‘sick’ cases for additional safety checks represent an opportunity for telemedicine to reach its potential. However, there are critical knowledge gaps on how to develop such models and integrate them into electronic clinical decision support (eCDS) tools. Methods To address this challenge, we developed a study design that utilized data from paired virtual and in-person exams at a telemedicine and medication delivery service (TMDS) in Haiti. Providers were allowed to mark respondent data as potentially unreliable. Artificial intelligence /machine learning (XGBoost) was applied to analyze paired data from participants across three consecutive implementation studies. Model derivation focused on identifying ‘sick’ patients (not-mild) and those requiring escalation. An ensemble method, based on gradient boosted decision-trees, was used given the limited sample size. The area under the receiver operating characteristic curve (AUC) was the primary outcome measure. Results A total of 683 paired records were available for this secondary analysis from 2225 participants enrolled. The median age was 15 months and 47% were female. For prediction of a ‘sick’ child, we found an AUC of 0.82 (95% CI 0.78-0.86) after 5-fold cross validation; calibration slope and intercept were 1.31 (95%CI:1.09-1.53) and 0.04 (95%CI:-0.14-0.23), respectively. For prediction of escalation, we found an AUC of 0.78 (95%CI:0.74-0.81); calibration slope and intercept were 0.63 (95%CI:0.52-0.74) and 0.05 (95%CI:0.52-0.74), respectively. Accounting for data marked as potentially unreliable had mixed effects. Interpretation These methods and findings offer an innovative and important proof-of-concept to improve pediatric telemedicine. The models require external validation prior to eCDS integration and deployment. Once validated, the models are designed to provide a critical safety check for experienced providers and digitally convey expertise to new providers. Funding National Institutes of Health (USA) grants to EJN (R21TW012332; DP5OD019893), internal funding at UF (Children’s Miracle Network), and private donations. RESEARCH IN CONTEXT Evidence before this study We conducted two Pubmed searches for reports published in all languages. The first search terms were (telemedicine) AND (artificial intelligence OR machine learning) AND (pediatrics OR paediatrics). The primary search criteria identified 153 publications and reviews were excluded leaving 101 papers all published after 1998. Enumerated results by named subspecialty were neurology (n=2), ophthalmology (n=13), otology (n=3), endocrinology (n=11), cardiology (n=6), pulmonology (n=3), gastroenterology(n=1), dermatology (n=1) and surgery (n=22). Ten of the publications focused on global health or low-middle income countries (LMIC) populations. The second search terms were ((Telemedicine) AND (delivery OR paramedicine) AND (pediatrics OR paediatrics)) AND (global health OR LMIC) which generated 99 publications and 76 papers remained after reviews were excluded, all published after 2015. After manual evaluation of the results from both searches, no publications were identified that fully met the scope of this paper. Examples of telemedicine research for concordance with paired exams does exist 1,2 . Added value of this study To the best of our knowledge, this is the first study that investigated pediatric disease severity prediction using virtual and in-person exams in the context of telemedicine -- for either high or low resourced settings. Therefore, the added value of this study is an innovative and important proof-of-concept to improve telemedicine research and clinical practice beyond the scope of global health. Implications of all the available evidence Inside the field of global health, there is a need to develop evidence-based approaches to extend care early to pediatric patients who may be isolated by poverty, geography or unrest. This must be done safely and this paper offers an approach to develop and incorporate disease severity prediction models into eCDS tools. In addition, these tools may serve as a welcomed safety check for experienced providers and a method to digitally convey expertise to new providers as these services scale.

Pathways to optimize a pediatric telemedicine and medication delivery service: a multi-level qualitative study in Haiti

BMC Health Services Research · 2025-10-21 · 1 citations

While telemedicine has become an established component of healthcare delivery globally, challenges to scaling emerging initiatives persist across multiple levels. Over the last 5 years, our team has developed a pediatric telemedicine and medication delivery service (TMDS) in Haiti that integrates clinical guidance with rapid access to medications. Building on successful proof-of-concept studies, we are now well positioned to characterize both general challenges to telemedicine scale-up and those unique to the TMDS model. In this descriptive qualitative study, we conducted focus group discussions and administered written questionaries to TMDS staff, including physicians, nurses, and medication delivery drivers. These qualitative survey instruments were designed to capture staff experiences and suggestions. Using framework matrix analysis, we identified key challenges and opportunities associated with the TMDS model. Fourteen nurses, 13 drivers, and 3 on-call physicians participated in the study. Areas for improvement related to obtaining quality information from virtual exams, the reliability of technology and communication infrastructure, conditions necessary for effective in-person exams, the limited scope of the TMDS workflow and clinical resources, and uncertainty surrounding long-term sustainability. These insights informed the development of targeted potential solutions categorized into three domains: cognitive, physical and mission-oriented. The findings will guide our internal scale-up strategy and may offer guidance to similar telemedicine initiatives.

Pathways to optimize a pediatric telemedicine and medication delivery service: A multi-level qualitative study in Haiti

medRxiv · 2025-03-27 · 1 citations

Background: While telemedicine has become an established component of healthcare delivery globally, challenges to scaling emerging initiatives persist across multiple levels. Over the last 5 years, our team has developed a pediatric telemedicine and medication delivery service (TMDS) in Haiti that integrates clinical guidance with rapid access to medications. Building on successful proof-of-concept studies, we are now well positioned to characterize both general challenges to telemedicine scale-up and those unique to the TMDS model. Methods: In this study, we conducted focus group discussions and administered written questionaries to TMDS staff, including physicians, nurses, and medication delivery drivers. Using framework matrix analysis we identified key challenges and opportunities associated with the TMDS model. Results: Areas for improvement related to obtaining quality information from virtual exams, the reliability of technology and communication infrastructure, conditions necessary for effective in-person exams, the limited scope of the TMDS workflow and clinical resources, and uncertainty surrounding long-term sustainability. These insights informed the development of targeted action items categorized into three domains: conceptual, physical and mission-oriented. Conclusion: The findings will guide our internal scale-up strategy and may offer guidance to similar telemedicine initiatives.

Diarrheal Disease Attributed to Shigella spp. and Enteroinvasive Escherichia coli among Children at Households in Haiti: A Case–Control Study

American Journal of Tropical Medicine and Hygiene · 2025-11-04

Infections from Shigella spp./enteroinvasive Escherichia coli (EIEC) are considered leading causes of diarrheal disease globally. However, there is a notable paucity of studies from Caribbean nations to guide regional public health interventions. A case-control study was conducted as part of a cross-sectional healthcare study in Haiti. Households were identified using a geospatially randomized method, and families with children under 5 years of age were consented and enrolled. Rectal swabs from child participants were tested for Shigella spp./EIEC via real-time polymerase chain reaction testing using the invasion plasmid antigen H gene target. Two case definitions were used: "diarrheal symptom" (DS) cases were defined as those reporting DSs ≤7 days ago; "acute diarrhea" (AD) cases were defined as those who also presented with ≥3 loose stools in the past 24 hours and onset <7 days ago. A total of 568 households were enrolled, and samples from 732 children were analyzed. The rates of Shigella spp./EIEC detection were 11% (22/193) and 6% (33/539) among DS cases and controls, respectively, and 19% (8/43) and 7% (47/689) among AD cases and controls, respectively. Shigella spp./EIEC was attributed to DS in 6% (95% CI: 0.4%-11%) of cases and AD in 13% (95% CI: 0%-25%) of cases. The adjusted odds of having DS increased by 84% (adjusted odds ratio [aOR] = 1.84; 95% CI: 1.02-3.27) and AD increased by 183% (aOR = 2.83; 95% CI: 1.14-6.36) when Shigella spp./EIEC was detected. The rates of bloody diarrhea were minimal (<1%; 6/732). In the present case-control study, the detection of Shigella spp./EIEC was common and attributed to symptomatic disease. These results align with previous global health studies. Shigella spp./EIEC represent an important public health target for intervention once the security situation improves in Haiti.

Diarrheal disease attributed to <i>Shigella</i> spp. and Enteroinvasive <i>Escherichia coli</i> among children at households in rural Haïti: A case-control study

medRxiv · 2025-01-31 · 1 citations

Infections from Shigella spp. and Enteroinvasive Escherichia coli (EIEC) are considered leading causes of symptomatic diarrheal disease, globally. However, there is a paucity of case-control studies from Caribbean nations to guide regional public health priorities and interventions. A case-control study was conducted within a larger cross-sectional healthcare study in Haïti. Participant households were identified using a geospatially randomized method; families with children under 5 years were consented and enrolled. Rectal swabs from child participants were tested for Shigella spp./EIEC by qPCR using the ipaH target. Two case-definitions were used: 'diarrheal symptom' (DS) cases were defined as those reporting diarrheal symptoms ≤7 days ago; 'acute diarrhea' (AD) cases were defined as those reporting diarrheal symptoms ≤7 days ago with ≥3 loose stools in the past 24 hours and onset &lt;7 days ago. Of 868 households screened, 568 were enrolled with 794 participating children; samples from 732 children were analyzed. Rates of Shigella spp./EIEC. detection among DS cases and controls were 11.4% (22/193) and 6.1% (33/539), respectively. Rates of detection among AD cases and controls were 18.6% (8/43) and 6.8% (47/689), respectively. The adjusted odds of having DS increased by 84% (aOR=1.84; 95%CI 1.02 to 3.27) and having AD increased by 183% (aOR=2.83; 95%CI 1.14 to 6.36) when Shigella spp./EIEC was detected. The attributable fractions for DS and AD were 5.62% (95%CI 0.44% to 10.9%) and 12.6% (95%CI 0% to 25.3%), respectively. Rates of bloody diarrhea (dysentery) were minimal (&lt;1%, 6/732). Within this case-control study in Haïti, Shigella spp./EIEC detection was common and attributed to symptomatic disease. These results align with prior global health studies. Shigella spp./EIEC represent an important public health target for intervention once the security situation in Haïti stabilizes.

Dengue and Other Arbovirus Infections among Schoolchildren, Haiti, 2021

Emerging infectious diseases · 2025-01-17 · 2 citations

In 2021, we screened 91 children in Haiti with acute undifferentiated febrile illness for arbovirus infections. We identified a major outbreak of dengue virus type 2, with 67% of the children testing positive. Two others were positive for chikungunya East/Central/South African IIa subclade, and 2 were positive for Zika virus.

Development and evaluation of a clinical guideline for a paediatric telemedicine service in a low-resource setting

BMJ Paediatrics Open · 2024-01-01 · 7 citations

OBJECTIVE: To develop and evaluate a guideline for a paediatric telemedicine and medication delivery service (TMDS). METHODS: . The guideline was deployed at a TMDS in Haiti and evaluated through a prospective cohort study; children ≤10 years were enrolled. For non-severe cases, paired virtual and in-person examinations were conducted at the call centre and household; severe cases were referred to the hospital. The performance of virtual examination components were evaluated by comparison with the paired in-person examination findings (reference). RESULTS: A total of 391 cases were enrolled. Among the 320 cases with paired examinations, no general WHO danger signs were identified during in-person examinations; 5 cases (2%) required hospital referral due to problem-specific danger signs or other reasons for escalation. Cohen's kappa for the virtual designation of mild cases was 0.78 (95% CI: 0.69 to 0.87). The sensitivity and specificity of a virtually reported fever were 91% (95% CI: 87% to 96%) and 69% (95% CI: 62% to 76%), respectively; the sensitivity and specificity of virtually reported 'fast breathing' were 47% (95% CI: 21% to 72%) and 89% (95% CI: 85% to 94%), respectively. Kappa for 'no' and 'some' dehydration indicated moderate congruence between virtual and in-person examinations (0.69; 95% CI: 0.41 to 0.98). At 10 days, 273 (95%) of the 287 cases reached by phone were better/recovered. CONCLUSION: Critical components of the virtual examination (triage, danger signs and dehydration assessment) performed well despite varied performance among the problem-specific components. The study and associated resources represents formative steps towards an evidence-based paediatric telemedicine guideline built on WHO clinical principles. In-person examinations for select cases were important to address limitations with virtual examinations and identify cases for escalation. TRIAL REGISTRATION NUMBER: NCT03943654.

Results of Integrated Transmission Assessment Surveys for Lymphatic Filariasis and Malaria in Haiti, 2017–2022

American Journal of Tropical Medicine and Hygiene · 2024-06-25 · 3 citations

Haiti is endemic for lymphatic filariasis (LF) and malaria, two mosquito-transmitted parasitic diseases targeted for elimination. The World Health Organization recommends a transmission assessment survey (TAS-1) to determine if LF prevalence is significantly beneath putative transmission thresholds (<2% antigen prevalence in Haiti, where Culex is the primary vector for Wuchereria bancrofti) to stop mass drug administration (MDA). Repeated TASs (TAS-2 and TAS-3) are recommended at 2-3-year intervals during post-treatment surveillance. From 2017 to 2022, The Carter Center assisted the Haitian Ministry of Public Health and Population in conducting 15 TASs in 11 evaluation units (EUs) encompassing 54 of the country's 146 districts. Children 6-7 years old were assessed for circulating filarial antigen (CFA) by Filariasis Test Strip: n = 5,239 in TAS-1; n = 11,866 in TAS-2; and n = 1,842 in TAS-3, of whom eight (0.15%), 20 (0.17%), and eight (0.43%) tested positive, respectively. The number of positive results in children was less than the threshold in each EU. When available, participants (n = 16,663) were also tested for malaria by rapid diagnostic test, with 31 (0.19%) children testing positive for Plasmodium falciparum. Integrated TASs provided an efficient means to collect epidemiological data for LF and malaria in Haiti. Results indicated thresholds for stopping and maintaining the halt of MDA for LF have been achieved in all EUs, with the halt of MDA for 571,358 people in four districts and the first TAS-3 surveys conducted in Haiti. Investigations are needed to assess the potential of ongoing LF transmission, especially in areas where CFA-positive samples were detected in TAS-3.