Retrieval-augmented generation for interpreting clinical laboratory regulations using large language models

Journal of Pathology Informatics · 2025-10-01 · 8 citations

Large language models (LLMs) have demonstrated strong performance on general knowledge tasks, but they have important limitations as standalone tools for question answering in specialized domains where accuracy and consistency are critical. Retrieval-augmented generation (RAG) is a strategy in which LLM outputs are grounded in dynamically retrieved source documents, offering advantages in accuracy, explainability, and maintainability. We developed and evaluated a custom RAG system called Raven, designed to answer laboratory regulatory questions using the part of the Code of Federal Regulations (CFR) pertaining to laboratory (42 CFR Part 493) as an authoritative source. Raven employed a vector search pipeline and a LLM to generate grounded responses via a chatbot-style interface. The system was tested using 103 synthetic laboratory regulatory questions, 88 of which were explicitly addressed in the CFR. Compared to answers generated manually by a board-certified pathologist, Raven's responses were judged to be totally complete and correct in 92.0% of those 88 cases, with little irrelevant content and a low potential for regulatory or medical error. Performance declined significantly on questions not addressed in the CFR, confirming the system's grounding in the source documents. Most suboptimal responses were attributable to faulty source document retrieval rather than model hallucination or misinterpretation. These findings demonstrate that a basic RAG system can produce useful, accurate, and verifiable answers to complex regulatory questions. With appropriate safeguards and with thoughtful integration into user workflows, tools like Raven may serve as valuable decision-support systems in laboratory medicine and other knowledge-intensive healthcare domains.

Analytical validation (accuracy, reproducibility, limit of detection) and gene expression analysis of FoundationOneRNA assay for fusion detection in 189 clinical tumor specimens

PLoS ONE · 2025-09-12 · 2 citations

Targeted DNA-based comprehensive genomic profiling (CGP) to detect clinically significant alterations is increasingly becoming standard for patients with advanced or recurrent cancer. RNA-based sequencing, however, may improve performance of fusion detection. We developed a robust targeted RNA sequencing assay (FoundationOne®RNA) and evaluated its analytic performance. FoundationOne®RNA is a hybrid-capture based targeted RNA sequencing test designed to optimally detect fusions (318 genes) and measure gene expression (1521 genes). Analytical validation studies were performed in College of American Pathologists (CAP)-accredited and Clinical Laboratory Improvement Amendments (CLIA)-certified lab to assess fusion call accuracy, assay reproducibility, limit of detection (LoD) and gene expression in 189 clinical solid tumor specimens. In the accuracy study, 160 out of 189 biopsy samples which were previously profiled using large-panel DNA- or RNA-based next-generation sequencing (NGS) passed quality control metrics and were studied using the FoundationOne®RNA assay. Analysis of all diagnostic fusions showed a positive percent agreement (PPA) of 98.28%, as well as a negative percent agreement (NPA) of 99.89% when compared to orthogonal assays. The FoundationOne®RNA assay was able to identify a low level BRAF fusion missed by orthogonal whole transcriptome RNA sequencing and was confirmed by fluorescence in situ hybridization (FISH). The range for the minimum RNA input and LoD was determined based on dilutions from 5 fusion-positive cell lines. It spans from 1.5ng (0.5% input) to 30ng (10% input) for RNA input and from 21 to 85 supporting reads for LoD. In the precision study, 10 out of 10 pre-defined target fusions had 100% reproducibility. In our gene expression analysis, multiple gene expression signatures were detected in fusion positive samples. FoundationOne®RNA assay successfully detected oncogenic fusions with high concordance to orthogonal NGS based tests, high reproducibility, and low limit of detection. This study demonstrated the robustness of FoundationOne®RNA and supports its use as a supplement to tissue DNA comprehensive genomic profiling (CGP) in routine clinical practice. Additional work is required to clarify optimal clinical scenarios for fusion detection and enable gene expression biomarkers for clinical use.

Frequency of Practice-Changing Findings Identified by Comprehensive Genomic Profiling in Non-Myeloid Hematologic Malignancies

Clinical Lymphoma Myeloma & Leukemia · 2024-08-26

Comprehensive Genomic Profiling in Non-Myeloid Hematologic Malignancies Identifies Variants That Can Alter Clinical Practice

Hematology Reports · 2024-09-30

BACKGROUND: Comprehensive genomic profiling (CGP) is frequently adopted to direct the clinical care of myeloid neoplasms and solid tumors, but its utility in the care of lymphoid and histiocytic cancers is less well defined. METHODS: In this study, we aimed to evaluate the frequency at which mutations identified by CGP altered management in non-myeloid hematologic malignancies. We retrospectively examined the CGP results of 105 samples from 101 patients with non-myeloid hematologic malignancies treated at an academic medical center who had CGP testing between 2014 and 2021. RESULTS: CGP revealed one or more pathogenic or likely pathogenic variant in 92 (88%) of samples and 73 (72%) of tested patients had one or more mutations with diagnostic, prognostic, or therapeutic significance. The identification of a resistance variant resulted in the suspension of the active treatment or affected subsequent treatment choice in 9 (69%) out of 13 patients. However, the presence of a therapy sensitizing variant only led to consideration of a biomarker-directed therapy in 6 (10%) out of 61 patients. CONCLUSIONS: Overall, CGP of non-myeloid hematologic malignancies identified clinically significant variants in 72% of patients and resulted in a change in management in 22% of patients.

CLL-048 Frequency of Practice-Changing Findings Identified by Comprehensive Genomic Profiling in Non-Myeloid Hematologic Malignancies

Clinical Lymphoma Myeloma & Leukemia · 2024-08-26

Addressing False Positives in High-Sensitivity Troponin I Testing: Mitigation Strategies

The Journal of Applied Laboratory Medicine · 2024-07-25

Expansion of an Academic Molecular Tumor Board to Enhance Access to Biomarker-Driven Trials and Therapies in the Rural Southeastern United States

Current Oncology · 2024-11-16 · 2 citations

Targeting tumor-specific molecular alterations has shown significant clinical benefit. Molecular tumor boards (MTBs) connect cancer patients with personalized treatments and clinical trials. However, rural cancer centers often have limited access to MTB expertise. We established an academic-community partnership expanding our academic MTB to affiliated rural community cancer centers. We developed a centralized molecular registry of tumors (MRT) to aggregate the comprehensive genomic profiling (CGP) results and facilitate multidisciplinary MTB review. Of the 151 patients included, 87 (58%) had actionable genomic biomarkers, 42 (28%) were eligible for a targeted off-label therapy, and 27 (18%) were matched to a clinical trial. Of those with a clinical trial match, only 1 of 27 (3%) was enrolled in the identified trial. One year into implementation, community oncology providers were anonymously surveyed on persistent barriers to precision treatment utilization. The primary barriers to clinical trial enrollment were the distance to the trial center (70%), lack of transportation (55%), and lack of local trials (50%). This study offers a framework to improve access to molecular expertise, but significant barriers to the equitable use of CGP and trial enrollment persist.

Supplementary Figure 1 from Characterization of an Oxaliplatin Sensitivity Predictor in a Preclinical Murine Model of Colorectal Cancer

2023-04-03

<p>PDF file - 39K, Fifty-nine samples from the NCI-60 cell panel were RMA normalized and base on 3D principal components analysis, MCF-7 was ommitted from further analysis.</p>

Supplementary Table 3 from Characterization of an Oxaliplatin Sensitivity Predictor in a Preclinical Murine Model of Colorectal Cancer

2023-04-03

<p>PDF file - 39K, Tumor growth inhibition of patient derived colorectal cancer explant to oxaliplatin.</p>

Supplementary Table 2 from Characterization of an Oxaliplatin Sensitivity Predictor in a Preclinical Murine Model of Colorectal Cancer

2023-04-03

<p>PDF file - 61K, Gene List of Oxaliplatin Sensitivity Signature.</p>