Reporting checklist for foundation and large language models in medical research (REFINE): an international consensus guideline

Diagnostic and Interventional Radiology · 2026-02-26 · 3 citations

PURPOSE: To develop the REporting checklist for FoundatIon and large laNguagE models (REFINE), an international reporting guideline for transparent and reproducible reporting of foundation model (FM) and large language model (LLM) studies in medical research, including imaging artificial intelligence (AI) applications. METHODS: The protocol was prespecified and publicly archived. A modified Delphi process was conducted to establish reporting standards for unimodal and multimodal FM and LLM applications involving text, imaging, and structured data. The steering committee coordinated protocol development, expert recruitment, all Delphi rounds, and the harmonization phase. Decisions were made based on predefined consensus thresholds. In Rounds 1 and 2, structured ratings and free-text feedback informed iterative revisions. In the post-Delphi harmonization phase, terminology was standardized, and detailed reporting instructions were finalized. RESULTS: The REFINE development group comprised 57 contributors from 17 countries, and 54 panelists from 16 countries completed Rounds 1 and 2. The harmonization phase was completed by three expert panelists and the steering committee. The entire process produced a 44-item, six-section framework with standardized terminology and detailed reporting instructions, supported by an online platform for practical use (https://refinechecklist.github.io/refine/checklist.html). CONCLUSION: The REFINE provides a comprehensive, consensus-based reporting standard for medical FM and LLM research, including imaging AI studies. The online version facilitates practical implementation. CLINICAL SIGNIFICANCE: The REFINE enables transparent, comparable, and reproducible reporting of FM and LLM studies, supporting reliable evidence synthesis in medical and imaging-focused AI studies.

Assessing the Effectiveness of the <i>Radiology</i> In Training Program in Fostering Highly Skilled Reviewers

Radiology · 2026-04-01

Radiology In Training effectively trains skilled scientific reviewers, and the majority of its members continue serving as Radiology reviewers after graduating from the program.

Dynamic breast MRI with Flexible Temporal Resolution Aided by Deep Learning

Nature Communications · 2026-05-19

MRI is the most effective method for screening high-risk breast cancer patients. While current exams rely on the qualitative evaluation of morphological features before and after contrast administration and less on contrast kinetic information, recent developments in fast acquisition methods aim to combine both. However, balancing spatial resolution, temporal resolution and scan time poses a considerable challenge in dynamic MRI. Here, we introduce a radial MRI reconstruction framework for Dynamic Contrast Enhanced (DCE) imaging, termed Enhanced Locally low-rank Imaging for Tissue contrast Enhancement (ELITE), to address these limitations. ELITE combines locally low-rank subspace modeling to capture spatially localized tissue dynamics with deep learning. We evaluate its effectiveness using the publicly available fastMRI breast initiative, demonstrating substantial improvements in CNR and noise reduction while enabling flexible temporal resolution down to 1 second. ELITE also shows benefits in neck and brain imaging, making it a viable alternative for other DCE-MRI applications. Dynamic Contrast Enhanced (DCE) MRI allows highly sensitive cancer screening, but balancing between temporal and spatial resolution poses a challenge in dynamic DCE-MRI. Here, the authors develop ELITE, an image reconstruction framework powered by AI, to overcome such limitations and enhance breast, head and neck, and brain DCE-MRI screening with high spatial and temporal fidelity.

Guidelines for Reporting Studies on Large Language Models in Radiology: An International Delphi Expert Survey

Radiology · 2026-02-01

Large language models (LLMs) have transformative potential in radiology, including textual summaries, diagnostic decision support, proofreading, and image analysis. However, the rapid increase in studies investigating these models, along with the lack of standardized LLM-specific reporting practices, affects reproducibility, reliability, and clinical applicability. To address this, reporting guidelines for LLM studies in radiology were developed using a two-step process. First, a systematic review of LLM studies in radiology was conducted across PubMed, IEEE Xplore, and the ACM Digital Library, covering publications between May 2023 and March 2024. Of 511 screened studies, 57 were included to identify relevant aspects for the guidelines. Then, in a Delphi process, 20 international experts developed the final list of items for inclusion. Items consented as relevant were summarized into a structured checklist containing 32 items across six key categories: general information and data input; prompting and fine-tuning; performance metrics; ethics and data transparency; implementation, risks, and limitations; and further/optional aspects. The final FLAIR (Framework for LLM Assessment in Radiology) checklist aims to standardize reporting of LLM studies in radiology, fostering transparency, reproducibility, comparability, and clinical applicability to enhance clinical translation and patient care. © The Author(s) 2026. Published by the Radiological Society of North America under a CC BY 4.0 license.

Retrospective BReast Intravoxel Incoherent Motion Multisite (BRIMM) multisoftware study

Frontiers in Oncology · 2025-02-24 · 1 citations

Introduction The intravoxel incoherent motion (IVIM) model of diffusion weighted imaging (DWI) provides imaging biomarkers for breast tumor characterization. It has been extensively applied for both diagnostic and prognostic goals in breast cancer, with increasing evidence supporting its clinical relevance. However, variable performance exists in literature owing to the heterogeneity in datasets and quantification methods. Methods This work used retrospective anonymized breast MRI data (302 patients) from three sites employing three different software utilizing least-squares segmented algorithms and Bayesian fit to estimate 1 st order radiomics of IVIM parameters perfusion fraction ( f p ), pseudo-diffusion ( D p ) and tissue diffusivity ( D t ). Pearson correlation ( r ) coefficients between software pairs were computed while logistic regression model was implemented to test malignancy detection and assess robustness of the IVIM metrics. Results D t and f p maps generated from different software showed consistency across platforms while D p maps were variable. The average correlation between the three software pairs at three different sites for 1 st order radiomics of IVIM parameters were D t min / D t max / D t mean / D t variance / D t skew / D t kurt : 0.791/0.891/0.98/0.815/0.697/0.584; f p max / f p mean / f p variance / f p skew / f p kurt : 0.615/0.871/0.679/0.541/0.433; D p max / D p mean / D p variance / D p skew / D p kurt : 0.616/0.56/0.587/0.454/0.51. Correlation between least-squares algorithms were the highest. D t mean showed highest area under the ROC curve (AUC) with 0.85 and lowest coefficient of variation (CV) with 0.18% for benign and malignant differentiation using logistic regression. D t metrics were highly diagnostic as well as consistent along with f p metrics. Discussion Multiple 1 st order radiomic features of D t and f p obtained from a heterogeneous multi-site breast lesion dataset showed strong software robustness and/or diagnostic utility, supporting their potential consideration in controlled prospective clinical trials.

Pitfalls and Best Practices in Evaluation of AI Algorithmic Biases in Radiology

Radiology · 2025-05-01 · 15 citations

statistical evaluations of bias. Although AI bias in radiology has been broadly reviewed in the recent literature, this article focuses specifically on underrecognized potential pitfalls related to the three key areas. By providing awareness of these pitfalls along with actionable practices to avoid them, exciting AI technologies can be used in radiology for the good of all people.

Best Practices for the Safe Use of Large Language Models and Other Generative AI in Radiology

Radiology · 2025-09-01 · 4 citations

As large language models (LLMs) and other generative artificial intelligence (AI) models are rapidly integrated into radiology workflows, unique pitfalls threatening their safe use have emerged. Problems with AI are often identified only after public release, highlighting the need for preventive measures to mitigate negative impacts and ensure safe, effective deployment into clinical settings. This article summarizes best practices for the safe use of LLMs and other generative AI models in radiology, focusing on three key areas that can lead to pitfalls if overlooked: regulatory issues, data privacy, and bias. To address these areas and minimize risk to patients, radiologists must examine all potential failure modes and ensure vendor transparency. These best practices are based on the best available evidence and the experiences of leaders in the field. Ultimately, this article provides actionable guidelines for radiologists, radiology departments, and vendors using and integrating generative AI into radiology workflows, offering a framework to prevent these problems.

Best Practices and Checklist for Reviewing Artificial Intelligence-Based Medical Imaging Papers: Classification

Journal of Imaging Informatics in Medicine · 2025-06-04 · 3 citations

Recent advances in Artificial Intelligence (AI) methodologies and their application to medical imaging has led to an explosion of related research programs utilizing AI to produce state-of-the-art classification performance. Ideally, research culminates in dissemination of the findings in peer-reviewed journals. To date, acceptance or rejection criteria are often subjective; however, reproducible science requires reproducible review. The Machine Learning Education Sub-Committee of the Society for Imaging Informatics in Medicine (SIIM) has identified a knowledge gap and need to establish guidelines for reviewing these studies. This present work, written from the machine learning practitioner standpoint, follows a similar approach to our previous paper related to segmentation. In this series, the committee will address best practices to follow in AI-based studies and present the required sections with examples and discussion of requirements to make the studies cohesive, reproducible, accurate, and self-contained. This entry in the series focuses on image classification. Elements like dataset curation, data pre-processing steps, reference standard identification, data partitioning, model architecture, and training are discussed. Sections are presented as in a typical manuscript. The content describes the information necessary to ensure the study is of sufficient quality for publication consideration and, compared with other checklists, provides a focused approach with application to image classification tasks. The goal of this series is to provide resources to not only help improve the review process for AI-based medical imaging papers, but to facilitate a standard for the information that should be presented within all components of the research study.

Dynamic MRI with Locally Low-Rank Subspace Constraint: Towards 1-Second Temporal Resolution Aided by Deep Learning

Research Square · 2025-02-27

Performance of Algorithms Submitted in the 2023 RSNA Screening Mammography Breast Cancer Detection AI Challenge

Radiology · 2025-08-01 · 5 citations

Algorithms submitted to the RSNA Screening Mammography Breast Cancer Detection AI Challenge detected different cancers; ensemble models had increased sensitivity while maintaining low recall rates; and sensitivity was higher for invasive versus noninvasive cancers.