We use cookies to ensure our website works properly and to personalise your experience. Cookies policy
1Department of Clinical Pharmacy, Mumbai (MS), India
2Department of Radiodiagnosis, Medical Imaging Centre, Kochi (KL), India
3Department of Oncology, Cancer Care Institute, New Delhi (DL), India
4Satyjeet College of Pharmacy, Mehkar (MS), India
Radiology reports serve as the primary communication medium between radiologists and referring clinicians, providing critical information for diagnosis, treatment planning, and disease monitoring. In oncology imaging, the growing volume and complexity of radiological examinations have increased the demand for efficient and accurate reporting systems. Recent advances in artificial intelligence (AI), particularly foundation models, have transformed the landscape of automated radiology report generation. Foundation models are large-scale pretrained neural networks capable of learning generalized representations from vast multimodal datasets and adapting to diverse downstream tasks. Their integration with medical imaging and natural language processing has enabled the development of sophisticated systems capable of generating clinically meaningful radiology reports. This review examines the evolution of automated radiology report generation from traditional rule-based approaches and convolutional neural network (CNN)-based architectures to transformer-based foundation models and multimodal large language models (LLMs). Particular emphasis is placed on oncology imaging applications, including computed tomography (CT), magnetic resonance imaging (MRI), positron emission tomography (PET), and hybrid imaging modalities. The review discusses major foundation models, datasets, benchmarking methodologies, clinical applications, advantages, and limitations. Furthermore, ethical, regulatory, and privacy concerns associated with clinical deployment are explored. Current evidence suggests that foundation models significantly improve report quality, contextual understanding, and generalizability compared with earlier approaches. However, challenges related to hallucination, explainability, data heterogeneity, and clinical validation remain substantial barriers to widespread adoption. Future research should focus on domain-specific multimodal foundation models, federated learning frameworks, explainable AI mechanisms, and prospective clinical evaluation studies. Foundation models have the potential to reshape radiology workflows by enhancing efficiency, consistency, and diagnostic support in oncology imaging.
Medical imaging plays a central role in modern oncology by supporting cancer detection, staging, treatment planning, and therapeutic response assessment [1,2]. The interpretation of radiological examinations generates large volumes of textual reports that communicate imaging findings and clinical recommendations. Increasing imaging workloads, radiologist shortages, and demands for standardized reporting have motivated the development of automated report generation systems [3]. Traditional AI approaches relied on handcrafted features and rule-based methods, which lacked scalability and adaptability. The emergence of deep learning introduced CNN-based architectures capable of extracting image features automatically. Subsequently, transformer architectures and foundation models revolutionized AI by enabling large-scale pretraining and transfer learning across multiple domains [4]. Foundation models, including vision transformers (ViTs), multimodal transformers, and large language models, have demonstrated remarkable capabilities in image understanding, language generation, and clinical reasoning. These advances are particularly relevant to oncology imaging, where comprehensive interpretation requires integration of complex anatomical, functional, and longitudinal information [5]. This review critically examines the role of foundation models in automated radiology report generation, with a particular focus on oncology imaging applications.
2. Foundation Models in Medical Imaging
Foundation models are large-scale neural networks trained on extensive datasets using self-supervised or weakly supervised learning strategies. Unlike task-specific models, foundation models learn generalized representations that can be adapted to multiple downstream tasks with minimal additional training [6].
Examples include:
The principal advantage of foundation models is their ability to leverage massive datasets and capture complex relationships between images and textual information. In medical imaging, these models can learn anatomical structures, disease patterns, and clinical terminology simultaneously [7]. Compared with CNN-based systems, foundation models exhibit superior scalability and transferability. CNNs excel at local feature extraction but often struggle with long-range dependencies and contextual understanding. Transformer-based foundation models overcome these limitations through self-attention mechanisms that capture global relationships within imaging data [8].
3. Evolution of Automated Radiology Report Generation
Automated radiology report generation has evolved through several technological phases.
3.1 Rule-Based Systems
Early automated radiology report generation systems were primarily based on predefined templates and expert-designed rule sets. These systems generated reports by matching detected imaging findings with predetermined phrases and reporting structures. Their main advantage was high interpretability, as every output could be traced back to specific rules created by domain experts. Additionally, they ensured consistency in report formatting and terminology. However, these approaches lacked flexibility and struggled to handle complex or unexpected clinical scenarios. Since they depended heavily on manually crafted rules, adapting them to new diseases, imaging modalities, or reporting styles was challenging. Consequently, their limited generalizability restricted widespread clinical applicability [9].
3.2 CNN-Based Encoder–Decoder Models
The introduction of deep learning significantly advanced the field of automated radiology report generation through the development of encoder–decoder architectures. In these systems, convolutional neural networks (CNNs) function as encoders that automatically extract meaningful visual features from medical images, such as chest X-rays, CT scans, or MRI images. Unlike traditional machine learning methods that rely on handcrafted features, CNNs learn hierarchical image representations directly from large datasets, enabling more accurate identification of anatomical structures and pathological abnormalities. The extracted image features are then passed to recurrent neural networks (RNNs), such as Long Short-Term Memory (LSTM) or Gated Recurrent Unit (GRU) networks, which act as decoders to generate descriptive textual reports.
One of the primary advantages of encoder–decoder architectures is their ability to perform automated feature extraction, reducing the need for manual intervention and expert-designed image descriptors. Additionally, these models are highly scalable and can be trained on large imaging datasets, allowing them to adapt to diverse reporting tasks. However, several limitations remain. RNN-based decoders often struggle to capture complex contextual relationships within medical images and reports. Their sequential processing nature makes it difficult to model long-term dependencies, leading to information loss in lengthy report generation tasks. Consequently, generated reports may lack clinical completeness and coherence, motivating the adoption of transformer-based and foundation-model approaches that better capture global contextual information and improve report quality [10].
3.3 Transformer-Based Architectures
The introduction of transformer architectures marked a significant advancement in automated radiology report generation. Unlike recurrent neural networks (RNNs), which process information sequentially, transformers employ self-attention mechanisms that enable the model to analyze all input features simultaneously. This capability allows transformers to capture both local and global relationships within medical images and associated textual information more effectively. Transformer Encoder–Decoder frameworks use an encoder to extract comprehensive image representations and a decoder to generate coherent and contextually relevant reports. By modeling long-range dependencies, transformers overcome the limitations of RNNs, which often struggle with information loss in lengthy sequences. As a result, transformer-based models produce reports with improved accuracy, consistency, and clinical relevance. Their ability to focus on important image regions through attention mechanisms enhances the description of abnormalities and disease patterns. Consequently, transformers have become the foundation of modern radiology reporting systems and have paved the way for the development of large multimodal foundation models. [11].
3.4 Foundation Models and Multimodal AI
Recent systems combine large-scale image encoders and language models. These architectures perform image interpretation, clinical reasoning, and report generation within a unified framework [12]. The transition from CNN-based systems to foundation models represents a paradigm shift from task-specific learning to generalized multimodal intelligence.
4. Application of Foundation Models in Oncology Imaging
Oncology imaging presents unique challenges due to tumor heterogeneity, longitudinal monitoring requirements, and multimodal data integration.
4.1 Computed Tomography
CT remains the most widely used modality for cancer staging and follow-up. Foundation models can identify lesions, quantify tumor burden, and generate structured reports describing disease extent [13].
4.2 Magnetic Resonance Imaging
MRI provides superior soft-tissue characterization. Foundation models have demonstrated promising performance in brain tumor characterization, prostate cancer assessment, and liver lesion evaluation [14].
4.3 PET and Hybrid Imaging
PET/CT and PET/MRI generate large amounts of functional and anatomical information. Automated reporting systems based on foundation models can integrate multimodal findings into coherent clinical narratives [15].
4.4 Longitudinal Cancer Monitoring
Foundation models are particularly valuable for comparing serial imaging studies and summarizing treatment responses according to standardized criteria such as RECIST [16].
5. Large Language Models and Vision-Language Models for Radiology Reporting
Large language models (LLMs) have emerged as a transformative technology in medical report generation due to their exceptional ability to understand and generate human-like text. Built on transformer architectures and trained on massive datasets, LLMs can capture complex linguistic patterns, medical terminology, and contextual relationships. In radiology, these models can convert imaging findings into coherent, structured, and clinically meaningful reports. Compared with traditional report generation methods, LLMs improve language fluency, consistency, and completeness. They can also integrate information from multiple sources, supporting more accurate clinical documentation. However, challenges such as hallucinations, bias, and explainability must be addressed before widespread clinical adoption.
5.1 Large Language Models
Models such as GPT-based architectures can generate fluent, contextually appropriate text. When combined with radiological image representations, they can produce reports that resemble those written by expert radiologists [17].
5.2 Vision-Language Models
Vision-language models (VLMs) integrate image analysis and natural language processing within a unified framework, enabling simultaneous understanding of visual and textual information. Examples include CLIP-inspired architectures and multimodal transformer models that learn associations between medical images and corresponding reports. In radiology, these models enhance automated report generation by accurately linking imaging findings with relevant clinical descriptions. A major advantage of VLMs is their improved alignment between visual features and textual content, resulting in more precise and clinically meaningful reports. They also improve report coherence by maintaining contextual consistency throughout the generated text. Furthermore, VLMs support advanced retrieval, summarization, and decision-support applications in medical imaging [18].
5.3 Comparative Analysis
|
Approach |
Strengths |
Limitations |
|
CNN-based |
Efficient feature extraction |
Limited context |
|
Transformer-based |
Global attention |
Data intensive |
|
Foundation models |
Transfer learning, scalability |
High computational cost |
|
Multimodal LLMs |
Clinical reasoning, report quality |
Hallucinations, explainability concerns |
6. Datasets and Benchmarking Methods
Robust datasets are essential for developing and evaluating automated reporting systems.
Major datasets include:
Evaluation metrics commonly include:
Although these metrics assess linguistic similarity, they may not adequately capture clinical correctness. Consequently, clinical efficacy metrics and expert evaluations are increasingly emphasized [19].
7. Clinical Applications and Benefits
Foundation-model-based reporting systems offer several clinical advantages.
Improved Efficiency
Automated report generation can reduce reporting time and alleviate radiologist workload [20].
Standardization
Structured reporting improves consistency across institutions and reduces inter-observer variability [21].
Decision Support
Foundation models can highlight abnormalities and suggest differential diagnoses, supporting clinical decision-making [22].
Enhanced Oncology Care
Automated summarization of tumor burden and treatment response may improve multidisciplinary cancer management [23].
Table 1. Representative Foundation Models for Automated Radiology Report Generation
|
Model |
Year |
Architecture |
Dataset |
Key Features |
Limitations |
|
BioViL |
2022 |
Vision-Language Transformer |
MIMIC-CXR |
Cross-modal learning |
Limited oncology-specific validation |
|
GatorTron |
2022 |
Large Language Model |
Clinical Text Corpora |
Clinical language understanding |
Requires adaptation for imaging |
|
Med-PaLM |
2023 |
Medical LLM |
Multisource Medical Data |
Medical reasoning |
Hallucination risk |
|
LLaVA-Med |
2024 |
Vision-Language Model |
Biomedical Images |
Multimodal interaction |
Limited regulatory validation |
|
Med-Gemini |
2025 |
Multimodal Foundation Model |
Multimodal Medical Data |
Integrated reasoning |
Computational complexity |
CHALLENGES AND LIMITATIONS
Despite significant advancements in automated radiology report generation, hallucinations remain a major challenge in large language models (LLMs). Hallucinations occur when a model generates information that appears plausible and linguistically correct but is not supported by the actual medical image or clinical data. In radiology, such errors may include reporting nonexistent abnormalities, omitting important findings, or providing inaccurate clinical interpretations. These mistakes can compromise diagnostic accuracy and patient safety, limiting the reliability of fully automated reporting systems. Therefore, robust validation methods, multimodal verification mechanisms, and human oversight by radiologists are essential to ensure the clinical accuracy and trustworthiness of AI-generated reports. [24].
Data Heterogeneity
Hallucinations represent one of the most critical limitations of large language models used in automated radiology report generation. A hallucination occurs when the model produces information that is not supported by the input medical image or clinical data. Although the generated text may appear accurate and convincing, it can contain incorrect findings, missed abnormalities, or fabricated clinical details. In oncology imaging, such errors may affect diagnosis, staging, and treatment decisions, potentially compromising patient care. Reducing hallucinations requires improved multimodal training, rigorous validation, explainable AI techniques, and continuous human supervision. Therefore, radiologist oversight remains essential for ensuring report accuracy and clinical reliability. [25].
Limited Explainability
Many foundation models operate as "black-box" systems, meaning their internal decision-making processes are difficult for clinicians to interpret and understand. Although these models can achieve high performance in image analysis and report generation, they often provide limited explanations for how specific conclusions or recommendations are derived. This lack of transparency can reduce clinician confidence and hinder the adoption of AI systems in routine clinical practice. In oncology imaging, where diagnostic decisions directly influence patient management, explainability is particularly important. Developing interpretable AI methods, attention visualization tools, and transparent decision-support mechanisms is essential to improve trust, accountability, and safe clinical implementation. [26].
Computational Requirements
One of the major challenges associated with foundation models in radiology is their substantial computational requirement. Training these large-scale models typically involves billions of parameters and requires access to high-performance computing infrastructure, including advanced graphics processing units (GPUs), tensor processing units (TPUs), extensive memory, and large-scale storage systems. The training process can be time-consuming and expensive, making it difficult for smaller healthcare institutions and research centers to develop or fine-tune such models independently. Additionally, deploying foundation models in clinical settings requires significant computational resources to ensure rapid and reliable report generation. These infrastructure demands can create accessibility barriers, particularly in low-resource environments and developing regions. Furthermore, increased energy consumption and operational costs raise concerns regarding sustainability. Therefore, developing more efficient model architectures, optimization techniques, and cloud-based deployment strategies is essential to improve accessibility and facilitate broader adoption of foundation-model-based radiology reporting systems [27].
9. Ethical, Regulatory, and Privacy Considerations
The clinical implementation of foundation models in radiology requires careful consideration of ethical, legal, and regulatory issues. Protecting patient privacy and ensuring data security are essential due to the sensitive nature of medical information used for model training and deployment. Algorithmic bias may arise from unrepresentative datasets, potentially leading to unequal performance across different patient populations. Another critical concern is accountability when AI-generated reports contain errors that affect clinical decisions. Additionally, compliance with healthcare regulations and medical device standards is necessary before clinical adoption. Consequently, healthcare authorities increasingly emphasize transparent validation, continuous performance monitoring, and post-deployment surveillance to ensure safe and responsible AI implementation [28].
10. Future Perspectives and Research Directions
Future research in automated radiology report generation should focus on developing oncology-specific foundation models trained on diverse cancer imaging datasets to improve disease-specific performance and clinical relevance. Federated learning frameworks can facilitate collaboration among institutions while preserving patient privacy and data security. Explainable multimodal AI systems are needed to enhance transparency and clinician trust by providing interpretable reasoning behind generated reports. Integrating genomic, pathological, and imaging data may support more comprehensive and personalized cancer assessment. Additionally, prospective multicenter clinical trials are essential to validate model performance across different healthcare settings and patient populations. Real-time clinical deployment studies should evaluate workflow integration, efficiency, and patient outcomes. Furthermore, human–AI collaborative reporting systems that combine the strengths of artificial intelligence and radiologist expertise may improve report accuracy, reduce workload, and promote safer adoption of AI technologies in oncology imaging.
CONCLUSION
Foundation models represent a major advancement in automated radiology report generation. By integrating visual understanding, language generation, and multimodal reasoning, these systems offer substantial improvements over traditional CNN-based and transformer-based approaches. In oncology imaging, foundation models have the potential to enhance reporting efficiency, standardization, and clinical decision support. However, challenges related to hallucination, explainability, data diversity, and regulatory approval remain significant. Continued interdisciplinary collaboration among radiologists, oncologists, computer scientists, and regulatory bodies will be essential for realizing the full clinical potential of foundation-model-driven radiology reporting.
REFERENCES
Sneha Waghmare*, Meera Nair, Arjun Verma, Shatrughna Nagrik, Foundation Models for Automated Radiology Report Generation in Oncology Imaging, Int. J. Med. Pharm. Sci., 2026, 2 (7), 499-508. https://doi.org/10.5281/zenodo.21284177
10.5281/zenodo.21284177