Imaging more of the body. Nat. Biomed. Eng. 6, 495–496 (2022).
Bouma, B. E. et al. Optical coherence tomography. Nat. Rev. Methods Prim. 2, 79 (2022).
Google Scholar
Beaton, L., Bandula, S., Gaze, M. N. & Sharma, R. A. How rapid advances in imaging are defining the future of precision radiation oncology. Br. J. Cancer 120, 779–790 (2019).
Google Scholar
Khalifa, M. & Albadawy, M. AI in diagnostic imaging: revolutionising accuracy and efficiency. Comp. Methods Program. Biomed. Update 5, 100146 (2024).
Park, J. et al. Clinical translation of photoacoustic imaging. Nat. Rev. Bioeng. 16, 4353–4379 (2024).
Zhao, C., Guo, L., Dong, J. & Cai, Z. Mass spectrometry imaging-based multi-modal technique: next-generation of biochemical analysis strategy. Innovation 2, 100151 (2021).
Duan, J., Xiong, J., Li, Y. & Ding, W. Deep learning based multimodal biomedical data fusion: an overview and comparative review. Inf. Fusion 112, 102536 (2024).
Ma, J. et al. Segment anything in medical images. Nat. Commun. 15, 654 (2024).
Google Scholar
Li, M., Jiang, Y., Zhang, Y. & Zhu, H. Medical image analysis using deep learning algorithms. Front. Public Health 11, 1273253 (2023).
Google Scholar
Li, X., Zhang, L., Yang, J. & Teng, F. Role of artificial intelligence in medical image analysis: a review of current trends and future directions. J. Med. Biol. Eng. 44, 231–243 (2024).
Thirunavukarasu, A. J. et al. Large language models in medicine. Nat. Med. 29, 1930–1940 (2023).
Google Scholar
Zhang, K. et al. A generalist vision–language foundation model for diverse biomedical tasks. Nat. Med. 30, 3129–3141 (2024).
He, Y. et al. Foundation model for advancing healthcare: challenges, opportunities and future directions. IEEE Rev. Biomed. Eng. 18, 172–191 (2024).
Ryu, J. S., Kang, H., Chu, Y. & Yang, S. Vision-language foundation models for medical imaging: a review of current practices and innovations. Biomed. Eng. Lett. 15, 809–830 (2025).
Haq, I. U. et al. Advancements in medical radiology through multimodal machine learning: a comprehensive overview. Bioengineering 12, 477 (2025).
Google Scholar
Bahadir, C. D. et al. Artificial intelligence applications in histopathology. Nat. Rev. Electr. Eng. 1, 93–108 (2024).
Google Scholar
Guarrasi, V. et al. A systematic review of intermediate fusion in multimodal deep learning for biomedical applications. Image Vis. Comput. 158, 105509 (2025).
Warner, E. et al. Multimodal machine learning in image-based and clinical biomedicine: survey and prospects. Int. J. Comp. Vis. 132, 1–17 (2024).
Acosta, J. N., Falcone, G. J., Rajpurkar, P. & Topol, E. J. Multimodal biomedical AI. Nat. Med. 28, 1773–1784 (2022).
Google Scholar
Rong, J. & Liu, Y. Advances in medical imaging techniques. BMC Methods 1, 10 (2024).
Google Scholar
Chen, J. & Chen, J. Multimodal image feature fusion for improving medical ultrasound image segmentation. Biomed. Signal Process. Control 89, 105705 (2024).
Google Scholar
Ottestad, A. L. et al. Associations between detectable circulating tumor dna and tumor glucose uptake measured by 18f-fdg pet/ct in early-stage non-small cell lung cancer. BMC Cancer 23, 646 (2023).
Google Scholar
Illimoottil, M. & Ginat, D. Recent advances in deep learning and medical imaging for head and neck cancer treatment: MRI, CT, and PET scans. Cancers 15, 3267 (2023).
Google Scholar
Kaniyala Melanthota, S. et al. Types of spectroscopy and microscopy techniques for cancer diagnosis: a review. Lasers Med. Sci. 37, 3067–3084 (2022).
Google Scholar
Zhang, H. et al. Mass spectrometry imaging for spatially resolved multi-omics molecular mapping. npj Imaging 2, 20 (2024).
Google Scholar
Doherty, T. et al. Feature fusion of Raman chemical imaging and digital histopathology using machine learning for prostate cancer detection. Analyst 146, 4195–4211 (2021).
Google Scholar
Nunes, J. B. et al. Integration of mass cytometry and mass spectrometry imaging for spatially resolved single-cell metabolic profiling. Nat. Methods 21, 1796–1800 (2024).
Google Scholar
Wang, D. et al. Automated machine-learning framework integrating histopathological and radiological information for predicting IDH1 mutation status in glioma. Front. Bioinforma. 1, 718697 (2021).
Google Scholar
Fitzgerald, S. et al. Multimodal raman spectroscopy and optical coherence tomography for biomedical analysis. J. Biophotonics 16, e202200231 (2023).
Google Scholar
Liu, Y., Valji, K., Monsky, W., Zheng, C. & Yang, X. Optical imaging guidance in oncologic surgery and interventional oncology. Pharmacol. Res. 212, 107612 (2025).
Lin, L. & Wang, L. V. The emerging role of photoacoustic imaging in clinical oncology. Nat. Rev. Clin. Oncol. 19, 365–384 (2022).
Google Scholar
Smith, B., Hermsen, M., Lesser, E., Ravichandar, D. & Kremers, W. Developing image analysis pipelines of whole-slide images: pre-and post-processing. J. Clin. Transl. Sci. 5, e38 (2021).
Google Scholar
Cammarasana, S., Nicolardi, P. & Patanè, G. Real-time denoising of ultrasound images based on deep learning. Med. Biol. Eng. Comput. 60, 2229–2244 (2022).
Google Scholar
Chen, H. et al. Low-dose ct with a residual encoder-decoder convolutional neural network. IEEE Trans. Med. Imaging 36, 2524–2535 (2017).
Google Scholar
Gao, Q. & Shan, H. Cocodiff: a contextual conditional diffusion model for low-dose CT image denoising. In Developments in X-Ray Tomography XIV, 12242, 92–98 (SPIE, 2022).
Lin, G. et al. Multimodal feature-guided diffusion model for low-count PET image denoising. Med. Phys. 52, 4403–4415 (2025).
Parvaiz, A. et al. Vision transformers in medical computer vision-a contemplative retrospection. Eng. Appl. Artif. Intell. 122, 106126 (2023).
Google Scholar
Singh, A. R. et al. An enhanced denoising system for mammogram images using deep transformer model with fusion of local and global features. Sci. Rep. 15, 6562 (2025).
Google Scholar
Song, T.-A., Yang, F. & Dutta, J. Noise2void: unsupervised denoising of pet images. Phys. Med. Biol. 66, 214002 (2021).
Google Scholar
Shen, Y. et al. Pads-net: Gan-based radiomics using multi-task network of denoising and segmentation for ultrasonic diagnosis of parkinson disease. Computerized Med. Imaging Graph. 120, 102490 (2025).
Google Scholar
Daras, G. et al. A survey on diffusion models for inverse problems. arXiv preprint arXiv:2410.00083 (2024).
Chung, H., Kim, J. & Ye, J. C. Diffusion models for inverse problems. arXiv:2508.01975 (2025).
Kim, S. et al. Unsupervised learning for motion correction and assessment in brain magnetic resonance imaging using severity-based regularized cycle consistency. Eng. Appl. Artif. Intell. 142, 109978 (2025).
Google Scholar
Guo, K. et al. Medgan: An adaptive gan approach for medical image generation. Computers Biol. Med. 163, 107119 (2023).
Google Scholar
Lim, A., Lo, J., Wagner, M. W., Ertl-Wagner, B. & Sussman, D. Motion artifact correction in fetal mri based on a generative adversarial network method. Biomed. Signal Process. Control 81, 104484 (2023).
Google Scholar
Chen, Z. et al. Portable head ct motion artifact correction via diffusion-based generative model. Computerized Med. Imaging Graph. 119, 102478 (2025).
Google Scholar
Islam, T., Hafiz, M. S., Jim, J. R., Kabir, M. M. & Mridha, M. A systematic review of deep learning data augmentation in medical imaging: recent advances and future research directions. Healthcare Anal. 5, 100340 (2024).
Dayarathna, S. et al. Deep learning based synthesis of MRI, CT and PET: review and analysis. Med. image Anal. 92, 103046 (2024).
Google Scholar
Nag, A. et al. Tumorganet: a transfer learning and generative adversarial network-based data augmentation model for brain tumor classification. IEEE Access 12, 103060–103081 (2024).
Guan, Q. et al. Medical image augmentation for lesion detection using a texture-constrained multichannel progressive gan. Computers Biol. Med. 145, 105444 (2022).
Google Scholar
Pombo, G. et al. Equitable modelling of brain imaging by counterfactual augmentation with morphologically constrained 3d deep generative models. Med. Image Anal. 84, 102723 (2023).
Google Scholar
Zhang, Y. et al. Unified multi-modal image synthesis for missing modality imputation. IEEE Trans. Med. Imaging 44, 4–18 (2024).
Saad, M. M., O’Reilly, R. & Rehmani, M. H. A survey on training challenges in generative adversarial networks for biomedical image analysis. Artif. Intell. Rev. 57, 19 (2024).
Google Scholar
Han, C. et al. Combining noise-to-image and image-to-image GANS: brain MR image augmentation for tumor detection. IEEE Access 7, 156966–156977 (2019).
Google Scholar
Liu, Q., Zhou, T., Cheng, C., Ma, J. & Hoque Tania, M. Hybrid generative adversarial network based on frequency and spatial domain for histopathological image synthesis. BMC Bioinforma. 26, 29 (2025).
Google Scholar
Qiang, N. et al. Modeling and augmenting of FMRI data using deep recurrent variational auto-encoder. J. neural Eng. 18, 0460b6 (2021).
Google Scholar
Huo, J. et al. Brain lesion synthesis via progressive adversarial variational auto-encoder. In International Workshop on Simulation and Synthesis in Medical Imaging, 101–111 (Springer, 2022).
Wei, R. & Mahmood, A. Recent advances in variational autoencoders with representation learning for biomedical informatics: a survey. IEEE Access 9, 4939–4956 (2020).
Google Scholar
Neloy, A. A. & Turgeon, M. A comprehensive study of auto-encoders for anomaly detection: efficiency and trade-offs. Mach. Learn. Appl. 17, 100572 (2024).
Kazerouni, A. et al. Diffusion models in medical imaging: a comprehensive survey. Med. Image Anal. 88, 102846 (2023).
Google Scholar
Yang, Z. et al. A foundation model for generalizable cancer diagnosis and survival prediction from histopathological images. Nat. Commun. 16, 2366 (2025).
Google Scholar
Wu, C. et al. Towards generalist foundation model for radiology by leveraging web-scale 2d&3d medical data. Nat. Commun. 16, 7866 (2025).
Google Scholar
Van Griethuysen, J. J. et al. Computational radiomics system to decode the radiographic phenotype. Cancer Res. 77, e104–e107 (2017).
Google Scholar
Singh, G. et al. Radiomics and radiogenomics in gliomas: a contemporary update. Br. J. cancer 125, 641–657 (2021).
Google Scholar
Boehm, K. M. et al. Multimodal data integration using machine learning improves risk stratification of high-grade serous ovarian cancer. Nat. Cancer 3, 723–733 (2022).
Google Scholar
Zhang, X. et al. Deep learning with radiomics for disease diagnosis and treatment: challenges and potential. Front. Oncol. 12, 773840 (2022).
Google Scholar
Kalamkar, S. et al. Multimodal image fusion: a systematic review. Decis. Anal. J. 9, 100327 (2023).
Azam, M. A. et al. A review on multimodal medical image fusion: compendious analysis of medical modalities, multimodal databases, fusion techniques and quality metrics. Computers Biol. Med. 144, 105253 (2022).
Google Scholar
Huang, B., Yang, F., Yin, M., Mo, X. & Zhong, C. A review of multimodal medical image fusion techniques. Comput. Math. methods Med. 2020, 8279342 (2020).
Google Scholar
Liu, J., Dian, R., Li, S. & Liu, H. Sgfusion: a saliency guided deep-learning framework for pixel-level image fusion. Inf. Fusion 91, 205–214 (2023).
Google Scholar
Zhou, J. et al. A fusion algorithm based on composite decomposition for PET and MRI medical images. Biomed. Signal Process. Control 76, 103717 (2022).
Google Scholar
Li, X., Zhou, F., Tan, H., Zhang, W. & Zhao, C. Multimodal medical image fusion based on joint bilateral filter and local gradient energy. Inf. Sci. 569, 302–325 (2021).
Google Scholar
Odusami, M., Maskeliūnas, R. & Damaševičius, R. Pixel-level fusion approach with vision transformer for early detection of alzheimer’s disease. Electronics 12, 1218 (2023).
Google Scholar
Bischof, J. et al. Multimodal bioimaging across disciplines and scales: challenges, opportunities and breaking down barriers. npj Imaging 2, 5 (2024).
Google Scholar
Yuan, Y. & Cheng, Y. Medical image segmentation with unet-based multi-scale context fusion. Sci. Rep. 14, 15687 (2024).
Google Scholar
Ma, J., Xu, H., Jiang, J., Mei, X. & Zhang, X.-P. DDCGAN: a dual-discriminator conditional generative adversarial network for multi-resolution image fusion. IEEE Trans. Image Process. 29, 4980–4995 (2020).
Google Scholar
Tang, W. & He, F. Fatfusion: A functional–anatomical transformer for medical image fusion. Inf. Process. Manag. 61, 103687 (2024).
Google Scholar
Tang, W., He, F., Liu, Y. & Duan, Y. Matr: multimodal medical image fusion via multiscale adaptive transformer. IEEE Trans. Image Process. 31, 5134–5149 (2022).
Google Scholar
Bierbrier, J., Gueziri, H.-E. & Collins, D. L. Estimating medical image registration error and confidence: a taxonomy and scoping review. Med. Image Anal. 81, 102531 (2022).
Google Scholar
Wang, Z. & Hong, Y. A2fseg: Adaptive multi-modal fusion network for medical image segmentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention, 673–681 (Springer, 2023).
Bi, Y., Abrol, A., Fu, Z. & Calhoun, V. D. A multimodal vision transformer for interpretable fusion of functional and structural neuroimaging data. Hum. Brain Mapp. 45, e26783 (2024).
Google Scholar
Song, H. et al. Multimodal separation and cross fusion network based on Raman spectroscopy and FTIR spectroscopy for diagnosis of thyroid malignant tumor metastasis. Sci. Rep. 14, 29125 (2024).
Google Scholar
Chen, J. et al. A deep learning-based multimodal medical imaging model for breast cancer screening. Sci. Rep. 15, 14696 (2025).
Google Scholar
Thakoor, K. A. et al. A multimodal deep learning system to distinguish late stages of amd and to compare expert vs. AI ocular biomarkers. Sci. Rep. 12, 2585 (2022).
Google Scholar
Dasanayaka, S., Shantha, V., Silva, S., Meedeniya, D. & Ambegoda, T. Interpretable machine learning for brain tumour analysis using mri and whole slide images. Softw. Impacts 13, 100340 (2022).
Google Scholar
Zhang, T. et al. Predicting breast cancer types on and beyond molecular level in a multi-modal fashion. npj Breast Cancer 9, 16 (2023).
Google Scholar
Wu, J. et al. Multimodal microscopic imaging with deep learning for highly effective diagnosis of breast cancer. Opt. Lasers Eng. 168, 107667 (2023).
Google Scholar
Zhou, C. et al. Multimodal data integration for predicting progression risk in castration-resistant prostate cancer using deep learning: a multicenter retrospective study. Front. Oncol. 14, 1287995 (2024).
Google Scholar
Morais, M., Calisto, F. M., Santiago, C., Aleluia, C. & Nascimento, J. C. Classification of breast cancer in mri with multimodal fusion. In 2023 IEEE 20th international symposium on biomedical imaging (ISBI), 1–4 (IEEE, 2023).
Barai, A. et al. A late fusion deep cnn model for the classification of brain tumors from multi-parametric mri images. In 2023 International Conference on Next-Generation Computing, IoT and Machine Learning (NCIM), 1–6 (IEEE, 2023).
Zhang, S. & Metaxas, D. On the challenges and perspectives of foundation models for medical image analysis. Med. Image Anal. 91, 102996 (2024).
Google Scholar
Li, Z., Jiang, Y., Lu, M., Li, R. & Xia, Y. Survival prediction via hierarchical multimodal co-attention transformer: a computational histology-radiology solution. IEEE Trans. Med. Imaging 42, 2678–2689 (2023).
Google Scholar
Lin, C., Chen, Y., Feng, S. & Huang, M. A multibranch and multiscale neural network based on semantic perception for multimodal medical image fusion. Sci. Rep. 14, 17609 (2024).
Google Scholar
Liu, Y. et al. Brain tumor segmentation in multimodal mri via pixel-level and feature-level image fusion. Front. Neurosci. 16, 1000587 (2022).
Google Scholar
Zhang, X. et al. Development of a large-scale medical visual question-answering dataset. Commun. Med. 4, 277 (2024).
Google Scholar
Sheth, D. & Giger, M. L. Artificial intelligence in the interpretation of breast cancer on mri. J. Magn. Reson. Imaging 51, 1310–1324 (2020).
Google Scholar
Senthilkumar, K., Kalanandhini, G., Aravind, A., Vijayalakshmi, G. & Gayathri, J. Image fusion based on dtdwt to improve segmentation accuracy in tumour detection. In AIP Conference Proceedings, vol. 2393 (AIP Publishing, 2022).
Tang, X. et al. Preoperative prediction of axillary lymph node metastasis in patients with breast cancer through multimodal deep learning based on ultrasound and magnetic resonance imaging images. Acad. Radiol. 9, 100327 (2024).
Oberije, C. J. et al. Assessing artificial intelligence in breast screening with stratified results on 306 839 mammograms across geographic regions, age, breast density and ethnicity: a retrospective investigation evaluating screening (aries) study. BMJ Health Care Inform. 32, e101318 (2025).
Google Scholar
Zhuang, X. et al. Cardiac segmentation on late gadolinium enhancement MRI: a benchmark study from multi-sequence cardiac MR segmentation challenge. Med. Image Anal. 81, 102528 (2022).
Google Scholar
Chen, J., Li, H., Zhang, J. & Menze, B. Adversarial convolutional networks with weak domain-transfer for multi-sequence cardiac mr images segmentation. In Statistical Atlases and Computational Models of the Heart. Multi-Sequence CMR Segmentation, CRT-EPiggy and LV Full Quantification Challenges: 10th International Workshop, STACOM 2019, Held in Conjunction with MICCAI 2019, Shenzhen, China, October 13, 2019, Revised Selected Papers 10, 317–325 (Springer, 2020).
Hatamizadeh, A. et al. Swin unetr: Swin transformers for semantic segmentation of brain tumors in mri images. In International MICCAI brainlesion workshop, 272–284 (Springer, 2021).
Shamshad, F. et al. Transformers in medical imaging: a survey. Med. Image Anal. 88, 102802 (2023).
Google Scholar
Boehm, K. M., Khosravi, P., Vanguri, R., Gao, J. & Shah, S. P. Harnessing multimodal data integration to advance precision oncology. Nat. Rev. Cancer 22, 114–126 (2022).
Google Scholar
Xia, Y. et al. CT-based multimodal deep learning for non-invasive overall survival prediction in advanced hepatocellular carcinoma patients treated with immunotherapy. Insights into Imaging 15, 214 (2024).
Google Scholar
Qi, Y. et al. Applications of Raman spectroscopy in clinical medicine. Food Front. 5, 392–419 (2024).
Google Scholar
Zhou, X., Chen, C., Zuo, E., Chen, C. & Lv, X. Cross branch co-attention network multimodal models based on raman and ftir spectroscopy for diagnosis of multiple selected cancers. Appl. Soft Comput. 166, 112204 (2024).
Google Scholar
Hsu, W.-W. et al. A weakly supervised deep learning-based method for glioma subtype classification using WSI and MPMRIS. Sci. Rep. 12, 6111 (2022).
Google Scholar
Lin, D. et al. Autofluorescence and white light imaging-guided endoscopic raman and diffuse reflectance spectroscopy for in vivo nasopharyngeal cancer detection. J. Biophotonics 11, e201700251 (2018).
Google Scholar
Lin, J., Li, Y., Lin, X. & Che, C. Decision-level data fusion based on laser-induced breakdown and raman spectroscopy: a study of bimodal spectroscopy for diagnosis of lung cancer at different stages. Talanta 275, 126194 (2024).
Google Scholar
Janßen, C. et al. Multimodal lung cancer subtyping using deep learning neural networks on whole slide tissue images and maldi msi. Cancers 14, 6181 (2022).
Google Scholar
Wu, C. et al. Multimodal fusion with hyperspectral images and rgb information for all-optical informed virtual staining. Opt. Express 33, 19820–19836 (2025).
Google Scholar
Chen, Z. et al. Multimodal optoacoustic imaging: methods and contrast materials. Chem. Soc. Rev. 53, 6068–6099 (2024).
Kang, E. Y.-C. et al. A multimodal imaging–based deep learning model for detecting treatment-requiring retinal vascular diseases: model development and validation study. JMIR Med. Inform. 9, e28868 (2021).
Google Scholar
Chua, J. et al. Utilizing deep learning to predict alzheimer’s disease and mild cognitive impairment with optical coherence tomography. Alzheimer’s. Dement. Diagn. Assess. Dis. Monit. 17, e70041 (2025).
Zhao, T. et al. A foundation model for joint segmentation, detection and recognition of biomedical objects across nine modalities. Nat. Methods 22, 166–176 (2025).
Google Scholar
Doan, L. M. T., Verma, S., Eftekhari, N., Angione, C. & Occhipinti, A. From bulk to single-cell and spatial data: an AI framework to characterise breast cancer metabolic dysregulations across modalities. Computers Biol. Med. 198, 111195 (2025).
Google Scholar
Verma, S. et al. Cross-attention enables deep learning on limited omics-imaging-clinical data of 130 lung cancer patients. Cell Rep. Methods 4, 100817 (2024).
Qoku, A., Katsaouni, N., Flinner, N., Buettner, F. & Schulz, M. H. Multimodal analysis methods in predictive biomedicine. Comput. Struct. Biotechnol. J. 21, 5829–5838 (2023).
Wu, J. et al. Integrating transcriptomics, genomics, and imaging in alzheimer’s disease: a federated model. Front. Radiol. 1, 777030 (2022).
Google Scholar
Ning, C. et al. Genome-wide association analysis of left ventricular imaging-derived phenotypes identifies 72 risk loci and yields genetic insights into hypertrophic cardiomyopathy. Nat. Commun. 14, 7900 (2023).
Google Scholar
Gomes, B. et al. Genetic architecture of cardiac dynamic flow volumes. Nat. Genet. 56, 245–257 (2024).
Google Scholar
Yu, Y. et al. MRI-based multimodal AI model enables prediction of recurrence risk and adjuvant therapy in breast cancer. Pharmacol. Res. 216, 107765 (2025).
Zhang, S. et al. Tumor initiation and early tumorigenesis: molecular mechanisms and interventional targets. Signal Transduct. Target. Ther. 9, 149 (2024).
Google Scholar
Lu, M. Y. et al. A multimodal generative AI copilot for human pathology. Nature 634, 466–473 (2024).
Google Scholar
Moor, M. et al. Foundation models for generalist medical artificial intelligence. Nature 616, 259–265 (2023).
Google Scholar
Lu, M. Y. et al. A visual-language foundation model for computational pathology. Nat. Med. 30, 863–874 (2024).
Google Scholar
Chen, R. J. et al. Towards a general-purpose foundation model for computational pathology. Nat. Med. 30, 850–862 (2024).
Google Scholar
Rao, V. M. et al. Multimodal generative AI for medical image interpretation. Nature 639, 888–896 (2025).
Google Scholar
Pérez-García, F. et al. Exploring scalable medical image encoders beyond text supervision. Nat. Mach. Intell. 7, 119–130 (2025).
Lee, J.-O., Zhou, H.-Y., Berzin, T. M., Sodickson, D. K. & Rajpurkar, P. Multimodal generative ai for interpreting 3d medical images and videos. npj Digit. Med. 8, 1–8 (2025).
Google Scholar
Tu, T. et al. Towards generalist biomedical AI, 2023. arXiv: 2307.14334 (2023).
Wang, L. et al. Interpretable Bilingual Multimodal Large Language Model for Diverse Biomedical Tasks. In International Conference on Learning Representations. (eds. Yue, Y., Garg, A., Peng, N., Sha, F. & Yu, R.) Vol. 2025, 26054–26086 (2025).
Huang, X. et al. Towards a multimodal large language model with pixel-level insight for biomedicine. In Proc. AAAI Conference on Artificial Intelligence, 39, 3779–3787 (AAAI, 2025).
Krizhevsky, A., Sutskever, I. & Hinton, G. E. Imagenet classification with deep convolutional neural networks. Adv. Neural Inf. Process. Syst. 25, 84–90 (2012).
Li, Q. et al. Medical image classification with convolutional neural network. In Proc. 13th International Conference on Control Automation Robotics & Vision (ICARCV), 844–848 (IEEE, 2014).
Ronneberger, O., Fischer, P. & Brox, T. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer-assisted intervention, 234–241 (Springer, 2015).
Goodfellow, I. et al. Generative adversarial nets. Adv. Neural Inf. Process. Syst. 27, 2672–2680 (2014).
Sohl-Dickstein, J., Weiss, E., Maheswaranathan, N. & Ganguli, S. Deep unsupervised learning using nonequilibrium thermodynamics. In International conference on machine learning, 2256–2265 (PMLR, 2015).
Gillies, R. J., Kinahan, P. E. & Hricak, H. Radiomics: images are more than pictures, they are data. Radiology 278, 563–577 (2016).
Google Scholar
Gómez-de Mariscal, E. et al. Deep-learning-based segmentation of small extracellular vesicles in transmission electron microscopy images. Sci. Rep. 9, 13211 (2019).
Google Scholar
Vaswani, A. et al. Attention is all you need. Adv. Neural Inf. Process. Syst. 30, 1–11 (2017).
Li, Y. et al. Supervision Exists Everywhere: A Data Efficient Contrastive Language-Image Pre-training Paradigm. In International Conference on Learning Representations(2022).
AlSaad, R. et al. Multimodal large language models in health care: applications, challenges, and future outlook. J. Med. Internet Res. 26, e59505 (2024).
Google Scholar
Bradshaw, T. J. et al. Large language models and large multimodal models in medical imaging: A primer for physicians. J. Nucl. Med. 66, 173–182 (2025).
Google Scholar
Najjar, R. Redefining radiology: a review of artificial intelligence integration in medical imaging. Diagnostics 13, 2760 (2023).
Google Scholar
Schwabe, D., Becker, K., Seyferth, M., Klaß, A. & Schaeffter, T. The metric-framework for assessing data quality for trustworthy AI in medicine: a systematic review. npj Digit. Med. 7, 203 (2024).
Google Scholar
Karthik, A. et al. Comprehensive assessment of imaging quality of artificial intelligence-assisted compressed sensing-based MR images in routine clinical settings. BMC Med. Imaging 24, 284 (2024).
Google Scholar
Balachandar, N., Chang, K., Kalpathy-Cramer, J. & Rubin, D. L. Accounting for data variability in multi-institutional distributed deep learning for medical imaging. J. Am. Med. Inform. Assoc. 27, 700–708 (2020).
Google Scholar
Kondylakis, H. et al. Data infrastructures for ai in medical imaging: a report on the experiences of five eu projects. Eur. Radiol. Exp. 7, 20 (2023).
Google Scholar
Carrillo-Perez, F., Pizurica, M., Marchal, K. & Gevaert, O. Synthetic multimodal data modelling for data imputation. Nat. Biomed. Eng. 9, 421–425 (2025).
Google Scholar
Chen, B., Oliva, J. & Niethammer, M. A unified model for longitudinal multi-modal multi-view prediction with missingness. In International Conference on Medical Image Computing and Computer-Assisted Intervention, 410–420 (Springer, 2024).
Qi, L. et al. A unified missing modality imputation model with inter-modality contrastive and consistent learning. In International Conference on Medical Image Computing and Computer-Assisted Intervention, 44–53 (Springer, 2025).
Srinivasan, S. et al. Multimodal biomedical image segmentation using multi-dimensional u-convolutional neural network. BMC Med. Imaging 24, 38 (2024).
Google Scholar
Han, L. et al. All-in-one medical image-to-image translation. Cell Rep. Methods 5, 101138 (2025).
Schäfer, R. et al. Overcoming data scarcity in biomedical imaging with a foundational multi-task model. Nat. Comput. Sci. 4, 495–509 (2024).
Google Scholar
Pahud de Mortanges, A. et al. Orchestrating explainable artificial intelligence for multimodal and longitudinal data in medical imaging. npj Digit. Med. 7, 195 (2024).
Google Scholar
Ghasemi, A., Hashtarkhani, S., Schwartz, D. L. & Shaban-Nejad, A. Explainable artificial intelligence in breast cancer detection and risk prediction: a systematic scoping review. Cancer Innov. 3, e136 (2024).
Google Scholar
Rotem, O. & Zaritsky, A. Visual interpretability of bioimaging deep learning models. Nat. Methods 21, 1394–1397 (2024).
Google Scholar
Sangnark, S., Rattanachaisit, P., Patcharatrakul, T. & Vateekul, P. Explainable multi-modal deep learning with cross-modal attention for diagnosis of dyssynergic defecation using abdominal x-ray images and symptom questionnaire. IEEE Access 12, 78132–78147 (2024).
Occhipinti, A., Verma, S., Doan, L. M. T. & Angione, C. Mechanism-aware and multimodal ai: beyond model-agnostic interpretation. Trends Cell Biol. 34, 85–89 (2024).
Google Scholar
Shortliffe, E. H. & Chiang, M. F.Biomedical Data: Their Acquisition, Storage, and Use, 45–75 (Springer, 2021).
Nasir, S., Khan, R. A. & Bai, S. Ethical framework for harnessing the power of ai in healthcare and beyond. IEEE Access 12, 31014–31035 (2024).
Google Scholar
Berlinger, N. et al. Ethical framework for health care institutions responding to novel coronavirus SARS-COV-2 (COVID-19) guidelines for institutional ethics services responding to COVID-19. Hastings Cent. 12, 1–12 (2020).
Tejani, A. S., Ng, Y. S., Xi, Y. & Rayan, J. C. Understanding and mitigating bias in imaging artificial intelligence. RadioGraphics 44, e230067 (2024).
Google Scholar
Lekadir, K. et al. FUTURE-AI: international consensus guideline for trustworthy and deployable artificial intelligence in healthcare. BMJ 388 (2025).
Sun, Z. et al. A scoping review on multimodal deep learning in biomedical images and texts. J. Biomed. Inform. 146, 104482 (2023).
Walter, A. et al. Correlated multimodal imaging in life sciences: expanding the biomedical horizon. Front. Phys. 8, 47 (2020).
Google Scholar
Koetzier, L. R. et al. Generating synthetic data for medical imaging. Radiology 312, e232471 (2024).
Google Scholar
Pai, S. et al. Foundation model for cancer imaging biomarkers. Nat. Mach. Intell. 6, 354–367 (2024).
Google Scholar
Perez-Lopez, R., Ghaffari Laleh, N., Mahmood, F. & Kather, J. N. A guide to artificial intelligence for cancer researchers. Nat. Rev. Cancer 24, 427–441 (2024).
Google Scholar
Azad, B. et al. Foundational models in medical imaging: A comprehensive survey and future vision. arXiv preprint arXiv:2310.18689 (2023).
Artsi, Y., Sorin, V., Glicksberg, B. S., Nadkarni, G. N. & Klang, E. Advancing clinical practice: the potential of multimodal technology in modern medicine. J. Clin. Med. 13, 6246 (2024).
Google Scholar
Kaczmarczyk, R., Wilhelm, T. I., Martin, R. & Roos, J. Evaluating multimodal AI in medical diagnostics. npj Digit. Med. 7, 205 (2024).
Google Scholar
Zhang, S., Dai, G., Huang, T. & Chen, J. Multimodal large language models for bioimage analysis. Nat. Methods 21, 1390–1393 (2024).
Google Scholar
Network, C. G. A. et al. Comprehensive molecular characterization of human colon and rectal cancer. Nature 487, 330 (2012).
Google Scholar
