Evolution of Hybrid CNN-Transformer Architectures and Explainability Techniques Applied to Dermatological Diagnosis Using Deep Learning
DOI:
https://doi.org/10.55204/trc.v6i2.e725Keywords:
Deep Learning, CNN, Vision Transformer, hybrid architectures, dermatological diagnosis, explainability, XAI, Grad-CAM, HAM10000, melanomaAbstract
Skin cancer is the most frequently diagnosed malignant neoplasm worldwide, with mortality rates that increase dramatically when melanoma is detected at advanced stages. Early diagnosis assisted by artificial intelligence systems represents a critical strategy to reduce such mortality. This scientific research reviews and synthesizes the state of the art in hybrid CNN-Transformer architectures applied to dermatological diagnosis using Deep Learning, with special emphasis on explainability techniques (XAI). Thirty scientific articles were analyzed through the PRISMA protocol from high-impact academic databases (PubMed, IEEE Xplore, Scopus, Web of Science). Results reveal an evolution across three generations: classical CNNs (83–88% balanced accuracy), pure Vision Transformers (88–91%), and hybrid CNN-Transformer architectures (90–94%), the latter being the most active research frontier. A methodology is proposed for an EfficientNet-B4 + Swin Transformer ensemble model with integrated XAI module (Grad-CAM++ + SHAP) on the HAM10000 dataset, with an expected balanced accuracy exceeding 92% with high clinical interpretability.Downloads
References
Adebiyi, A., Abdalnabi, N., Hoffman Smith, E., Hirner, J., Simoes, E.J., Becevic, M., & Rao, P. (2024). Accurate Skin Lesion Classification Using Multimodal Learning on the HAM10000 and ISIC 2017 Datasets. medRxiv. https://doi.org/10.1101/2024.05.30.24308213
Ahmed, I., Bushon Routh, B., Rahman Kohinoor, Md.S., Sakib, S., Mahfuzur Rahman, M., & Azzedin, F. (2024). Multi-model attentional fusion ensemble for accurate skin cancer classification. IEEE Access, 12, 181009–181024. https://doi.org/10.1109/ACCESS.2024.3509053
Aladhadh, S., Alsanea, M., Aloraini, M., Khan, T., Habib, S., & Islam, M. (2022). An effective skin cancer classification mechanism via medical vision transformer. Sensors, 22(11), 4008. https://doi.org/10.3390/s22114008
Arnold, M., Singh, D., Laversanne, M., Vignat, J., Vaccarella, S., Meheus, F., & Bray, F. (2022). Global burden of cutaneous melanoma in 2020 and projections to 2040. JAMA Dermatology, 158(5), 495–503. https://doi.org/10.1001/jamadermatol.2022.0160
Arshed, M.A., Mumtaz, S., Ibrahim, M., Ahmed, S., Tahir, M., & Shafi, M. (2023). Multi-class skin cancer classification using vision transformer networks and convolutional neural network-based pre-trained models. Information, 14(7), 415. https://doi.org/10.3390/info14070415
Ayas, S. (2023). Multiclass skin lesion classification in dermoscopic images using swin transformer model. Neural Computing and Applications, 35, 6713–6722. https://doi.org/10.1007/s00521-022-08053-z
Bogrekci, I., Alhammadi, A., & Ozcelik, S. (2024). Enhancing Melanoma Diagnosis with Advanced Deep Learning Models Focusing on Vision Transformer, Swin Transformer, and ConvNeXt. Dermatopathology, 11(3), 26. https://doi.org/10.3390/dermatopathology11030026
Cassidy, B., Kendrick, C., Brodzicki, A., Jaworek-Korjakowska, J., & Yap, M.H. (2022). Analysis of the ISIC image datasets: Usage, benchmarks and recommendations. Medical Image Analysis, 75, 102305. https://doi.org/10.1016/j.media.2021.102305
Chanda, T., Das, S., Sinhamahapatra, P., Koller, S., Bertschinger, N., Atzori, M., & Weichenthal, M. (2024). Dermatologist-like explainable AI enhances trust and confidence in diagnosing melanoma. Nature Communications, 15(1), 524. https://doi.org/10.1038/s41467-023-44440-3
Combalia, M., Codella, N.C.F., Rotemberg, V., Helba, B., Vilaplana, V., Reiter, O., & Malvehy, J. (2019). BCN20000: Dermoscopic Lesions in the Wild. Scientific Data, 11(1), 641. https://doi.org/10.1038/s41597-024-03387-w
Dagnaw, G.H., El Mouhtadi, M., & Mustapha, M. (2024). Skin cancer classification using vision transformers and explainable artificial intelligence. Journal of Medical Artificial Intelligence, 7. https://doi.org/10.21037/jmai-24-29
Ding, Y., Yi, Z., Xiao, J., Hu, M., Guo, Y., Liao, Z., & Wang, Y. (2024). CTH-Net: A CNN and Transformer hybrid network for skin lesion segmentation. iScience, 27(4), 109442. https://doi.org/10.1016/j.isci.2024.109442
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., & Houlsby, N. (2021). An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. ICLR 2021. https://arxiv.org/abs/2010.11929
Esteva, A., Kuprel, B., Novoa, R.A., Ko, J., Swetter, S.M., Blau, H.M., & Thrun, S. (2017). Dermatologist-level classification of skin cancer with deep neural networks. Nature, 542(7639), 115–118. https://doi.org/10.1038/nature21056
Hauser, K., Kurz, A., Haggenmüller, S., Maron, R.C., von Kalle, C., Utikal, J.S., & Krieghoff-Henning, E. (2022). Explainable artificial intelligence in skin cancer recognition: a systematic review. European Journal of Cancer, 167, 54–69. https://doi.org/10.1016/j.ejca.2022.02.025
He, K., Chen, X., Xie, S., Li, Y., Dollár, P., & Girshick, R. (2022). Masked Autoencoders Are Scalable Vision Learners. IEEE/CVF CVPR, pp. 16000–16009. https://doi.org/10.1109/CVPR52688.2022.01553
He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. IEEE/CVF CVPR, pp. 770–778. https://doi.org/10.1109/CVPR.2016.90
Himel, G.M.S., Rahat, M.M.A., Hasan, Md.K., Ema, R.R., Uddin, M.N., Hossain, M.S., & Islam, M.K. (2024). Skin Cancer Segmentation and Classification Using Vision Transformer for Automatic Analysis in Dermatoscopy-Based Noninvasive Digital System. International Journal of Biomedical Imaging, 2024, 3861399. https://doi.org/10.1155/2024/3861399
Karthik, R., Menaka, R., Atre, S., Cho, J., & Easwaramoorthy, S.V. (2024). A hybrid deep learning approach for skin cancer classification using swin transformer and dense group shuffle non-local attention network. Expert Systems with Applications. https://doi.org/10.1016/j.eswa.2024.124578
Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., & Guo, B. (2021). Swin Transformer: Hierarchical Vision Transformer using Shifted Windows. IEEE/CVF ICCV, pp. 10012–10022. https://doi.org/10.1109/ICCV48922.2021.00986
Lundberg, S.M., & Lee, S.I. (2017). A Unified Approach to Interpreting Model Predictions. NeurIPS, 30, 4765–4774. https://arxiv.org/abs/1705.07874
Maron, R.C., Weichenthal, M., Utikal, J.S., Hekler, A., Berking, C., Hauschild, A., & Brinker, T.J. (2019). Systematic outperformance of 112 dermatologists in multiclass skin cancer image classification by convolutional neural networks. European Journal of Cancer, 119, 57–65. https://doi.org/10.1016/j.ejca.2019.06.013
Mohan, J., Sivasubramanian, A., Sowmya, V., & Ravi, V. (2025). Enhancing skin disease classification leveraging transformer-based deep learning architectures and explainable AI. Computers in Biology and Medicine, 190, 110007. https://doi.org/10.1016/j.compbiomed.2025.110007
Munjal, G., Bhatia, P., Srivastava, G., & Munjal, D. (2024). SkinSage XAI: An explainable deep learning solution for skin lesion diagnosis. Health Care Science, 3(6), 438–455. https://doi.org/10.1002/hcs2.121
Nie, Y., Sommella, P., Carratù, M., O'Nils, M., & Lundgren, J. (2023). A Deep CNN Transformer Hybrid Model for Skin Lesion Classification of Dermoscopic Images Using Focal Loss. Diagnostics, 13(1), 72. https://doi.org/10.3390/diagnostics13010072
Nunnari, F., Kadir, M.A., & Sonntag, D. (2021). On the overlap between Grad-CAM saliency maps and explainable visual features in skin cancer images. Lecture Notes in Computer Science, Springer, pp. 241–253. https://doi.org/10.1007/978-3-030-84060-0_16
Rezaee, K., & Zadeh, H.G. (2024). Self-attention transformer unit-based deep learning framework for skin lesions classification in smart healthcare. Discover Applied Sciences, 6, 3. https://doi.org/10.1007/s42452-024-05655-1
Ribeiro, M.T., Singh, S., & Guestrin, C. (2016). "Why Should I Trust You?": Explaining the Predictions of Any Classifier. KDD 2016, pp. 1135–1144. https://doi.org/10.1145/2939672.2939778
Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2020). Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization. International Journal of Computer Vision, 128(2), 336–359. https://doi.org/10.1007/s11263-019-01228-7
Tan, M., & Le, Q.V. (2019). EfficientNet: Rethinking model scaling for convolutional neural networks. ICML, 97, 6105–6114. https://arxiv.org/abs/1905.11946
Tschandl, P., Rosendahl, C., & Kittler, H. (2018). The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions. Scientific Data, 5, 180161. https://doi.org/10.1038/sdata.2018.161
Wu, Y., Chen, B., Zeng, A., Pan, D., Wang, R., & Zhao, S. (2022). Skin cancer classification with deep learning: a systematic review. Frontiers in Oncology, 12, 893972. https://doi.org/10.3389/fonc.2022.893972
Xin, C., Wen, H., Zhang, L., Xu, X., Sun, J., & Zhang, Y. (2022). An improved transformer network for skin cancer classification. Computers in Biology and Medicine, 149, 105939. https://doi.org/10.1016/j.compbiomed.2022.105939
Yousaf, N., Amin, J., Butt, W.H., et al. (2026). Advanced hybrid transformer CNN framework for improved skin lesion classification and segmentation. Scientific Reports. https://doi.org/10.1038/s41598-026-43376-0
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Jaime David Camacho Castillo, Andrés Steven Chichande Rodríguez, Juan Stalin Quezada Ruiz, Rosa Angélica Nuñez Vera, Erwin Stalin Caiño Cujilema, Edison Efrain Remache Gualacio
This work is licensed under a Creative Commons Attribution 4.0 International License.
The authors retain the moral and patrimonial rights of their works. They only give to the magazine Tesla Revista Científica the right to the first publication of this. Since Tesla Revista Científica is an open access publication, readers can fully or partially reproduce its content as long as they properly credit the corresponding authors and the journal itself. Tesla Revista Científica undertakes not to make commercial use of the texts it receives and/or publishes.
Our journal is governed by the international policies SHERPA/RoMEO: Green journal: They allow the self-archiving of both the pre-print (draft of a paper) and the post-print (the version corrected and reviewed by peers) and even the final version ( layout as it will be published in the journal).
See also "Copyright and licences".
