In a convolutional neural network (CNN) classification model for diagnosing medical images, transparency and interpretability of the model’s behavior are crucial in addition to high classification accuracy, and it is highly important to explicitly demonstrate them. In this study, we constructed an interpretable CNN-based model for breast density classification using spectral information from mammograms. We evaluated whether the model’s prediction scores provided reliable probability values using a reliability diagram and visualized the basis for the final prediction. In constructing the classification model, we modified ResNet50 and introduced algorithms for extracting and inputting image spectra, visualizing network behavior, and quantifying prediction ambiguity. From the experimental results, our proposed model demonstrated not only high classification accuracy but also higher reliability and interpretability compared to the conventional CNN models that use pixel information from images. Furthermore, our proposed model can detect misclassified data and indicate explicit basis for prediction. The results demonstrated the effectiveness and usefulness of our proposed model from the perspective of credibility and transparency.
References
[1]
Breast Cancer Facts and Statistics. https://www.breastcancer.org/facts-statistics
[2]
Early Detection: Breast Health Awareness and Early Strategies. The Global Breast Cancer Initiative. https://www.paho.org/hq/dmdocuments/2016/KNOWLEDGE-SUMMARY---EARLY-DETECTION.pdf
[3]
Broeders, M., Moss, S., Nyström, L., et al. (2012) The Impact of Mammographic Screening on Breast Cancer Mortality in Europe: A Review of Observational Studies. Journal of Medical Screening, 19, 14-25. https://doi.org/10.1258/jms.2012.012078
[4]
Boyd, N.F., Lockwood, G.A., Martin, L.J., Knight, J.A., Jong, R.A., et al. (1999) Mammographic Densities and Risk of Breast Cancer among Subjects with a Family History of This Disease. Journal of the National Cancer Institute, 91, 1404-1408. https://doi.org/10.1093/jnci/91.16.1404
[5]
Ursin, G., Ma, H., Wu, A.H., Bernstein, L., Salane, M., Parisky, et al. (2003) Mammographic Density and Breast Cancer in Three Ethnic Groups. Cancer Epidemiology, Biomarkers & Prevention, 12, 332-338. https://pubmed.ncbi.nlm.nih.gov/12692108/
[6]
Boyd, N.F., Rommens, J.M., Vogt, K., Lee, V., Hopper, J.L., et al. (2005) Mammographic Breast Density as an Intermediate Phenotype for Breast Cancer. The Lancet Oncology, 6, 798-808. https://doi.org/10.1016/S1470-2045(05)70390-9
[7]
Breast Composition Is an Important Factor in Managing Breast-Cancer Risk. https://www.uclahealth.org/Workfiles/clinical_updates/oncology/16v3-11-BreastDensity.pdf
[8]
Dense Breasts: Answers to Commonly Asked Questions. https://www.cancer.gov/types/breast/breast-changes/dense-breasts
[9]
America College of Radiology (2019) ACR BI-RADS Atlas 5th Edition. https://www.acr.org/Clinical-Resources/Reporting-and-Data-Systems/Bi-RadsDeep
[10]
Bodewes, F.T.H., van Asselt, A.A., Dorrius, M.D., Greuter, M.J.W. and de Bock, G.H. (2022) Mammographic Breast Density and the Risk of Breast Cancer: A Systematic Review and Meta-Analysis. Breast, 66, 62-68. https://doi.org/10.1016/j.breast.2022.09.007
[11]
Mohamed, A.A., Berg, W.A., Peng, H., et al. (2018) A Deep Learning Method for Classifying Mammographic Breast Density Categories. Medical Physics, 45, 314-321. https://doi.org/10.1002/mp.12683
[12]
Berg, W.A., Campassi, C., Langenberg, P. and Sexton, M.J. (2000) Breast Imaging Reporting and Data System: Inter- and Intraobserver Variability in Feature Analysis and Final Assessment. American Journal of Roentgenology, 174, 1769-1777. https://doi.org/10.2214/ajr.174.6.1741769
[13]
Winkler, N.S., Raza, S., Mackesy, M. and Birdwell, R.L. (2015) Breast Density: Clinical Implications and Assessment Methods. RadioGraphics, 35, 316-324. https://doi.org/10.1148/rg.352140134
[14]
Chan, H.-P. and Helvie, M.A. (2019) Deep Learning for Mammographic Breast Density Assessment and Beyond. Radiology, 290, 59-60. https://doi.org/10.1148/radiol.2018182116
[15]
Lawrence, S., Giles, C.L., Tsoi, A.C. and Back, A.D. (1997) Face Recognition: A Convolutional Neural-Network Approach. IEEE Transactions on Neural Networks, 8, 98-113.
[16]
Pan, S.J. and Yang, Q. (2010) A Survey on Transfer Learning. IEEE Transactions on Knowledge and Data Engineering, 22, 1345-1359. https://doi.org/10.1109/TKDE.2009.191
[17]
Boyd, N.F., Guo, H., Martin, L.J., et al. (2007) Mammographic Density and the Risk and Detection of Breast Cancer. The New England Journal of Medicine, 56, 227-236. https://doi.org/10.1056/NEJMoa062790
[18]
McCormack, V.A. and dos Santos Silva, I. (2006) Breast Density and Parenchymal Patterns as Markers of Breast Cancer Risk: A Meta-Analysis. Cancer Epidemiology, Biomarkers & Prevention, 15, 1159-1169. https://doi.org/10.1158/1055-9965.EPI-06-0034
[19]
Mahmood, T., Li, J., Pei, Y., Akhtar, F., Rehman, M.U. and Wast, S.H. (2022) Breast Lesions Classifications of Mammographic Images Using a Deep Convolutional Neural Network-Based Approach. PLOS ONE, 17, 1-25. https://doi.org/10.1371/journal.pone.0263126.
[20]
Li, S., Wei, J., Chan,H.-P., Helvie, M.A., Roubidoux, M.A., et al. (2018) Computer-Aided Assessment of Breast Density: Comparison of Supervised Deep Learning and Feature-Based Statistical Learning. Physics in Medicine & Biology, 63, Article ID: 025005. https://doi.org/10.1088/1361-6560/aa9f87
[21]
Dağlarli, E. (2020) Explainable Artificial Intelligence (xAI) Approaches and Deep Meta-Learning Models. In: Aceves-Fernandez, M.A., Ed., Advances and Applications in Deep Learning, IntechOpen, London, 1-17. https://www.intechopen.com/chapters/72398 https://doi.org/10.5772/intechopen.92172
[22]
Singh, A., Sourya, S. and Lakshminarayanan, V. (2020) Explainable Deep Learning Models in Medical Image Analysis. Journal of Imaging, 6, 52. https://doi.org/10.3390/jimaging6060052
[23]
van der Velden, B.H.M., Kuijf, H.J., Gilhuijs, K.G.A. and Viergever, M.A. (2022) Explainable Artificial Intelligence (XAI) in Deep Learning-Based Medical Image Analysis. Medical Image Analysis, 79, Article ID: 102470. https://doi.org/10.1016/j.media.2022.102470
[24]
Brunese, L., Mercaldo, F., Reginelli, A. and Santone, A. (2020) Explainable Deep Learning for Pulmonary Disease and Coronavirus COVID-19 Detection from X-Rays. Computer Methods and Programs in Biomedicine, 196, Article ID: 105608. https://doi.org/10.1016/j.cmpb.2020.105608
[25]
Cheng, C.-T., Ho, T.-Y., Lee, T.-Y., Chang, C.-C., et al. (2019) Application of a Deep Learning Algorithm for Detection and Visualization of Hip Fractures on Plain Pelvic Radiographs. European Radiology, 29, 5469-5477. https://doi.org/10.1007/s00330-019-06167-y
[26]
Huff, D.T., Weisman, A.J. and Jeraj, R. (2021) Interpretation and Visualization Techniques for Deep Learning Models in Medical Imaging. Physics in Medicine and Biology, 66, 04TR01. https://doi.org/10.1088/1361-6560/abcd17
[27]
Lee, H., Yune, S., Mansouri, M., Kim, M., Tajmir, S.H., et al. (2019) An Explainable Deep-Learning Algorithm for the Detection of Acute Intracranial Haemorrhage from Small Datasets. Nature Biomedical Engineering, 3, 173-182. https://www.nature.com/articles/s41551-018-0324-9#citeas https://doi.org/10.1038/s41551-018-0324-9
[28]
Liao, W., Zou, B., Zhao, R., Chen, Y., He, Z. and Zhou, M. (2019) Clinical Interpretable Deep Learning Model for Glaucoma Diagnosis. IEEE Journal of Biomedical and Health Informatics, 24, 1405-1412. https://doi.org/10.1109/JBHI.2019.2949075
[29]
Lei, Y., Tian, Y., Shan, H., Zhang, J., Wang, G. and Kalra, M.K. (2020) Shape and Margin-Aware Lung Nodule Classification in Low-Dose CT Images via Soft Activation Mapping. Medical Image Analysis, 60, Article ID: 101628. https://doi.org/10.1016/j.media.2019.101628
[30]
Matsuyama, E., Takehara, M. and Tsai, D.-Y. (2020) Using a Wavelet-Based and Fine-Tuned Convolutional Neural Network for Classification of Breast Density in Mammographic Images. Open Journal of Medical Imaging, 10, 17-29. https://doi.org/10.4236/ojmi.2020.101002
[31]
Mohamed, A.A., Berg, W.A., Peng, H., et al. (2018) A Deep Learning Method for Classifying Mammographic Breast Density Categories. Medical Physics, 45, 314-321. https://doi.org/10.1002/mp.12683
[32]
Guo, C., Pleiss, G., Sun, Y. and Weinberger, K.Q. (2017) On Calibration of Modern Neural Networks. Proceedings of the 34th International Conference on Machine Learning, PMLR, Vol. 70, 1321-1330. https://arxiv.org/pdf/1706.04599.pdf
[33]
van der Maaten, L. and Hinton, G. (2008) Visualizing Data Using t-SNE. Journal of Machine Learning Research, 9, 2579-2605. http://www.jmlr.org/papers/v9/vandermaaten08a.html
[34]
Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D. and Batra, D. (2017) Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization. 2017 IEEE International Conference on Computer Vision, Venice, 22-29 October 2017, 618-626. https://doi.org/10.1109/ICCV.2017.74
[35]
He, K., Zhang, X., Ren, S. and Sun, J. (2016) Deep Residual Learning for Image Recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, 27-30 June 2016, 770-778. https://doi.org/10.1109/CVPR.2016.90
[36]
ImageNet. http://www.image-net.org
[37]
The Cancer Imaging Archive (TCIA). https://www.cancerimagingarchive.net
[38]
Matsuyama, E., Tsai, D.-Y., Lee, Y., et al. (2013) A Modified Undecimated Discrete Wavelet Transform Based Approach to Mammographic Image Denoising. Journal of Digital Imaging, 26, 748-758. https://doi.org/10.1007/s10278-012-9555-6 https://link.springer.com/article/10.1007/s10278-012-9555-6
[39]
Daubechies, I. (1992) Ten Lectures on Wavelets. The Society for Industrial and Applied Mathematics, Philadelphia, US Patent No. 821393. https://doi.org/10.1137/1.9781611970104
