Multimodality Imaging Data-Driven Prediction Architecture for Breast Cancer

Breast cancer is a significant public health concern and one of the primary causes of cancer-related fatalities among women worldwide. Early identification and diagnosis of breast cancer are critical for successful treatment and enhanced patient outcomes. Deep learning algorithms have shown significant potential in identifying and diagnosing breast cancer in recent years. This study presents a method for classifying breast cancer into three categories: Normal, Benign, and Malignant, based on a combination of five transfer learning architectures. This paper proposes a stacked approach for progressively learning and combining multimodal features for the classification using Computer Tomography (CT) and ultrasound images (UT). The entire image is first converted into high-level features for each modality by building a deep 3D-CNN. To combine the most important attributes for image classification, the suggested stacking of VGG-19, ResNet-50, Inception-V3, Xception, DenseNet-121, and MobileNet-V2 models is proposed. The proposed stacking approach achieved an accuracy of 94.01%, 92.33% and 91.21% for Malignant, Benign and Normal. Results showed that compared to preexisting individual learners, the proposed ensemble model achieves higher accuracy both globally and inside individual classes.