Optimized Hybrid Cascaded Approach for Accurate Oral Cancer Detection in Histopathology Images Using Deep CNNs

Abstract

Oral Squamous Cell Carcinoma (OSCC) is a prevalent and deadly form of oral cancer, responsible for approximately 3% of all cancer cases worldwide and over 330,000 deaths annually. Early detection is critical for effective treatment and improved outcomes. Traditional diagnosis through manual examination of histopathological images is time-consuming and subjective. Recent advances in AI-guided computer vision have shown promise, though many approaches require extensive computational resources and large datasets. This paper introduces a cascaded network that integrates deep learning and traditional machine learning techniques for detecting Oral Squamous Cell Carcinoma (OSCC) from histopathology images. The proposed framework employs a comprehensive approach that includes three feature selection strategies—Principal Component Analysis (PCA), Bacterial Foraging Optimization (BFO), and a nooptimization baseline—to assess the impact of feature refinement on classification performance. An exhaustive search strategy was used to evaluate five widely adopted Convolutional Neural Network (CNN) models for feature extraction. The extracted features were subsequently either optimized using PCA or BFO or left unoptimized to identify the most informative subsets. These feature sets were then fed into seven traditional machine learning classifiers to perform OSCC detection. The MobileNetV2-PCA-LR cascaded network demonstrated the best overall performance among the configurations evaluated. This model achieved near-perfect results with an accuracy, precision, recall, and F1-score of 99.70%, while also offering faster image processing times. The proposed cascaded framework thus delivers a balanced solution that combines efficiency, high accuracy, scalability, and robustness, making it a strong candidate for practical and automated OSCC screening in clinical environments.