A multi-scale CNN with atrous spatial pyramid pooling for enhanced chest-based disease detection

Introduction

Lung diseases, including pneumonia, lung cancer, and tuberculosis, are the leading causes of death globally. In 2019–2020, a virus called SARS-CoV-2 caused a widespread outbreak known as COVID-19, impacting numerous individuals worldwide. The primary organ affected by this virus was the lung, leading to difficulties in breathing for those infected. As of October 15, 2021, the total count of officially confirmed COVID-19 cases worldwide was 239.4 million (Zheng et al., 2022).

COVID-19 is a viral illness that first surfaced as an outbreak in China in December 2019 and escalated into a global pandemic. Common symptoms include a dry cough, headache, fever, loss of taste or smell, and shortness of breath; other symptoms can be found here (BMC, 2021). The coronavirus obtained its name due to its characteristic appearance, resembling a halo, when observed under an electron microscope. This virus is primarily transmitted through respiratory droplets and can lead to severe illnesses, including pneumonia and fatality. The viral genome enters host cells and has a higher incidence in males than females and adults than children. The mortality rates for children are estimated to range from 1% to 5%. The prevailing diagnostic method is the reverse transcriptase polymerase chain reaction (RT-PCR). Still, the test results can be time-consuming, and there may be limitations in the availability of test kits. In addition, test sensitivity is a concern due to potential laboratory and sample errors. Other methods, such as computed tomography (CT) and chest X-ray imaging, are also used but require specialist analysis of the images (Mousavi et al., 2022; Celik, 2023).

Pneumonia is a severe infection caused by viruses, bacteria, or other germs. It is a leading cause of death among children and the elderly worldwide. Chest X-rays serve as a primary diagnostic tool for pneumonia. However, interpreting these images can be time-consuming and less precise without the expertise of specialists.

The challenge is that other medical conditions, such as lung cancer or excess fluid, may exhibit similar anomalies on X-ray images. Consequently, there is an urgent need for an accurate and efficient method for the early identification of chest-related diseases (Jaiswal et al., 2019).

The most accurate method of identifying chest diseases is through chest X-ray images. However, reading and interpreting these images can be time-consuming and inaccurate without professional knowledge and experience. Therefore, there is a need for an accurate and efficient method to identify lung diseases using X-ray images. In recent years, the use of technology has drastically increased in many fields. From farming to launching space crafts, in almost every area, AI is widely used. The use of AI has also affected the healthcare sector with several advancements. It is being commonly used in the field of medicine (Hicks et al., 2022). The article (Radhi, 2023) reviews the application of and role of AI in dealing with and diagnosing diseases like COVID-19. With the help of AI, especially its applications like Image Processing, we can detect different abnormalities in X-rays. We can reduce the effort and time being put in by a specialist to diagnose a disease.

Researchers have proposed various computational algorithms and supporting diagnostic tools for analyzing X-ray images. However, these tools may need to provide more information to help doctors make decisions. Machine learning is a promising approach in this area, but traditional methods such as vector quantization and recurrent neural networks have limitations regarding accuracy and computation time (Mousavi et al., 2022; Celik, 2023; Jaiswal et al., 2019).

Given these limitations, researchers are exploring using artificial intelligence (AI) and deep learning (DL)-based methods for the early detection of COVID-19 and other related lung diseases. These methods can help specialists make quick and accurate diagnoses, speeding up the treatment process. Deep learning methods, in particular, have gained widespread popularity in this field because they eliminate the need for feature extraction during the preprocessing stage. Instead, they can learn the most relevant features directly from the used dataset.

Present-day deep learning techniques predominantly focus on convolutional neural network (CNNs) for chest-based disease detection. These models are well suited for medical image analysis because they can automatically learn hierarchical features directly from the images. This reduces the need for manual feature engineering. CNNs are designed to capture spatial hierarchies in chest X-ray images, where the earlier layers detect simple patterns while the more profound layers recognize complex structural abnormalities in the lung. This capability allows CNNs to exceed traditional machine learning techniques in accuracy and speed, facilitating rapid diagnosis and decision-making.

We present a hybrid CNN architecture with an atrous spatial pyramid The pooling layer captures fine and large-scale features, optimizing performance and accuracy. With this, we hope to provide a highly scalable and accurate architecture that can boost the diagnosis of chest-based diseases.

Related work

CNNs have been used extensively for chest-based disease detection, especially in diseases like COVID-19. These networks readily learn spatial hierarchies of features such as edges, textures, and complex patterns. CNNs deep layers can capture minor variations in lung texture and appearance, making them more suitable and accurate than conventional image analysis methods.

This article (Abdullah & Abdullah, 2024) provides a comprehensive overview of various deep-learning techniques for detecting COVID-19. These predominantly include CNNs, which is used with transfer learning and segmentation done on input images to extract the specific region of interest.

This article Tang et al. (2021) focuses on diagnosing COVID-19 by building the architecture on top of COVID-Net, a deep CNN tailored specifically for COVID-19 detection. The approach generates multiple model snapshots by using a cosine annealing learning rate. The models are then ensembled with a weighted averaging ensembling (WAE) method. This approach looks to assign more weightage to better-performing models for specific classes, resulting in a more robust architecture.

Another relevant article (Sheela et al., 2024) to efficiently predict lung-based diseases such as lung cancer by first preprocessing the images with a weighted average filter. Then, a region split and merge technique is used for grouping pixels with similar features, which works by segmenting the varying pixel values. The segmented images are then fed as input into a CNN, which extracts valuable features and labels the specific lung condition.

Although both approaches perform exceedingly well, their methods have areas for improvement, such as noise resilience, which the approach (Sheela et al., 2024) can struggle with. We aim to mitigate this by using an atrous spatial pyramid pooling (ASPP) to capture context at multiple scales, improving the model’s robustness to noise and artifacts. This, in turn, would handle limitations in preprocessing approaches like weighted-average filters.

Moreover, ASPP enables better detection across varying conditions and scales within a single architecture for lung-based disease detection, improving accuracy without complex ensemble setups or manual segmentation methods. Also, ASPP reduces the need for ensemble methods as used in Tang et al. (2021), offering better generalization with fewer models. This reduces the required computing power and improves model efficiency.

Materials, methodology, and preprocessing

This segment will explain the algorithm we used for COVID-19 detection and the dataset on which we trained our model.

Dataset

The dataset we gathered was from two unique sources. Duong et al. (2024) was used to collect 142 COVID-19 positive and 142 normal images whose images are of size $224\times 224$. Duong et al.’s (2024) dataset consists of the 284 posterior anterior chest X-ray images collected from Cohen, Morrison & Dao (2021), a renowned repository. The Pneumonia dataset was collected from the Kaggle repository (Mooney, 2022) whose images too are of size $224\times 224$ pixels. For the purpose of training our model we uploaded the Pneumonia dataset to the public drive link (Bukhari & Bukhari, 2024), Duong et al.’s (2024) dataset was used to train the model on the COVID-19 dataset.

An equal number of COVID-19 and normal chest X-ray images were used to address the class imbalance problem. We used 80% of our data (or 224 images) for training, while 20% of our data (or 60 images) were used for validation. The Kaggle dataset had a distribution of 1,739 pneumonia and 1,739 normal images, of which we randomly selected 294 pneumonia and 294 normal images to train and validate the model. The pneumonia branch of our architecture was trained on 234 pneumonia and normal images each. It was validated on 60 images each for both pneumonia and normal images.

While the dataset size is relatively modest, it was sourced from Cohen, Morrison & Dao (2021), a renowned repository, encompassing both COVID-19 positive and normal images. Figure 1 shows both a regular and COVID-19 positive lung side by side. Most studies against which we bench-marked our results too utilized a comparable dataset. These benchmark studies reveal that most incorporated datasets with fewer than 200 images. Even in cases where the dataset size exceeded 500 images, such as the study by Ozturk et al. (2020), which included 625 images, the distribution remained imbalanced with 125 COVID-positive and 500 normal images. The inherent imbalance in dataset distribution gives rise to a class imbalance issue, wherein the model exhibits bias, demonstrating superior recognition performance for the class with a larger dataset. Consequently, only the 125 normal images are effectively utilized in this context, rendering the remaining 375 normal images redundant. To rectify this class imbalance, achieving parity between the number of COVID-19 and normal images at 125 instances each becomes imperative.

For the Pneumonia dataset, we had 1,739 images each for both normal and COVID-19. Only 588, however, were selected for training and validation purposes. This was because we only trained and validated our approach on the posterior anterior view of the lung to decrease the overall training time.

Suggested method

Some earlier results from our approach have previously appeared in a conference article (Bukhari et al., 2024). The suggested approach for both COVID-19 and pneumonia detection is explained in Fig. 2. The model aims to output two probabilities of COVID-19 and pneumonia. This encompasses two crucial phases: preprocessing and classification using transfer learning over CNNs. After the pre-trained model, an ASPP layer is added, improving the architecture’s ability to capture features at multiple scales using varying dilation rates. A comprehensive account of each stage is provided in the following segments.

Preprocessing

In this segment, we will go through the two techniques we used for preprocessing. Preprocessing is a technique for improving the performance of a CNN, as discussed in this article (Lasek, 2021).

Preprocessing stage 1: normalization

Data normalization is commonly employed in CNN-based architectures to ensure numerical stability. By normalizing the data, the CNN model is expected to exhibit faster learning and more stable gradient descent (Swati et al., 2019). In this study, the pixel values of the input images were normalized to a range of 0–1. The dataset used consisted of grayscale images. The normalization was done by multiplying the pixel values by 1/255. The impacts and advantages of data normalization are discussed in the following article (Cabello-Solorzano et al., 2023).

Preprocessing stage 2: data augmentation

CNNs require large amounts of training data to learn and generalize well on new and unseen data, which was impossible with our available 284 images (for COVID-19). To address this issue, we employed data augmentation techniques that expanded the dataset size and enhanced our model’s performance on unseen data.

Data augmentation is essential for improving the performance and generalization of CNNs, especially when constricted by a small dataset size. By artificially increasing the diversity and size of training data through transformation techniques such as flipping, rotating, cropping, etc., data augmentation helps CNNs learn more features. Furthermore, data augmentation also prevents overfitting by exposing the model to different variants of the same images, enabling it to generalize much better on unseen data. Furthermore, data augmentation can help balance classes in imbalanced datasets, making the model more accurate and reliable. The following articles (Hernández-García & König, 2018; Rebuffi et al., 2021) particularly emphasize the benefits of data augmentation while using CNNs. They also highlight how data augmentation techniques can improve model robustness.

The following data augmentation techniques were used:

• 1.

  The images were subjected to a shear range of up to 0.2.

• 2.

  The images were subjected to a zoom range of up to 0.15.

• 3.

  The images were subjected to a horizontal flip.

• 4.

  The images were subjected to a width shift range of up to 0.2.

• 5.

  The images were subjected to a height shift range of up to 0.2.

• 6.

  The images were subjected to a rotation of up to 20 degrees.

• 7.

  The images were subjected to a channel shift range of up to 20.

• 8.

  The images were subjected to a brightness adjustment factor between 0.5 and 1.5.

These data augmentation techniques were chosen as they boost CNNs performance on chest-based X-rays according to this article (Han et al., 2021).

Ultimately, an extra training set comprising 5,536 images was acquired, constituting a multi-fold increase compared to the original training images. This is shown in Table 1.

Table 1:

Distribution of dataset before and after augmentation.

Class	Original dataset		Augmented dataset
	Train	Val	Train	Val
COVID-19	112	30	896	30
Pneumonia	234	60	1,872	60
Normal (Against COVID-19)	112	30	896	30
Normal (Against Pneumonia)	234	60	1,872	60
Total (For COVID-19)	224	60	1,792	60
Total (For Pneumonia)	468	120	3,744	120

DOI: 10.7717/peerj-cs.2686/table-1

Figure 3 shows each of the eight data augmentation techniques applied.

Figure 4 shows the effect of data augmentation on our original image.

Experiments and their findings

This segment comprehensively evaluates our approach by showcasing the results across various hyperparameters and pre-trained models.

Scaling our approach for multi-class detection

This segment will explain our multi-branch approach to scale our model to predict both COVID-19 and pneumonia while still using sigmoid as our activation function.

Implementing a multi-branch architecture in TensorFlow Functional API

The code in Fig. 5 demonstrates the merging of two branches in TensorFlow’s Functional API. Since both the inputs and the outputs are a Python list, we can have any number of branches. Also, the code in the branches can be different. The number of convolutional layers, the activation functions, and the training data can differ for all the branches. Further reading on TensorFlow’s functional API can be found in Keras (2025).

Final approach for both COVID-19 and pneumonia detection

The hyper-parameters we used in the architecture were the same for both branches, as a batch size of 32, an epoch size of 15, and an Adams optimizer with a learning rate of 0.001 were shown to perform optimally for our use case. Figure 6 shows chest X-rays of pneumonia and COVID-19. Notice the similarities; hence, we concluded that the same approach could perform well with the Pneumonia dataset.

Figure 7 shows an abstract explanation of our approach. We used two branches because we wanted our model to predict COVID-19 and pneumonia. Both branches have a pre-trained model at the top, followed by an atrous spatial pyramid pooling block, followed by multiple dense layers with progressively decreasing neurons. A sigmoid activation function then follows the dense layers. We preferred sigmoid over softmax as the multi-layer architecture converted our problem to a binary classification problem, and generally, sigmoid performs better than softmax for binary classification. The architecture outputs two probabilities. The first probability tells us whether the chest X-rays are COVID-19 positive. The second probability tells us whether or not the chest X-rays are pneumonia positive.

Performance evaluation using 5-fold cross validation for pneumonia and COVID-19 detection

In our research, five-fold cross-validation is paramount, particularly in evaluating machine learning models on COVID-19 and pneumonia datasets. Its importance and application are further discussed in the article (de Rooij & Weeda, 2020). Employing this technique allows us to systematically partition the datasets into five subsets, rotating each subset as the validation set while the remaining four serve as the training data. This approach ensures a comprehensive assessment of our model’s performance across different data splits, mitigating the impact of data variability and enhancing robustness. Given the potential heterogeneity and imbalances within COVID-19 and Pneumonia datasets, cross-validation provides a more accurate estimation of our model’s predictive capabilities. It enables us to evaluate the model’s ability to generalize to unseen instances of both COVID-19 and pneumonia cases, bolstering its reliability for practical deployment in medical settings. Moreover, by iterative training and evaluating the model on distinct data splits, we can effectively identify any biases or fluctuations in performance, facilitating informed model selection and optimization decisions. Thus, five-fold cross-validation is vital in our efforts to validate and fine-tune machine learning models for diagnosing and prognosis COVID-19 and pneumonia, ultimately advancing healthcare decision-making and patient care. Tables 5 and 6 show the detailed average results of our approach on the COVID-19 and Pneumonia datasets, respectively, after applying five-fold cross-validation. The results show that our approach achieved a validation accuracy of 98.66% on the COVID-19 dataset and a validation accuracy of 83.75% on the Pneumonia dataset. Figures 8 and 9 is the confusion matrix and ROC curve of the average 5-fold cross-validation on the COVID-19 dataset. Here, COVID-19 represents the positive class.

While Figs. 10 and 11 are the confusion matrix and ROC curve of the average five-fold cross-validation on the Pneumonia dataset. Here, pneumonia represents the positive class. The hyperparameters used are Adams as an optimizer, batch size 32, and learning rate 0.001.

A comparison of the results before and after performing data augmentation has been done in the Table 7. As can be seen, the results show an improvement in performance after data augmentation.

Dealing with overfitting

Overfitting in CNNs occur when the model crams a pattern specific to the training dataset rather than capturing generalized features. This normally occurs when the neural network has too many parameters relative to the size of the dataset, which causes the network to memorize the training examples instead of learning to generalize them. Overfitting is a major cause of concern as it leads to poor performance on raw/unseen data, which defeats the model’s ability to make reliable predictions.

We used the following techniques to prevent our model from overfitting.

1) A dropout layer was used, and a dropout of 0.3 to 0.5 was used, which drops 30% to 50% of the neurons in the previous layer; the organization of the dropout layers can be found in Fig. 7. A dropout layer prevents overfitting by randomly deactivating a subset of neurons. This forces the neural network to learn distributed representations rather than relying on some specific neurons, which in turn helps the model make better predictions on unseen data.

2) We implemented a rigorous five-fold cross-validation process to prevent overfitting. This method involves dividing the dataset into five equal parts, training the model in four parts, and validating it on the remaining part. This cycle is repeated five times, ensuring the model is thoroughly evaluated on all data portions. By exposing the model to different training and validation datasets, this practice enhances its ability to generalize, thereby reducing the risk of overfitting to any specific portion of the data.

3) Data augmentation was used, which reduces overfitting by artificially increasing the versatility of the training dataset; this, in turn, helps the model learn more generalized patterns rather than memorizing specific details from the original dataset. Transformation techniques like rotations, flips, zooms, and brightness adjustments create new, slightly changed versions of the existing data. This forces the model to recognize objects or patterns in different contexts, focusing on essential features rather than spurious noise or unnecessary details. By increasing the adequate size of the dataset, data augmentation reduces the probability of the model overfitting.

Ethical implications of artificial intelligence on medical diagnosis

The use of AI in medical diagnosis raises multiple ethical concerns that must be carefully addressed to ensure responsible and equitable healthcare. One of the primary concerns is bias in AI algorithms, which has the potential to arise if the training data does not belong to diverse populations. This could lead to inaccurate diagnoses for less-represented groups. This could worsen the existing healthcare inequalities. Transparency of AI decisions is crucial; doctors and patients need to have a high-level understanding of how AI systems arrive at their conclusions to trust the technology and ensure that it complements and does not replace human judgment.

Moreover, privacy and data security are significant concerns, as AI systems rely on large amounts of sensitive medical imaging data, necessitating robust measures to protect patient confidentiality. There is also the issue of accountability, which is about determining if an AI system makes an incorrect diagnosis that leads to harmful outcomes. These ethical challenges must be addressed through regulation, careful design, and collaboration between developers and healthcare professionals.

Scaling the approach for accurate predictions across diverse datasets

Our approach has been shown to achieve high validation accuracy. However, it can be challenging if the dataset we validate it on is highly different from the one we trained our model on. Even though we used data augmentation techniques, which should mitigate these concerns, they still have the potential to surface. To address these concerns, we propose a prototype website allowing users to train the model on a custom dataset. This prototype website, called “Chest Disease Detector,” allows users to train the model on custom Pneumonia and COVID-19 datasets. The user can then make predictions, and because they are trained on the local dataset, we expect the model to predict with high accuracy. Figure 12 visually represents how our prototype looks and behaves.

Conclusion

In this research, we aimed to introduce a highly efficient COVID-19 and pneumonia detector. Our findings demonstrated that our approach had achieved a validation accuracy of 97.5 Our approach achieved better results than other state-of-the-art methods in binary classification and outperformed them in multi-class classification. What further distinguishes our approach and renders it particularly suitable for adoption in the health sector is our incorporation of cross-validation, a crucial step to alleviate concerns regarding overfitting, a step omitted in other state-of-the-art approaches. Additionally, our approach boasts greater flexibility and scalability compared to its counterparts. We can tailor distinct methodologies to each branch, enhancing performance for specific diseases. Moreover, the multi-branch design facilitates the detection of current diseases more effectively and holds promise for identifying additional diseases in the future. Our approach is also highly scalable as we can train individual branches on local datasets to boost performance for a specific population.

Our approach performed much better on the COVID-19 dataset than the Pneumonia dataset. Modifying the second branch to tailor it more specifically to pneumonia can address this.

Furthermore, our approach has great potential for scalability. We can train and test the model on more branches by increasing the number of branches. Moreover, the uncoupled nature of the branches allows each branch to be asymmetrical. This is important, mainly because different diseases might require different architectures due to the varying nature of their features.

Supplemental Information