Enhancing skin lesion classification: a CNN approach with human baseline comparison

Aggregation of expert opinions

The first research direction focuses on aggregating opinions, which proves beneficial when the ensemble members exhibit similar performance levels (Hosseinzadeh et al., 2024; Freeman et al., 2020; Kay et al., 2018; Lane et al., 2017; Shaikh & Khoja, 2012). This approach stands to enhance clinical performance but demands substantial resources, as a majority consensus requires soliciting input from multiple clinicians for each case (Wubineh, Deriba & Woldeyohannis, 2024; Hussain et al., 2023; Bejnordi et al., 2017).

CNNs vs. human experts

The second research direction asserts that CNNs demonstrate comparable or superior proficiency to humans in categorizing skin lesion malignancies (Navarrete-Dechent, Liopyris & Marchetti, 2020). However, further inquiry remains imperative in this domain as current algorithms have yet to reach a level where they can replace human classification (Chatterjee, Gil & Byun, 2024; Esteva et al., 2017). Consequently, investigating hybrid methodologies represents a logical progression in research (Hasan et al., 2022). Various hybrid approaches have been presented, demonstrating superior performance compared to individual human or machine capabilities (Bozkurt, 2023; Goceri, 2020). Despite the potential of hybrid intelligence to save lives (Brinker et al., 2019; Liu et al., 2020; Tschandl et al., 2019; Ketkar & Santana, 2017), a critical gap in the literature pertains to the expenses associated with such methods (Wubineh, Deriba & Woldeyohannis, 2024; Chollet & Chollet, 2021), which can make even highly effective approaches impractical for real-world implementation (Topol, 2019). For instance, the study by Hekler et al. (2019) involved the use of multiple clinicians to assess a single image, which would be infeasible in clinical practice.

Hybrid methodologies and the challenges of AI

Similar findings were observed by Houssein et al. (2024), Koçak et al. (2024), Secinaro et al. (2021), Tschandl et al. (2020), who highlighted that less experienced clinicians benefited the most from computer predictions. However, they cautioned that faulty AI could mislead clinicians across all experience levels, posing a threat to the usability of hybrid approaches (Kaluarachchi, Reis & Nanayakkara, 2021; Han et al., 2018; Chollet & Chollet, 2021). Blindly trusting AI decisions could undermine the superior performance achieved through human-machine collaboration, as demonstrated in these studies (Brinker et al., 2018; Goodfellow, 2016; Shaikh & Khoja, 2011). For example, Ha, Liu & Liu (2020), Marchetti et al. (2020) pursued a systematic approach where human participants rated their confidence in each image. By supplementing low-confidence ratings with machine-generated predictions, performance improved, particularly for medical residents, but less so for professional dermatologists. Although this method mitigates the risk of over-reliance on CNN predictions, it remains susceptible to flawed AI (Gómez-Carmona et al., 2024; Krakowski et al., 2024; Keerthana, Venugopal & Nath, 2023).

Ensemble approaches for skin lesion classification

The third research direction is the use of ensemble approaches (Aboulmira et al., 2025; Hekler et al., 2019). By aggregating predictions using a boosting algorithm, Gulli & Pal (2017) found that combining human and computer decisions yielded the best results for multiclass and binary lesion classification. This ensemble approach prevents human reliance solely on machine-generated results and appears to be a more feasible real-world implementation (Yang et al., 2024; Reddy et al., 2024). However, comprehensive comparisons of these approaches are still lacking (Ali et al., 2023). In the examination of existing literature (De, Mishra & Chang, 2024; Kumar et al., 2024; Tschandl et al., 2020; Shaikh & Khoja, 2014), the majority of studies focused on skin lesion classification relied on the utilization of one or more pre-trained networks as a foundational framework for their models.

In Table 1 we present a well-rounded comparative analysis of recent advancements, hybrid models, and interpretability-focused essential and recent studies in skin lesion classification literature that align closely with the objectives and key themes of this study.

Data preprocessing

During the research, preparing the data was crucial to train CNN models efficiently (Pérez & Ventura, 2022; Marchetti et al., 2020). These initial steps were necessary to get the dataset ready, for training and evaluation guaranteeing that the CNNs could learn well and provide predictions (Koçak et al., 2024; Hekler et al., 2020; Brinker et al., 2019).

All images were uniformly downsized to a resolution of 256 × 384 pixels. This decision balanced the need for detail preservation and computational efficiency, as higher resolutions would increase training time without guaranteeing better performance (Aboulmira et al., 2025; Ali et al., 2021). To enhance the training dataset, an ImageDataGenerator was used to perform various augmentation techniques such as rotation, zoom, and horizontal flipping (Bozkurt, 2023; Shaikh et al., 2021b; Goceri, 2020). This approach helped in creating a more robust model by exposing it to a variety of image transformations, thereby improving generalization (Hekler et al., 2020; Han et al., 2018).

During the training and validation phases, we opted for a batch size of 64 to ensure a mix of classes, in each batch which is important for addressing class imbalances (Aboulmira et al., 2025). When it came to testing, we used a batch size of 1 to maintain the image order and ensure alignment between predictions and actual data (Shaikh et al., 2024; Shaikh & Khoja, 2014). The images were normalized to standardize the input data aiding in speeding up the CNNs learning process by ensuring that the data distribution has an average of zero and a standard deviation of one (Chollet & Chollet, 2021; Brinker et al., 2019).

Justification of model types and selection method

Computing infrastructure

We required enormous computing power to conduct this research because of the nature and large scale of the deep learning models and datasets utilized (NVIDIA Corporation, 2020).

The research made use of Linux-based operating systems for their reliability and efficiency (Gulshan et al., 2016), in handling tasks and compatibility with various deep learning frameworks (Khalil et al., 2023). Training learning models, such as CNNs like EfficientNet demand GPU power (Jouppi et al., 2017). The study employed high-performance NVIDIA GPUs optimized for learning tasks (Kang & Tian, 2018; NVIDIA Corporation, 2020). While the research considered solutions utilizing TPUs (Tensor Processing Units) known for their efficiency in tensor operations in networks, the practical implementation primarily depended on GPU resources due to their availability and infrastructure limitations (Jouppi et al., 2017; Abadi et al., 2016; Nair & Hinton, 2010). Multi-core CPUs played a role in preprocessing data and overseeing workflow management.

Handling large datasets such as BCN20000, ISIC-2019, and ISIC-2020 requires high-capacity storage solutions (Ali et al., 2022; Nguyen et al., 2019; ISIC, 2024). Fast SSDs were used to ensure quick data access and processing speeds (Aboulmira et al., 2025; Kumar et al., 2024). Training complex models and processing large datasets necessitated large amounts of RAM to handle data efficiently during training phases.

The main tool for developing and training models was TensorFlow (Jang, 2025; Pang, Nijkamp & Wu, 2020; Dillon et al., 2017; Abadi et al., 2016). This framework is widely supported on Linux (Abadi et al., 2016; Nair & Hinton, 2010). It provides a range of tools for creating and improving neural networks. Kaggle Notebooks were used for model testing (Mostafavi Ghahfarokhi et al., 2024; Mukhlif et al., 2024). To benefit from community-shared solutions, these notebooks offered an adaptable environment for conducting experiments with computing requirements. Various personalized scripts—for processing, enhancing, and assessing the data—were created to customize the workflows according to the study’s needs (Akhund et al., 2024b; Banachewicz & Massaron, 2022; Abadi et al., 2016).

The computing setup relied on GPUs, ample memory, spacious storage, and reliable deep-learning software, on a Linux system to support smooth and productive model training and assessment procedures (Haggenmüller et al., 2021; Nair & Hinton, 2010).

Evaluation data

Training data

A substantial volume of high-quality images is crucial to train a CNN effectively (Ali et al., 2022; Rotemberg et al., 2021). For instance, the ImageNet Challenge used 1.2 million images across 1,000 categories, averaging 1,200 images per category (Hekler et al., 2019; Gutman et al., 2016; Deng et al., 2009). However, gathering and verifying these images is labor-intensive, resulting in limited datasets for computer vision tasks (Tan et al., 2024). Exploring skin lesion datasets, the PH2 dataset provided only 200 images, insufficient for CNN training (Gouda et al., 2022). The SIIM-ISIC challenges between 2016 and 2020 saw a significant expansion (ISIC, 2024) as shown in Table 2.

The ISIC-2020 dataset included 33,126 images (ISIC, 2024; Rotemberg et al., 2021), curated from various sources after thorough quality checks. Despite the volume, the ISIC-2020 dataset had a low positive incidence rate of 1.76%, potentially causing imbalances in training. To address this, merging the BCN20000 dataset with a 17.85% incidence rate was necessary, providing more positive instances for better model learning. Both datasets were used for this reason. The inclusion of metadata (patient age, sex, lesion site) in skin lesion classification models has shown promise. Studies suggest that incorporating such data enhances diagnostic accuracy for clinicians (Reddy et al., 2024; Haggenmüller et al., 2021). However, some winning Kaggle models (Lee et al., 2025; Ha, Liu & Liu, 2020; Lopez et al., 2017) did not benefit from metadata fusion, opting for different strategies. Haenssle et al. (2018) also highlight the potential of metadata but suggest varying benefits depending on the model structure (Naseri & Safaei, 2025; Codella et al., 2018). Aligning labels for both datasets, as shown in Table 3, involved treating the “MEL” class as positive and others as negative. Yet, using multiclass labels poses a risk: it might improve performance but limit overall utility (Shen et al., 2019; Deng et al., 2009). To counter this, we experimented with training models using both binary and multiclass labels.

Train-test split

Neural networks, due to their high capacity, are prone to overfitting, making it unsuitable to evaluate them on the same data used for training (Abbas et al., 2025; Ophir et al., 1991). Standard practice involves splitting the data into three sets: training, validation, and test sets. The training set is used to optimize the model’s performance, but overfitting can occur if the model memorizes the training data (Alotaibi & AlSaeed, 2025; Salman & Liu, 2019). A validation set, derived from the training set, helps identify overfitting by monitoring the loss on both training and validation sets after each epoch (Liu et al., 2025; LeCun et al., 1998). Overfitting is detected when training loss decreases while validation loss increases (Srivastava et al., 2014). The validation set also determines when to stop training to avoid further overfitting (Debelee, 2023). However, models optimized excessively on the validation set risk “overfitting to the validation set”, where some models perform better by chance rather than reflecting the true model. To mitigate this, a test set—a separate subset—is used to assess the final model’s ability to generalize new, unseen data. After optimizing the validation set, the model’s performance on the test set ensures its suitability for predicting novel images (Aboulmira et al., 2025; Tan & Le, 2019).

In this study, we employed a 15% validation split (Liu et al., 2024, 2020; Chollet & Chollet, 2021). Due to differing incidence rates in the ISIC-2019 and ISIC-2020 datasets (Hernández-Pérez et al., 2024; ISIC, 2024; Rotemberg et al., 2021), a stratified train-validation split was implemented, using an 85-15 division for each dataset before combining them (Naseri & Safaei, 2025; Ali et al., 2022; Deotte, 2020). Table 4 shows the class distribution for these splits. The test set provided by the 2020 Kaggle competition was used directly, omitting the need for a custom test set. This test set of the ISIC-2020 dataset comprises 10,982 images without ground truth, preventing model-specific tuning (ISIC, 2024). Instead, Kaggle accepts model predictions for scoring, facilitating performance comparisons with competition participants (ISIC, 2024; Ha, Liu & Liu, 2020).

Regularization

Deep neural networks, specifically Deep CNNs, exhibit extensive model capacity, making them susceptible to overfitting. Techniques employed to counter overfitting are known as regularization methods and encompass various approaches.

In this study, four key regularization techniques—data augmentation, dropout layers, capacity regulation, and weight regularization—are optimized following prior research (Srivastava et al., 2014; Argenziano et al., 2003). Chollet & Chollet (2021) suggests exploring model capacity until overfitting appears, followed by applying regularization methods to improve test performance. This process is iterative, time-consuming, and requires expertise from the data scientist (Hekler et al., 2019). Data augmentation prevents overfitting by altering input data, ensuring the model learns general features rather than specific images. This technique is crucial for small datasets and also improves generalization in larger datasets (Goodfellow, 2016). Rotation, zooming, and horizontal flipping are used in this study to introduce variations, helping the model handle novel orientations (Hekler et al., 2019; Zalaudek et al., 2010). Although shear is a common augmentation strategy, Zalaudek et al. (2006) highlight its potential to distort asymmetrical features, crucial for detecting malignant melanomas, which led to its exclusion. Based on prior literature observations (Hernández-Pérez et al., 2024; Chollet & Chollet, 2021; Goodfellow, 2016; Srivastava et al., 2014), data augmentation remains a vital part of regularization in all models.

Dropout layers, situated between hidden layers in a neural network, employ a binary mask to deactivate part of the activation, compelling subsequent layers to operate with incomplete information (Hernández-Pérez et al., 2024; Srivastava et al., 2014). This technique effectively converts the model into a correlated ensemble without multiple model instances (Hekler et al., 2019). Regulating the capacity of a neural network involves adjusting its depth and width, impacting the number of parameters. EfficientNets were developed by striking a balance between these dimensions, aiming for an optimal structure (Mukhlif et al., 2024; Haggenmüller et al., 2021; Tan & Le, 2019; Dillon et al., 2017; Goodfellow, 2016). Chollet & Chollet (2021) suggest an approach involving the deliberate construction of a complex network, followed by applying regularization techniques to counter overfitting. However, due to the substantial complexity of datasets and models, an alternative strategy of concurrently training multiple models with varied capacities and regularization methods is employed in this study. Weight decay, a regularization technique imposing a penalty function on complexity, reduces the network’s tendency to overfit (Houssein et al., 2024; Loshchilov & Hutter, 2016). $L^2$ weight decay, a common form, adjusts the relative contribution of the norm penalty function through a weight parameter. Finding an optimal alpha value (determining the degree of regularization) necessitates experimentation tailored to the specific situation (Sterkenburg, 2025; Hastie, Tibshirani & Friedman, 2009). Initial attempts to implement $L^2$ weight decay into the network using specific values encountered technical challenges, causing conflicts within TensorFlow and subsequent crashes (Jang, 2025; Pang, Nijkamp & Wu, 2020; Hekler et al., 2019; Hastie, Tibshirani & Friedman, 2009). As a result, prioritizing other aspects of the study overcame these obstacles.

Metrics

When assessing the performance of a model in a classification task, a frequently employed and straightforward metric is prediction accuracy (Mohammed & Meira, 2020; Davis & Goadrich, 2006). This metric, as defined by Mohammed & Meira (2020), serves as a common evaluation criterion:

(1) $$Accuracy={\displaystyle \frac{1}{n}\sum _{i=1}^{n}I(y_i=\widehat{y_i}).}$$

In a scenario with ‘n’ observations, accuracy measures how well a model’s estimates match the actual answers. Yet, accuracy can fall short for imbalanced datasets, where it may inflate due to a majority class bias, rendering them less useful (Liaw et al., 2025; He & Garcia, 2009). To counter this, the receiver operating characteristic curve (ROC)/area under curve (AUC) analysis proves valuable, unaffected by such imbalances (Ozel et al., 2025; Ha, Liu & Liu, 2020; Esteva et al., 2019; Fawcett, 2006). Additionally, the connection between CNN outputs and ROC thresholds has made ROC analysis prevalent in evaluating skin lesion CNNs. ROC analysis hinges on the true positive rate (TPR) and false positive rate (FPR), explained through confusion matrices (Gouda et al., 2022; Yap, Yolland & Tschandl, 2018). In a binary classification scenario with ‘n’ observations, ‘ $y_i$’ represents true labels, and ‘ $\widehat{y_i}$’ signifies estimated labels for each observation indexed from 1 to ‘n’, distinguishing positive (‘c₁’) and negative (‘c₂’) classes (Hernández-Pérez et al., 2024; Hekler et al., 2019; Kassem et al., 2021).

Hence, in line with commonly used metrics in skin lesion classification research, as demonstrated in Houssein et al. (2024), Ali et al. (2022), Yap, Yolland & Tschandl (2018), Esteva et al. (2019), we employ AUC-ROC, TPR, and FPR for performance evaluation.

Preliminary training approach

The primary objective was to train a CNN using the EfficientNetB3 architecture (Alhichri et al., 2021), with a Flattening layer followed by three Dense layers. The goal was to explore a large hyperparameter space that extended beyond the scope outlined in “Empirical study”. A full search, excluding learning rate tuning, would require training 1,024 models, which was impractical. To address this, the Keras tuner package Hyperband (Li et al., 2018) was used, applying a multi-armed bandit strategy to systematically evaluate models within the hyperparameter space (Ali et al., 2022). Hyperband’s heuristic approach creates a subset of models, trains them briefly for a few epochs, saves the models, and discards inferior performers (Alhichri et al., 2021; Hekler et al., 2019). The top-performing models are iteratively refined through further training, allowing resources to focus on the most promising candidates (Chen et al., 2024; Li et al., 2018). A preliminary test with Hyperband used 320 training images (less than 1% of the total 50,000) and 160 validation images (192 × 158 pixels). Training for 15 epochs took approximately 7 h, during which 90 models were evaluated, identifying the best-performing one. This model’s base was unfrozen and fine-tuned with 10 additional epochs. As shown in Table 5, the final model had more parameters in the Dense layers, driven by the Flattening layer’s large output size.

However, four issues emerged during the evaluation:

• 1)

  Image dimension error: Images were incorrectly encoded as (192,158) instead of (128,192), affecting performance (Goodfellow, 2016).

• 2)

  Runtime limitations: Testing on a cloud setup led to suboptimal node utilization, with training marked by performance spikes and inefficiencies. Enlarging the images to (512,768) increased the epoch runtime by 12×, rendering a full-scale test infeasible (Chollet & Chollet, 2021).

• 3)

  Dataset inconsistency: Initially, only the HAM10000 subset of the 2019 dataset was downloaded, encoding eight classes instead of nine. The complete dataset was later obtained and re-encoded to meet the required specifications (Tschandl et al., 2019).

• 4)

  Prediction imbalance: Despite achieving low loss values, the model’s predictions overestimated prevalent classes while assigning low probabilities to less common ones, limiting its generalization (Bria, Marrocco & Tortorella, 2020).

Optimizing the training strategy

The initial strategy faced obstacles that required significant time and effort to resolve. An analysis of the winning solution from the 2019 and 2020 Kaggle competition (GitHub, 2024a, 2024b; Lin et al., 2017) revealed available code on GitHub (GitHub, 2024a, 2024b), but the training process demanded extensive computational resources, exceeding our infrastructure. Additional exploration uncovered contributions from a Kaggle grandmaster, who utilized TPUs—resources not available to us (Banachewicz & Massaron, 2022). Another submission by Jang (2025) provided valuable insights but also relied on TPUs, requiring image rescaling to a 1:1 ratio. We evaluated this model after resizing images to (256,256), achieving an AUC score of 0.773. Through iterative reviews and small-scale testing, we identified promising adjustments that improved training by focusing on image features rather than class distributions. Key observations included:

• Implementation of GlobalAveragePooling instead of Flattening layers between the base and head, substantially enhancing performance while reducing the parameters in the initial dense layer (Ali et al., 2022; Minderer et al., 2022).

• Experimentation with binary-labeled training sets, inspired by Tensorflow (Jang, 2025; Pang, Nijkamp & Wu, 2020; Dillon et al., 2017), demonstrated quick and satisfactory results despite potential information loss compared to multilabel training (Houssein et al., 2024; Bria, Marrocco & Tortorella, 2020).

• Adoption of a learning rate schedule aligned with Chollet & Chollet’s (2021) recommendation, diverging from the reviewed models’ approaches, showcased substantial performance enhancements (Ha, Liu & Liu, 2020; Loshchilov & Hutter, 2016).

• Construction of models with shallow heads akin to Tensorflow (Jang, 2025; Pang, Nijkamp & Wu, 2020; Dillon et al., 2017) designs resulted in significantly fewer parameters than conventionally proposed by Chollet & Chollet (2021).

• Integration of class weights into the optimizer, following the Tensorflow approach (Pang, Nijkamp & Wu, 2020; Dillon et al., 2017), counteracted skewed class distributions, notably improving model predictions (Lin et al., 2017).

The collective impact of these methodologies substantially enhanced our models’ capability to discern image features rather than relying solely on class distribution information (Hernández-Pérez et al., 2024). Consolidating these strategies, a new model is crafted, expected to surpass the evaluation set performance achieved by the Tensorflow model (Pang, Nijkamp & Wu, 2020; Dillon et al., 2017).

Final model training and evaluation

The final configuration focused on three key parameters: label encoding (binary/multiclass), model head capacity (shallow/deep), and dropout layer strength (0/0.2/0.4/0.6). Initially, we attempted to use two dropout sizes (0/0.5) and an L2 kernel regularizer in the first Dense layer. However, this setup caused compatibility issues with EfficientNetB3, leading to recurring errors. As L2 regularization was absent in similar models, we shifted focus to exploring more dropout configurations. A total of 16 models were trained with varying parameter combinations.

To balance convergence and training time, we set the maximum epoch limit to 50 with an early stopping callback (patience = 7). The best model, based on validation loss, was retained. Early stopping minimized computational overhead by halting training when no improvement was observed over seven consecutive epochs (Caruana & Niculescu-Mizil, 2006).

While EfficientNet models can handle higher resolutions, they increase computational demands without guaranteed performance improvements (Tan & Le, 2019). Thus, we downscaled all images to (256,384) for training, validation, testing, and evaluation. Initial trials showed each epoch took just over two hours. Following prior studies (Ali et al., 2022; Shaikh & Khoja, 2013), we estimated 35–40 h of total training time per model over 15 epochs.

To speed up training, we ran models in parallel, grouped by target label and model capacity. Four separate scripts were developed, each using one of the four dropout configurations. These scripts applied identical preprocessing steps with an ImageDataGenerator to resize and augment images (Chollet & Chollet, 2021). Training batches were set to 64 to ensure the representation of smaller categories, while test batches used a size of 1 to maintain image order alignment with predictions. Each script’s total execution time ranged between 8 and 12 days.

Augmented hybrid approach

In certain instances, image classification poses varying levels of difficulty, presenting disparities between human and machine capacities (Kaluarachchi, Reis & Nanayakkara, 2021). The study by Han et al. (2018) demonstrated discrepancies in image interpretation, where image challenges for human subjects exhibited notably high performance when processed by algorithms. Similarly, Marchetti et al. (2020) revealed enhancements in performance by replacing uncertain human responses with predictions generated by a CNN. Defining a mutual relationship in the difficulty of image interpretation—where some images present challenges for humans but not for machines, while others are easily interpreted by humans yet pose complexity for computational systems (Archana & Jeevaraj, 2024). Moreover, it proposes leveraging CNN prediction values as indicators to delineate images deemed “easy” or “difficult” by the system (Sun et al., 2023). If substantiated, this insight could facilitate the construction of a synthesized list comprising both human and computer predictions, derived from the certainties inherent in CNN predictions (Jackson et al., 2025).

Testing differences between humans and CNNs

The hybrid algorithm operates under the premise that humans and CNNs exhibit varying proficiency in analyzing distinct images, with their performances showing minimal correlation (Bozkurt, 2022). A strong correlation exists between their performances, substituting one entity’s prediction with the other should yield negligible or no impact. To validate this presumption, the CNN predictions were segregated into inner and outer lists following Eqs. (5) and (7). A value of s = 4 was employed to maintain equal list sizes (Wang, Wong & Lu, 2020; Fawcett, 2006). Both lists underwent evaluation by both the baseline CNN model and human evaluators. The outcomes, depicted in Fig. 3, highlight differential ease in identifying certain images. The CNN’s performance appears least optimal for the inner list, registering an AUC of 0.664, while demonstrating superior performance on the outer list, yielding an AUC of 0.917. Similarly, human evaluators displayed lower performance on the inner list and higher performance on the outer list. Notably, their TPR exhibited variation between the lists, whereas the FPR remained relatively consistent. This substantiates the assertion that certain images pose greater classification challenges while suggesting the feasibility of utilizing CNN predictions to specifically target these challenging images (Müller et al., 2024). For the inner list, CNN’s performance significantly trails behind human performance. Conversely, on the outer list, CNN’s performance marginally surpasses human performance (Mahmood et al., 2024). These findings corroborate the notion that humans and CNNs encounter difficulties in analyzing dissimilar images, thereby supporting the argument against a strong correlation between their performances (Mahmood et al., 2024).

Augmented hybrid results

After demonstrating variations in image difficulty for classification and the lack of a strong correlation between human and CNN performance, the subsequent phase involves ensembling human and CNN responses into a unified list (Ganaie et al., 2022; Wu et al., 2022). It is postulated that an arrangement where human answers constitute the inner elements and CNN predictions form the outer elements will outperform the baseline performances (Akram et al., 2025; Archana & Jeevaraj, 2024). To evaluate this hypothesis, Algorithm 1 is executed twice: once with the inner list as the “Subset index list” and once with the outer list in the same role. Both instances of the algorithm are run 1,000 times, with an expectation of convergence towards the anticipated ROC curve and AUC values. Figure 4 presents the outcomes derived from generating and scrutinizing the inner and outer substitution lists.

The outer substitution list broadly mirrors the CNN baseline model, deviating slightly at the extremities, showcasing an AUC score of 0.798—marginally lower than the baseline CNN score of 0.822 (Archana & Jeevaraj, 2024). In contrast, the inner substitution line exhibits a distinct pattern: below the CNN baseline within FPR [0, 0.17], surpassing the baseline within FPR [0.17, 0.4], and descending below the baseline within FPR [0.4, 1]. Its AUC score of 0.772 falls slightly beneath the CNN baseline. Nevertheless, the inner substitution line boasts a higher TPR (0.782) compared to the human baseline at similar FPR levels, while also demonstrating a lower FPR (0.182) compared to the human baseline at similar TPR values (Shahid et al., 2025; Caruana & Niculescu-Mizil, 2006).

Ethical considerations

The ethical considerations around having machine-driven decisions in life-critical medical situations using the pure augmented hybrid algorithm are not fully explored. Granting final decision authority to humans is mentioned as a solution but not fleshed out. This study is the first step toward a hybrid algorithm. Nonetheless, this could be addressed by granting a human final decision-making authority which is aligned with the global concept of human-in-the-loop (Sasseville et al., 2025; Schuitmaker et al., 2025; Siddique et al., 2024; van den Berg, 2024; Topol, 2019).

Limited dataset size and generalizability

The study’s findings may not be directly applicable to all dermatological settings due to the specific dataset used, which consists of 150 images focused on particular characteristics of skin lesions (Reddy et al., 2023; Shahid et al., 2025; Tschandl et al., 2019). The small size of the evaluation dataset and the potential selection bias limits the generalizability of the results (Sabazade et al., 2025; Esteva et al., 2017). The study does not explicitly address whether the dataset fully represents real-world clinical scenarios (Liu et al., 2024; Bejnordi et al., 2017). Different populations, imaging techniques, or lesion types may yield different results (Grzybowski, Jin & Wu, 2024; Winkler et al., 2019; Bejnordi et al., 2017). A larger, more diverse dataset in future studies would help address these concerns and strengthen the conclusions (Yan et al., 2025; Chen et al., 2024; Tschandl et al., 2020).

Model complexity and interpretability

While the CNN models used, particularly EfficientNetB3, demonstrate strong performance, they inherently function as “black boxes,” limiting interpretability (Räz, 2024). This lack of transparency is a critical concern in medical settings, where clinicians require not only accurate predictions but also insights into the reasoning behind them to build trust and validate results. Without interpretability, it becomes difficult to detect biases, troubleshoot errors, or confidently apply the model’s predictions in high-stakes scenarios. Incorporating techniques such as saliency maps, attention mechanisms, or SHAP values could enhance transparency by identifying which features or regions of an image influence the model’s output (Cohen-Inger et al., 2025; Deng et al., 2024; Soomro, Niaz & Choi, 2024). Hybrid approaches, combining interpretable rule-based models with deep learning, may also strike a balance between performance and explainability (Khalil et al., 2023; Caruana & Niculescu-Mizil, 2006). While this study focuses primarily on performance, interpretability remains a crucial area for future research to ensure reliable clinical integration (Coots et al., 2025; Baumann et al., 2024). Addressing the trade-off between model complexity and interpretability will be key to gaining practitioner trust and achieving better patient outcomes (Guyton, Pak & Rovira, 2025; Bria, Marrocco & Tortorella, 2020).

Inclusion of metadata

The decision to exclude metadata from the model may overlook potentially valuable information that could improve diagnostic accuracy (Khan et al., 2025; Duan et al., 2024; Wang, Wong & Lu, 2020). Incorporating metadata, such as patient age and lesion location, might enhance the model’s performance (Guermazi et al., 2024; Kania, Montecinos & Goldberg, 2024; Esteva et al., 2019).

Resource intensive training

Training deep learning models, especially on large datasets with complex architectures, requires significant computational resources and time (Rahman et al., 2021; Bria, Marrocco & Tortorella, 2020; Litjens et al., 2017). This limitation could hinder the broader adoption and replication of the study’s findings, particularly in resource-constrained settings (Zhang et al., 2024a, 2024b; Topol, 2019).

Dependency on histopathological examination

The ground truth labels for the dataset were established via histopathological examination, which itself has limitations (McCaffrey et al., 2024; Göndöcs & Dörfler, 2024), including sampling bias (Wang et al., 2024; Webb et al., 2024) and interobserver variability (Pinello et al., 2025; Shinde et al., 2025). Reliance solely on histopathology may introduce errors in the dataset labels (Zhang et al., 2024b).

Human baseline variability

The human baseline performance was established using a subset of participants experienced in dermoscopy. However, individual variability in human performance (Naseri & Safaei, 2025; Stevens et al., 2025; Naeem et al., 2024), even among experienced dermatologists (Gupta et al., 2025; Rubegni et al., 2024), could introduce uncertainty (Sanz-Motilva et al., 2024) in comparison with the CNN models (Miller et al., 2024; Ali et al., 2023).

Dataset imbalance

The dataset used for training and evaluation may suffer from class imbalance issues (Liu et al., 2020; He & Garcia, 2009), particularly with the low incidence rate of malignant cases (Gurcan & Soylu, 2024). This imbalance could affect the model’s performance and generalizability (Fang et al., 2025).

Exploring model enhancements for improved performance

This study demonstrates how a hybrid approach—combining CNN predictions with human expertise—outperforms individual baselines (Nugroho, Ardiyanto & Nugroho, 2023), including both standalone CNNs (Liu et al., 2025) and human assessments (Selvaraj et al., 2024; Esteva et al., 2017). However, the AUC scores achieved are lower than those reported in recent studies, such as Houssein et al. (2024) and Nugroho, Ardiyanto & Nugroho (2023), which utilize more advanced CNN architectures and training techniques.

Future work will focus on testing our hybrid approach with these newer methods and exploring ways to adapt or incorporate them into our framework to achieve further performance gains. Benchmarking against these recent advancements will help ensure our approach remains competitive and aligned with the latest developments in skin lesion classification.