Multiscale attention-based network to enhance detection and classification of autism spectrum disorders using convolutional neural network

SE-CNN model

The squeeze-and-excitation convolutional neural network (SE-CNN) structure is presented in Fig. 1.

This module combines CNN channels with spatial information through the use of many convolutional modules (Xu et al., 2020; Kadri et al., 2021). Additionally, it incorporates geographical dependencies into its structure to enhance the network’s representational capacity. The squeeze-and-excitation module in SE-CNN enhances the connections between convolutional feature channels, thereby improving the network’s representational quality while preserving the original feature map size. Consider the input feature set to be defined as

(1) $$\mathbf{V}=\{v_{1,1},v_{1,2},\dots ,v_{i,j},\dots ,v_{H,W}\},where-v_{i,j}\in {\mathbb{R}}^{C\times 1\times 1}.$$

This module performs a “squeeze” operation to capture channel-level global features, where the input feature map is compressed using global average pooling. This process generates a feature map with dimensions of 1 $\times$ 1 $\times$ C, defined as:

(2) $${\mathbf{V}}_{\mathrm{c}\mathrm{S}\mathrm{E}}=\mathbf{V}\otimes \mathbf{w},$$where $\otimes$ indicates element-wise multiplication.

Figure 2 shows three fully connected (FC) layers that capture channel dependency and learn channel-specific properties. To capture localized spatial characteristics of a feature map, the 2D convolution operation applies a filter (or kernel) that moves along the height and width dimensions.

This method captures important patterns, such as edges, curves, and textures, which can help identify distinct facial characteristics associated with autism. The convolution operation is calculated as:

$$\mathbf{O}(i,j)=\sum _{p=0}^{k-1}\sum _{q=0}^{k-1}\sum _{r=0}^{C-1}\mathbf{I}(i+p,j+q,r)\cdot \mathbf{F}(p,q,r).$$

The revised result is then obtained by rescaling the feature map to incorporate the information at the channel’s attention. An “excitation” process is applied to these global features to simulate the interchannel interactions and obtain appropriate channel-specific weights. The resulting descriptors pass through two dense layers to model inter-channel dependencies. The first reduces the size by a factor of 16, followed by ReLU activation. Then, it expands with a sigmoid activation, followed by a sigmoid activation, to produce channel-wise weights. Through the application of a single-channel convolution and normalization of the resulting weights using a sigmoid activation function, which is

(3) $${\mathbf{V}}_{\mathrm{s}\mathrm{S}\mathrm{E}}=\bigcup _{x=1}^H\bigcup _{y=1}^{W}S(\varphi \ast v_{x,y})v_{x,y}.$$

A set of trainable weights, $\gamma$ and $\delta$, is introduced to balance spatial and channel attention, resulting in the output:

(4) $${\mathbf{V}}_{\mathrm{s}\mathrm{c}\mathrm{S}\mathrm{E}}=\gamma {\mathbf{V}}_{\mathrm{s}\mathrm{S}\mathrm{E}}+\delta {\mathbf{V}}_{\mathrm{c}\mathrm{S}\mathrm{E}}.$$

The SE module mitigates less informative channel features while strengthening salient ones by performing an attention mechanism along the channel dimension.

Multiscale view module

Multi-view imaging has been applied in several applications, including left ventricle detection (Wang et al., 2022), quantification of coronary artery stenosis (Zhang et al., 2021a), and studies on artery-specific calcification (Zhang et al., 2021b). In these studies, the term “multi-view” typically refers to the simultaneous use of multiple types of 2D images as input to deep neural networks, which are then fused. According to Zhang et al. (2021b), artery-specific calcification was examined using axial, coronal, and sagittal views within a multitask learning framework.

In the current study, a multiscale view (MV) module is employed to enhance the neural network’s sensitivity to correlations between regions with varying spatial distances (as shown in Fig. 3). In contrast to conventional multi-view techniques, there is no need to provide the network with different inputs to achieve multiscale views. It uses parallel convolutional layers with varying kernel sizes to extract features from various receptive fields (shown in Fig. 4).

This module constructs several parallel routes using t e feature maps produced by the previous convolutional neural layer. All paths have the same number of convolutional layers, and each convolutional layer’s feature maps are the same size. Different convolutional kernels were utilized in various routes to facilitate multiscale feature extraction as follows:

• 1.

  Path initialization using CNN: The model’s core is built upon a convolutional model’s network backbone, which progressively extracts features from the input image. The CNN consists of several convolutional layers, each followed by max-pooling to reduce spatial dimensions and focus on hierarchical feature extraction:

  • •

    Block 1 consists of a Conv2D layer with 32 filters followed by max-pooling.

  • •

    Block 2 increases feature extraction depth using 64 filters.

  • •

    Block 3 further enhances the feature representation with 128 filters.

• These layers capture both low-level and high-level features by progressively learning spatial hierarchies, which are essential for image recognition tasks. $${\mathbf{F}}_{\mathrm{i}\mathrm{n}}\in {\mathbb{R}}^{H\times W\times D\times N},$$

• where H, W, D are the input feature map’s height, width, and depth, and N is the number of channels. In this case, the number of channels is reduced to $\frac{N}{M}$, where M represents the number of parallel paths: $${\mathbf{F}}_i^{(1)}={\mathrm{C}\mathrm{o}\mathrm{n}\mathrm{v}}_1({\mathbf{F}}_{\mathrm{i}\mathrm{n}}),{\mathbf{F}}_i^{(1)}\in {\mathbb{R}}^{\frac{H}{2}\times \frac{W}{2}\times \frac{D}{2}\times \frac{N}{M}},$$

• where $\mathrm{C}\mathrm{o}\mathrm{n}{\mathrm{v}}_1$ is a convolutional operation with a stride of $2\times 2\times 2$ to downsample the feature map.

• An additional convolutional layer is applied to extract features at different scales in each direction. The kernel size for the $i$-th path is defined as $(2i+1)\times (2i+1)\times (2i+1)$, therefore enabling the network to capture features from multiple receptive fields: $${\mathbf{F}}_i^{(2)}={\mathrm{C}\mathrm{o}\mathrm{n}\mathrm{v}}_{2,i}({\mathbf{F}}_i^{(1)}),\mathrm{k}\mathrm{e}\mathrm{r}\mathrm{n}\mathrm{e}\mathrm{l}\mathrm{s}\mathrm{i}\mathrm{z}\mathrm{e}=(2i+1)\times (2i+1)\times (2i+1).$$

• 2.

  Squeeze-and-excitation (SE) blocks To increase the representational power of the model, we incorporate squeeze-and-excitation blocks after the key convolutional layer. SE blocks are designed to recalibrate the feature maps by emphasizing informative features and suppressing less useful ones. This attention mechanism works in two stages:

  • Squeeze: A global average pooling operation aggregates spatial information across each channel, producing a descriptor that summarizes the international information of each feature map.

  • Excitation: The resulting descriptors pass through two fully connected layers. The first layer reduces the dimensionality by a predefined ratio (e.g., 1/16) and is followed by an activation function, such as ReLU. The second layer restores the original dimensionality, followed by a sigmoid activation, to produce channel-wise weights.

  • Recalibration: These learned weights are used to scale the original feature maps, effectively enhancing important features and suppressing irrelevant ones.

• This mechanism enables the network to focus on the most discriminative features in the data. SE blocks are applied to multiple layers in the network to enhance feature maps at different levels of abstraction, including early, mid-level, and high-level features.

• 3.

  Multi-scale feature extraction: The model leverages multiscale feature fusion by extracting feature maps from multiple layers and combining them to capture information at various resolutions. This process enables the model to preserve coarse and fine-grained details, which is particularly important for object recognition, where features at different scales provide complementary information. The feature maps from different layers, with varying channel depths, are resized to a common spatial resolution of 16 $\times$ 16. This is done by:

  • Down-sampling: The lowest-resolution feature map is resized using average pooling.

  • Up-sampling: The highest resolution feature map is resized using up-sampling.

  • Intermediate layers: Intermediate feature maps are retained without resizing.

• Once these feature maps are resized, they are concatenated along the channel dimension to form a unified representation. This fusion enables the model to utilize features captured at different resolutions, thereby improving its robustness to scale variations in the input data.

• 4.

  Channel concatenation: The concatenated multiscale feature representation is flattened and passed through a fully connected dense layer with 128 units and ReLU activation to learn non-linear combinations of the fused features. A compilation of all paths, $${\mathbf{F}}_i^{(2)}\in {\mathbb{R}}^{\frac{H}{2}\times \frac{W}{2}\times \frac{D}{2}\times \frac{N}{M}},$$

• The final output layer consists of a single neuron with a sigmoid activation to predict a probability for binary classification. $${\mathbf{F}}_{\mathrm{o}\mathrm{u}\mathrm{t}}=\mathrm{C}\mathrm{o}\mathrm{n}\mathrm{c}\mathrm{a}\mathrm{t}({\mathbf{F}}_1^{(2)},{\mathbf{F}}_2^{(2)},\dots ,{\mathbf{F}}_M^{(2)}),{\mathbf{F}}_{\mathrm{o}\mathrm{u}\mathrm{t}}\in {\mathbb{R}}^{\frac{H}{2}\times \frac{W}{2}\times \frac{D}{2}\times N}.$$

White shark optimization (WSO) algorithm

While self-attention-based Transformers excel at modeling long-range relationships, their efficiency can be limited due to the absence of local inductive bias. To address this limitation, bio-inspired optimization strategies for hyperparameter tuning offer significant benefits (Goel et al., 2020; Wang, Tan & Liu, 2018). Such algorithms provide dynamic and adaptive adjustments to hyperparameters, enhancing both feature selection and the convergence speed and stability of the model. These optimization techniques help improve the transformer architecture and attention weight distributions, enabling the model to capture the local and global representations of features more effectively. WSO is an algorithm based on bio-inspired principles designed to address challenging optimization problems, including feature selection and CNN hyperparameter tuning (Braik et al., 2022). Using neuroimaging datasets, WSO offers a behavior-rich, biologically inspired approach for optimizing CNN attention hyperparameters. WSO’s strength lies in its structured navigation through exploration, exploitation, and refinement phases, mimicking the hunting behavior of sharks.

During each cycle, WSO transitions from global search to targeted pursuit and finally to local refinement. At each stage, shark agents update their positions based on prey-sensing mechanisms and dynamic principles. WSO’s position updating rule is as follows:

$$W_{pbes{t}_i}(t+1)=\{\begin{array}{cc}{W}_{pbes{t}_i}(t)+\alpha \cdot r_1\cdot (W_{pbes{t}_j}(t)-W_{pbes{t}_k}(t))+\beta \cdot \sin (2\pi r_2),\hfill & \mathrm{E}\mathrm{x}\mathrm{p}\mathrm{l}\mathrm{o}\mathrm{r}\mathrm{a}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}(\mathrm{g}\mathrm{l}\mathrm{o}\mathrm{b}\mathrm{a}\mathrm{l}\mathrm{s}\mathrm{e}\mathrm{a}\mathrm{r}\mathrm{c}\mathrm{h})\hfill \\ W_{pbe{s}_{i}t}(t)+\gamma \cdot r\cdot (W_{gbes{t}_i}-W_{pbest}(t)),\hfill & \mathrm{E}\mathrm{x}\mathrm{p}\mathrm{l}\mathrm{o}\mathrm{i}\mathrm{t}\mathrm{a}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}(\mathrm{t}\mathrm{a}\mathrm{r}\mathrm{g}\mathrm{e}\mathrm{t}\mathrm{s}\mathrm{e}\mathrm{a}\mathrm{r}\mathrm{c}\mathrm{h})\hfill \\ G(t)+\delta \cdot \mathcal{N}(0,\sigma ),\hfill & \mathrm{S}\mathrm{e}\mathrm{a}\mathrm{r}\mathrm{c}\mathrm{h}\mathrm{I}\mathrm{n}\mathrm{t}\mathrm{e}\mathrm{n}\mathrm{s}\mathrm{i}\mathrm{f}\mathrm{i}\mathrm{c}\mathrm{a}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}(\mathrm{r}\mathrm{e}\mathrm{f}\mathrm{i}\mathrm{n}\mathrm{e}\mathrm{m}\mathrm{e}\mathrm{n}\mathrm{t})\hfill \end{array}$$where: $W_{pbes{t}_i}(t)$: position of shark $i$, $G(t)$: global best solution at iteration $t$. $r,r_1,r_2\sim \mathcal{U}(0,1)$: uniformly distributed random scalars, $\alpha ,\beta ,\gamma ,\delta$ are algorithm-specific parameters, with Gaussian distribution $\mathcal{N}(0,\sigma )$ mean 0 and standard deviation $\sigma$. Through this multi-stage movement, WSO can adaptively switch between exploring a large area of hyperparameter space (such as the size of CNN kernels or attention dropout rates) and fine-tuning promising configurations to improve classification performance and robustness across subject variability. Also, dynamic search refinement methods prevent premature convergence while ensuring effective traversal of high-dimensional search spaces. A significant feature of WSO for ASD detection is its computational efficiency, with a complexity of $(N.d)$ (Braik et al., 2022). Several continuous and discrete optimizations on applications have been successfully implemented using it. Moreover, its capability to rank features and reduce dimensionality using a threshold-based method enhances its relevance for improving model accuracy and overall performance, making it an effective tool for addressing complex optimization problems in ASD facial image classification. Table 1 summarizes the key features of WSO in the context of ASD detection, highlighting its biologically inspired mechanisms, adaptive search strategies, and the optimal optimization of deep learning models.

For mathematically simulating the collective behavior of a school of white sharks, the two best-performing solutions were kept, and the remaining sharks’ locations were adjusted to these ideal solutions with a random number rand uniformly distributed in the range $[0,1]$ as follows:

(5) $$W_{pbes{t}_i}^{k+1}=W_{pbes{t}_i}^k+\frac{W_{pbes{t}_i}^{k+1}}{2}\times \mathrm{r}\mathrm{a}\mathrm{n}\mathrm{d}.$$

During the evaluation of each function, the white shark’s position is updated as follows: Eq. (6).

(6) $$W_{pbes{t}_i}^{k+1}=\{\begin{array}{cc}{W}_{pbes{t}_i}^k\cdot \mathrm{\neg }\oplus W_{pbes{t}_o}+u\cdot a+l\cdot b,\hfill & \mathrm{i}\mathrm{f}\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{d}<mv\hfill \\ W_{pbes{t}_i}^k+\frac{v_i^k}{f},\hfill & \mathrm{i}\mathrm{f}\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{d}\ge mv\hfill \end{array}$$where $W_{pbes{t}_o}$ is the optimal solution vector, $u$ and $l$ are control coefficients. $v_i^k$ represents the velocity and $f$ represents the scaling factor with negation operator $\mathrm{\neg }$ in one-dimensional binary vectors $a$ and $b$.

The white shark’s ability to hear and smell is represented by the parameter $mv$, which increases with each successive iteration.

(7) $$mv=\left(a_0+\frac{e^{(K/2-k)}}{a_1}\right)$$where $a_0$ and $a_1$ are two positive constants used to control exploration and exploitation behaviors. Based on the simulated steps in the WSO algorithm, if any shark moves outside the defined search space, it is repositioned within the valid boundaries. For every function evaluation, the fitness criterion is recalculated to determine the best-fitting white shark. The White Shark Optimization (WSO) method (shown in Algorithm 1) begins with the initialization of a population of N sharks, each with random coordinates $W_i$ and initial velocities $V_i=0$ (Block 1). To determine the first global be t location $W_{gbest}$, a fitness function $f(W)$ is defined to assess the candidate solution. A velocity update is performed during each iteration by using the following equation (Block 2–3):

$$V_i=\omega \cdot V_i+r_1\cdot (W_{pbest}-W_{pbes{t}_i})+r_2\cdot (W_{gbest}-W_{pbes{t}_i}).$$

There are two random coefficients $r_1$ and $r_2$, the inertia factor $omega$, and each shark’s personal best position $W_{pbest}$ shark’s award, the position update $i$ computed using $W_{pbes{t}_i}=W_{pbes{t}_i}+V_i$, ensuring that $W_i$ remains within the search space bounds (lines 13–15). Upon identifying better solutions, $W_{pbest}$ and $W_{gbest}$ are updated after evaluating the fitness of the new positions $f(W_{pbes{t}_i})$. Block 4–6 of this process is repeated recursively (Block 7–8) until the maximum number of iterations T is reached or the convergence criteria are met. Following T iterations, the ideal solution (Block 9), $W_{gbest}$, is returned, along with thresholds for feature selection if necessary.

Dataset collection and preprocessing

In this study, the first dataset, the Piosenka dataset (https://www.kaggle.com/cihan063/autism-image-data), comprises facial photos of children with and without autism, collected from the publicly available Kaggle website (Piosenka, 2021). A total of 2,866 photographs were collected of children with and without autism. These pictures were taken from websites and Facebook pages that included autism-related content. A pre-processing procedure was employed to clean and crop the photos, preparing the data for training deep learning models. The preprocessing step was required since the dataset was compiled from online sources by Piosenka (2021). During the creation of the dataset, the author cropped the faces of the original photos; the dataset was then divided into three groups (as shown in Table 2): 280 photos for testing, 80 for validation, and 2,526 for training. Scaling the pictures was accomplished using a normalizing method, which changed the pixel value from the range [0, 255] to [0, 1].

A total of 75 videos have been compiled in the second dataset (YTUIA dataset: https://rolandgoecke.net/research/datasets/ssbd/; YouTube data curated at University Islam Antarabangsa), which is a dataset analyzing behaviors associated with autism (Rajagopalan, Dhall & Goecke, 2013) frames were extracted from 100 movies, even though only 50 of these are publicly available on YouTube. The remaining 50 were obtained from therapists and professional organizations. Standard control groups consisted of YouTube videos featuring kindergarten activities within the relevant age range.

In the first phase, the MTCNN algorithm was used to detect faces in every frame. Subsequently, a detailed preprocessing pipeline, which included alignment, cropping, and resizing, was implemented.

To ensure the integrity of model evaluation and eliminate any potential information leakage in the YTUIA dataset, we conducted a rigorous split validation and re-clustering procedure illustrated in Algorithm 2. Given the video-derived nature of the dataset, traditional frame-level splitting risks subject-level or temporal leakage. Therefore, we employed a perceptual clustering strategy based on deep features to enforce subject-consistent groupings (Truong, Jush & Lenga, 2024; Ahn et al., 2019). Initially, cosine similarity between features extracted from training and test images using a pre-trained ResNet-50 model revealed significant overlap (maximum similarity = 0.9714), indicating a substantial likelihood of leakage. To address this, we applied K-Means clustering on the deep features of all 1,163 images. We systematically experimented with different numbers of clusters (ranging from 30 to 150) to assess their effect on inter-set similarity. The fixed value provided a functional trade-off between cluster granularity and computing efficiency (Ahn et al., 2019).

For each clustering configuration, clusters were split into training (80%), validation (10%), and test (10%) sets. Following the initial split, cosine similarity between training and test features was computed. When the maximum similarity exceeded the conservative threshold of 0.90, highly similar test images were reassigned to the training set. Table 3 summarizes the outcomes of all cluster configurations.

All configurations successfully reduced the final maximum similarity below the 0.95 leakage threshold. However, we selected the 100-cluster configuration as the final adopted split for the following reasons:

• It maintained a balanced test set size (68 images), providing sufficient statistical support for evaluation.

• The final cosine similarity (0.8989) was below the strict 0.90 cutoff, ensuring minimal residual overlap.

• The number of reassigned images (28) was moderate, indicating an effective but not overly disruptive adjustment.

This configuration provided the best trade-off between leakage prevention, test set sufficiency, and minimal reassignment overhead.

Accordingly, the final YTUIA dataset split used throughout all experiments consists of 951 training, 144 validation, and 68 test images, with a verified final maximum cosine similarity of 0.8989, confirming the absence of subject- or frame-level leakage. The same methodology was applied to the Piosenka dataset, where the original split was retained (maximum similarity = 0.90, below the threshold), as no significant overlap was detected. These validated and leakage-free splits were saved and used consistently in all subsequent training, validation, and evaluation stages for both SE-CNN and SE-MSF-CNN models.

Table 4 illustrates the statistics of the dataset and optimal representation, dividing the dataset into a test set of 100 samples and a training set of 1,068 samples, maintaining a 1:1 ratio of people with autism spectrum disorders to non-autistic individuals.

Training settings and hyperparameter configuration

The training process for all models was accelerated using an NVIDIA Tesla T4 GPU on Google Colab (Google, 2024). The Python 3.8 environment included a range of libraries necessary for model development, data processing, and performance evaluation. The training was conducted using Jupyter notebooks, with batch sizes ranging from 32 to 128, and was supported by 12 GB of RAM and 16 GB of GPU memory.

The optimal weights for every model were chosen using early stopping during training with the patience of five epochs and validation loss monitoring $monitor=\text{'}val\mathrm{\_}loss\text{'}$. Table 5 summarizes the key training parameters and hyperparameter configurations used to optimize the developed models. Five-epoch patience is implemented to stop training when the validation loss plateaus, helping prevent overfitting. Through model checkpointing, the best-performing model based on validation loss is saved. Additionally, if the validation loss does not improve over three consecutive epochs, the learning rate is reduced by a factor of 0.1, allowing for finer adjustments. The parameter bounds are set between [0.0001, 10, 10] for the lower limits and [0.1, 200, 200] for the upper limits, optimizing for smaller and larger learning rates and model sizes. A total of 10 search agents (sharks) are employed in the training process, which optimizes three parameters (dimension = 3) and is limited to two iterations to ensure a balance between parameter exploration and computational efficiency.

Performance evaluation and assessment

The model’s performance was assessed based on accuracy, training parameters, processing time, and comparison with existing methods. The technique was evaluated against conventional models, demonstrating its effectiveness in classifying Autism. Additionally, statistical significance was ensured through multiple model evaluations. Accuracy, precision, recall, F1-score, kappa score, and processing time were compared to validate performance.

A description of the evaluation metrics is provided as follows:

• -

  **Accuracy**: $$\mathrm{A}\mathrm{c}\mathrm{c}\mathrm{u}\mathrm{r}\mathrm{a}\mathrm{c}\mathrm{y}=\frac{TP+TN}{TP+TN+FP+FN}$$

• where TP (true positives), TN (true negatives), FP (false positives), and FN (false negatives) represent classification outcomes.

• -

  **Precision**: $$\mathrm{P}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}=\frac{TP}{TP+FP}.$$

• Identifying the percentage of all projected positive possibilities that were accurately forecasted.

• -

  **Recall** (Sensitivity): $$\mathrm{R}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}=\frac{TP}{TP+FN}.$$

• The percentage of accurately projected positive cases among all the real positive cases.

• -

  **F1-score** (Harmonic Mean of recall and accuracy.): $$\mathrm{F}1\text{-}\mathrm{s}\mathrm{c}\mathrm{o}\mathrm{r}\mathrm{e}=\frac{2\times \mathrm{P}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}\times \mathrm{R}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}}{\mathrm{P}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}+\mathrm{R}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}}.$$

• -

  **Cohen’s Kappa Score**: $$\kappa =\frac{pCohen\mathrm{\prime }se}{1-p_e}$$

• where $p_o$ is the observed concurrence between the model predictions and their observed convergence, and $p_e$ is the expected consistency based on chance.

The SE-CNN model shows (shown in Figs. 5A and 5B) consistent improvement in both validation loss and accuracy. The model achieves a validation accuracy of 80% in the first epoch and reaches 97.5% by the 20th epoch. The best validation loss was reached at 0.1553 on the 20th epoch, corresponding to a validation accuracy of 97.5%.

Initial training accuracy for the SE-MSF-CNN model (given in Figs. 5C and 5D) was relatively low (59.08%), but the validation accuracy was significantly higher (82.50%). The model checkpointed, saving the model with a validation loss of 0.4344. Figure 6 illustrates the performance evaluation for dataset #1 and dataset #2. By epoch 5, the validation accuracy reached 92.50%, and the validation loss had decreased to 0.2494 as the model learned to represent features more effectively. The model continued to improve, only slightly slower than in the first few epochs. With validation, loss fluctuated but improved to 0.1446 by epoch 9, with validation accuracy stabilizing at 96.25%. Training accuracy improved significantly, reaching nearly 100% by epoch 16. The validation loss decreased until epoch 18 (0.09365), reaching its lowest observed value to date. Validation accuracy remained high at 95–96%. The SE-CNN had the lowest precision at 88.05%. The SE-CNN achieved a perfect recall of 100%, correctly identifying all positive cases. The optimized versions had slightly lower recall values but performed exceptionally well. The SE-MSF-CNN also had a competitive F1-score of 94.41%. In the case of dataset #2 (Figs. 7, and 8), the SE-CNN model’s F1-score (0.8866) is a good balance between precision and recall, while its Cohen’s Kappa (0.78) and specificity (0.92) overall agreement and correct identification of negatives. The SE-MSF-CNN had an accuracy of 0.85, precision of 0.8723, and recall of 0.82. However, the specificity (0.88) is still satisfactory, reflecting its ability to identify negatives correctly.

A paired t-test was performed across multiple runs to account for variability due to random initialization and data shuffling and to determine the statistical significance of the differences between the models. Table 6 presents the results, showing that SE-MSF-CNN consistently outperformed SE-CNN across all evaluated metrics. The improvements were statistically significant, as indicated by p-values less than 0.05 for each metric. The SE-CNN model demonstrated moderate stability across five runs, achieving a mean accuracy of 90% with a standard deviation of 0.02. In contrast, SE-MSF-CNN exhibited greater consistency due to its multiscale feature fusion, achieving a higher mean accuracy of 94% and a lower standard deviation of 0.01. Performance metrics such as precision, recall, F1-score, Cohen’s kappa, and specificity followed similar patterns, with SE-MSF-CNN consistently outperforming SE-CNN. The paired t-test results yielded p-values below 0.05 for all metrics, indicating that the performance improvements of SE-MSF-CNN over SE-CNN are statistically significant.

To ensure that models for ASD remain reliable outside of the original training environment, they must be validated on various datasets. Testing on external datasets ensures that the model captures generalizable ASD-related variables rather than dataset-specific biases because variations in imaging methods, equipment, and patient characteristics may affect the model’s performance. Building on this benefit, we tested our ASD detection models on a different dataset after training them on the initial dataset, as illustrated in Table 7. Specifically, we tested both SE-MSF-CNN and SE-MSF-CNN with and without optimization on the YTUIA dataset by conducting five runs after training them on the Piosenka dataset and vice versa. This process, known as cross-dataset validation, allowed us to assess the model’s ability to generalize across diverse demographics and imaging conditions, thereby ensuring its robustness and practical applicability in various clinical settings. The findings demonstrate strong performance: SE-MSF-CNN with WSO obtained a mean accuracy of 90% ( $\pm$0.03, 95% CI [89.6–90.4%]) and an area under the curve (AUC) of 0.94 ( $\pm$0.03, 95% CI [93.6–94.4%]) when trained on the YTUIA dataset and tested on Piosenka, in contrast to 89% ( $\pm$0.03, 95% CI [88.6–89.4%]) and 0.93 ( $\pm$0.03, 95% CI [92.6–93.4%]) respectively for SE-MSF-CNN without WSO. In addition, SE-MSF-CNNs with WSO achieved an accuracy of 89% ( $\pm$0.03, 95% CI [88.6–89.4%]) and an AUC of 0.93 ( $\pm$0.03, 95% CI [92.6–93.4%]) when trained on the Piosenka dataset and evaluated on the YTUIA dataset. These findings indicate that our models perform well across diverse datasets, demonstrating their generalizability. However, they are marginally lower than the internal split results (e.g., a 95% accuracy for SE-MSF-CNN on the YTUIA dataset).

Computational efficiency comparison

The computational efficiency of the developed CNN models based on WSO for ADS detection using facial images is presented in Fig. 9, which compares the computation time and parameter count for each model in binary classification tasks. The SE-CNN model is a lightweight architecture with approximately 17 million parameters, requiring only 4 min for training initially and 12 min after applying the WSO optimization process. In contrast, the SE-MSF-CNN model took significantly longer, with a CPU time of 7 min and 44 s and a wall time of 49 min and 36 s, likely due to its increased complexity.

To facilitate fair comparisons of bio-inspired WSO optimization and conventional hyperparameter tuning methods, traditional techniques such as grid search and Bayesian optimization were conducted. These methods were used to tune the hyperparameters of the SE-MSF-CNN model using the YTUIA dataset. With widely recognized settings as illustrated in Table 8. Each strategy aimed to adjust the learning rate and the number of convolutional filters, two factors critical to a CNN-based model’s overall performance. All methods were evaluated under equivalent computational conditions (NVIDIA Tesla T4 GPU, 12 GB RAM, 16 GB GPU memory).

A grid search was conducted using two fixed sets: [16, 32, 64] for the filters and 0.0001, 0.001, 0.01 for the learning rate. In Bayesian optimization, we examined the learning rate using a Gaussian Process with Expected Improvement and filters, employing [16, 64] as the number of parameters, as well as the filter using [16, 64]. Cross-validation was used to assess accuracy using a fivefold approach. The grid search was performed on all cores (n_jobs = −1), and Bayesian optimization was restricted to 10 iterations to ensure reproducibility and fairness. Regarding stopping criteria, all techniques adhered to a combination of early stopping (based on monitoring validation loss, with patience set to five epochs and a maximum number of iterations). Specifically, WSO was limited to five iterations, Bayesian optimization to approximately 100 iterations, and grid search continued until all combinations were evaluated or early stopping was triggered. Early stopping halted training if the validation loss did not improve for five consecutive epochs, accompanied by a learning rate reduction (factor = 0.1, patience = 3) to prevent overfitting. These criteria were consistently applied across all methods to ensure comparability.

As illustrated in Table 9, the WSO performed 50 assessments (10 sharks $\times$ 5 iterations), as defined by its algorithm parameters. Bayesian optimization was configured to perform approximately 100 evaluations, leveraging its adaptive sampling strategy to explore the search space efficiently. In contrast, grid search evaluated approximately 1,000 combinations due to its exhaustive nature. These differences reflect the inherent designs of the methods, with WSO and Bayesian optimization requiring fewer evaluations than grid search due to their intelligent search strategies. Across five runs, we observed that WSO required approximately 1 h to complete, achieving a mean accuracy of 95% ( $\pm$0.01) and an AUC of 0.98 ( $\pm$0.02). In comparison, Bayesian optimization attained an accuracy of 94% ( $\pm$0.02) and an AUC of 0.97 ( $\pm$0.02) in 1.5 h, while grid search achieved an accuracy of 93% ( $\pm$0.02) and an AUC of 0.96 ( $\pm$0.03) over 2 h. The time difference of 1,800 s (30 min, or 33.3% reduction) between WSO and Bayesian optimization is significant in resource-constrained settings, such as clinical applications for ASD detection, where time savings enhance workflow efficiency. Moreover, WSO achieved this with fewer evaluations (50 vs. 100 for Bayesian optimization) and higher accuracy (0.95 vs. 0.94), with comparable stability (standard deviation $\pm$0.01 vs. $\pm$0.02). The exhaustive nature of grid search, as noted by the reviewer, naturally results in longer computation times (2 h), making WSO’s 50% time reduction compared to grid search particularly notable. This efficiency, combined with WSO’s ability to optimize a moderately complex search space with fewer evaluations, supports its computational advantage over both methods.

The results indicate that WSO is an effective solution for time-sensitive or large-scale applications, as it reduces tuning time by minimizing the number of evaluation steps without compromising model accuracy. While grid search, despite its exhaustive nature, shows diminished performance due to its longer runtime and lower accuracy, Bayesian optimization remains a strong competitor, achieving high accuracy with a moderate computational cost. WSO can effectively balance exploration and exploitation, enabling the avoidance of local minima and efficient navigation of vast hyperparameter spaces. Due to its efficiency, WSO is a more effective alternative to hyperparameter optimization, yielding faster tuning times and improved accuracy compared to exhaustive techniques, such as grid search.

The precision-recall (PR) curve analysis is a critical tool for evaluating the effectiveness of machine learning models, particularly in instances where class imbalance is evident and in binary classification tasks. CNNs are complex topologies that require advanced evaluation techniques to ensure practical application, which is particularly important for assessing deep learning models. PR curves illustrate the relationship between precision (positive predictive value) and recall (sensitivity), which contrasts with conventional measurements like accuracy, which can be misleading when datasets are unbalanced. A comparison of optimized and non-optimized models using PR curve analysis illustrates the specific advantages of optimization methods. By observing how the trade-offs between accuracy and recall have changed due to model improvements, stakeholders can determine whether optimization enhances the model’s ability to detect positive classes. The area under the curve precision recall (AUC-PR) for the first three models (Figs. 10A, 10B and 10C) indicates a similar aggregate performance with each achieving an AUC-PR of 97%. For both datasets, compared with WSO-SE-MSF-CNN, WSO-SE-CNN exhibits a more rapid increase in precision when recall is low, making it more suitable for tasks in which it is essential to reduce the number of false positives. Based on an AUC-PR of 96% shown in Fig. 10D, model WSO-SE-MSF-CNN performs marginally worse than the others for dataset #2, but it exhibits a more stable PR curve overall, suggesting robustness rather than optimal performance.

Abalation study

An ablation study is conducted to assess the impact of key components on the framework’s performance. The following model variations are considered:

• w/o WSE: The SE-CNN model is trained with incorporating WSO optimization.

• w/o SE: Optimization is excluded from the SE-CNN model.

• w/o WMSF: The SE-MSF-CNN model is trained with incorporating WSO optimization.

• w/o MSF: The model is trained using MVA but without optimization.

The ablation results are presented in Tables 10 and 11 for Datasets #1 and #2, respectively. Notably, the inclusion of the WSO significantly improves the model’s performance. From Table 11, the SE-CNN model achieved a relatively high accuracy of 0.89, with substantial precision (0.9149), meaning that most of the predicted positives were correct. The recall (0.86) indicates that it identified 86% of the actual positives. The model’s F1-score (0.8866) strikes a good balance between precision and recall, while its Cohen’s kappa (0.78) and specificity (0.92) indicate solid overall agreement and accurate identification of negatives. Additionally, the SE-MSF-CNN model exhibited slightly lower performance, with an accuracy of 0.85, precision of 0.8723, and recall of 0.82. The F1-score (0.8454) and Cohen’s kappa (0.7) indicate more variability than SE-CNN, suggesting that adding multi-scale fusion may not have yielded better results. However, the specificity (0.88) is still relatively high, indicating its ability to identify negatives correctly. Among the models tested, the WSO-SE-MSF-CNN model with dataset #1 demonstrated the highest accuracy rate, achieving 95.36% accuracy for dataset #2. The optimized SE-CNN model achieved a validation accuracy of 91.00%. The experimental findings indicate that channel attention-based models enhance model efficiency and accuracy in detecting ASD.

To assess the statistical performance of the models, McNemar’s test and paired t-tests were applied. McNemar’s test evaluates whether the models’ predictions on the same dataset differ significantly across various classification outcomes, focusing on paired nominal data. Meanwhile, paired t-tests compare the mean performance parameters (accuracy, precision, and recall) across multiple runs. Using the YTUIA dataset, we computed the mean, standard deviation, and 95% confidence intervals for each performance metric (accuracy, precision, recall, F1-score, Cohen’s kappa, specificity, and AUC). The SE-MSF-CNN model achieved an average accuracy of 95% (95% CI [94.8–95.2%]) when enhanced with WSO, compared to 90% for the SE-CNN model (95% CI [89.98–90.02%]). Paired t-tests were conducted to compare the baseline models with their WSO-enhanced counterparts and SE-CNN against SE-MSF-CNN. All metrics yielded p-values less than 0.05, indicating statistically significant improvements. To further validate the superiority of SE-MSF-CNN with WSO, McNemar’s test was applied to the model predictions, resulting in p-values of 0.001. A shows these statistically significant findings. The statistically significant differences between SE-MSF-CNN and its WSO-optimized counterpart were demonstrated in Table 12, which produced p-values below 0.01, providing strong evidence that the prediction differences between SE-MSF-CNN and its WSO-optimized counterpart are statistically significant.

XAI for ASD detection

Grad-CAM visualization highlights how the deep learning model identifies critical features in RGB facial images for both class recognition tasks. By overlaying a heatmap on the input images, the visualization enhances the model’s interpretability, illustrating the precise regions that contribute to predictions. This approach improves practical applicability, offering insights into the model’s decision-making process.

Several features of RGB facial recognition and attention methods, such as SE and MSF, are combined with automated model optimization to improve performance and interoperability. To identify which spatial regions the model prioritizes during facial recognition, Grad-CAM is applied to the different attention-generated feature maps. Each feature map corresponds to a unique attention focus, highlighting specific facial regions or patterns that influence the model’s decisions. These multi-scale attention feature maps are displayed using Grad-CAM heatmaps in Figs. 11–13 to illustrate the hierarchical nature of the model’s interpretability, with lower-level attention modules detecting local details and higher-level maps highlighting more global, semantically rich regions. Depending on the distinctive features assessed as most discriminative by the model, we observe a dynamic shift in focus across various facial areas, including the eyes, nose, mouth, and eyebrows. Some maps emphasize fine-grained textures and edges, while others capture broader contextual information, illustrating the complementary roles of multiple attention layers. Additionally, the model can be configured to prioritize the most distinguishing features for ASD detection by dynamically adjusting feature weights using the SE mechanism. Additionally, it can adapt to varying input conditions, such as changes in lighting, occlusions, or facial expressions, by utilizing multiple feature maps. Additionally, Grad-CAM offers transparency by highlighting key regions that the attention mechanism focuses on, providing insight into the model’s decision-making process.

Comparison with state-of-the-art approaches

Table 13 provides an overview of similar works utilizing DL-based methods for identifying and classifying ADS using images or videos. The proposed optimized CNN-based-MSF model achieved state-of-the-art performance in ASD detection using facial images when optimized with the WSO function, attaining 95.36% accuracy, 98.57% sensitivity, and 92.62% specificity. Similarly, the model by Alam et al. (2022), which utilized ResNet-50 and Xception, achieved 95.0% accuracy, 98.0% AUC, and 95.0% precision on the same dataset. The CNN-based-MSF model, enhanced with WSO, also achieved 91.00% accuracy, 92.00% sensitivity, and 90.19% specificity for multimodal video datasets. However, video-based ASD diagnosis presents additional challenges, including temporal variations, motion artifacts, changing lighting conditions, and the need for sequential feature extraction. Since video frames capture dynamic facial expressions and movements, models must effectively learn temporal relationships while minimizing noise and redundancy. Despite these challenges, the proposed model demonstrated promising results, confirming its robustness in multimodal and facial image-based classification tasks.