SA3C-ID: a novel network intrusion detection model using feature selection and adversarial training

Map and compass operator stage

The SABPIO algorithm introduces a simulated annealing mechanism, a multidimensional similarity strategy, and a mutation mechanism in the map and compass operator stage to address the issue of traditional PIO algorithms converging too quickly and falling into local optima too early.

In each iteration of the SABPIO algorithm, an inner loop based on the simulated annealing algorithm is appended to the map and compass operator stage. Within this loop, each value in the random feature subset is subjected to random perturbation, after which the fitness is recalculated. The new feature subset is accepted probabilistically using the Metropolis criterion, as demonstrated in Eq. (2).

(2) $$p(P_{global}\Rightarrow {P_i}^{\prime })=\{\begin{array}{cc}1\hfill & \mathrm{\Delta }E<0\hfill \\ e^{-\frac{\mathrm{\Delta }E}{T}}\hfill & otherwise\hfill \end{array}$$as where $p(P_{global}\Rightarrow {P_i}^{\prime })$ denotes the probability of accepting the new solution ${P_i}^{\prime }$, and ΔE denotes the energy difference (i.e., the difference in fitness between the old and new solutions).

When updating the position of the pigeon flock, the SABPIO algorithm proposed a position update formula derived from the Pearson correlation coefficient, cosine similarity, and Jaccard similarity coefficient. Utilizing weighted calculation enables the mitigation of limitations associated with the measurement method, thereby circumventing the defects inherent to a solitary measurement method, as illustrated in Eq. (3).

(3) $$\begin{array}{rl}{V}_i& =CompositeSim(P_i,P_{global})\\ & ={\omega }_{1}Pearson(P_i,P_{global})+{\omega }_{2}Consine(P_i,P_{global})+{\omega }_3[2Jaccard(P_i,P_{global})-1].\end{array}$$

The utilization of the twofold Jaccard correlation coefficient minus one in the formula is intended to reduce the domain of values to an equivalent range as the Pearson correlation coefficient and cosine similarity. ${\omega }_1$, ${\omega }_2$ and ${\omega }_3$ are weighting factors and ${\omega }_1+{\omega }_2+{\omega }_3=1$.

In order to circumvent the occurrence of duplicate solutions, which have been demonstrated to diminish the efficacy of the algorithm’s search capability, the SABPIO algorithm has been engineered to ascertain the presence of any existing solutions prior to the process of updating a given solution. In the event that a duplicate solution is identified, the SABPIO algorithm employs a process of random mutation, modifying each dimension of the solution according to a uniform random number. This approach serves to augment the search space, thereby enhancing the algorithm’s capacity to explore a more extensive range of possibilities.

Landmark operator stage

In the landmark operator stage, all solutions are sorted according to their fitness values in each iteration. Solutions with lower fitness values are then eliminated. SABPIO proposed the population decay factor $\alpha$ to control the decay rate of the population in this stage. The population decay is shown in Eqs. (4) and (5).

(4) $$\alpha =\beta \cdot e^{-{\displaystyle \frac{t}{{\log }_{2}Nu{m}_{pigeon}}}}$$

(5) $$Nu{m}_{pigeon}^t=\alpha \cdot sort(Nu{m}_{pigeon}^{t-1}).$$

In Eq. (4), $\beta$ is defined as the interval between 0 and 1, and $t$ is the number of landmark operator iterations. The SABPIO algorithm improves the traditional pigeon inspired optimizer algorithm by halving the decay with each iteration, and updates the population size according to Eq. (5) to prevent rapid population decay.

Problem mapping

This article models the detection process of network attack behavior as a Markov decision process (MDP). The modeled MDP is defined as a four-tuple $M=<S,A,R,\gamma >$, where $S$ is the state space, $A$ is the action space, $R$ is the reward space, and $\gamma$ is the reward discount factor. In standard MDPs, the state transition probability $P(s^{\prime }|s,a)$ is typically defined as the likelihood of the environment transitioning to the next state $s^{\prime }$ given a state $s$ and an action $a$. However, in this article, the state transition probability is not explicitly modeled. The SAC algorithm is a model-free method that utilizes reinforcement learning. It obtains state transition experience through interaction with the environment and uses experience replay to update the strategy and value function. The algorithm adaptively learns the optimal strategy for attack detection without explicitly modeling the dynamic characteristics of the environment.

For the state space $S(t)=\{s_1,s_2...s_t\}$, we consider all data features in the network traffic data except labels as the current state $s_t$, including various statistics and metrics of the network traffic, such as packet sizes, protocol types, traffic rates, etc. Each state $s_t$ in $S(t)$ is a feature vector that describes the current situation of the network traffic.

In each state $s_t$, the agent samples the probability distribution of the action output by the action strategy function ${\pi }^{\ast }\left(a|s\right)$ to obtain the specific action, that is, the label of the network traffic behavior, to form the action space $A(t)=\{a_1,a_2...a_t\}$. The calculation formulas are shown in Eqs. (6) and (7).

(6) $${\pi }^{\ast }\left(a|s\right)=\underset{\pi }{\mathrm{a}\mathrm{r}\mathrm{c}\mathrm{m}\mathrm{a}\mathrm{x}}{E}_{\pi }\left[\sum _{t=0}^T{\gamma }^t\left(r\left(s_t,a_t\right)+\alpha H\left(\pi \left(\cdot |s_t\right)\right)\right)\right]$$

(7) $$H\left(\pi \left(\cdot |s_t\right)\right)=-\log \pi \left(\cdot |s_t\right)$$where $\pi$ denotes the policy, $r$ denotes the reward, $\alpha$ denotes the entropy regularization coefficient that determines the relative importance of the entropy term with respect to the reward, and $H\left(\pi \left(\cdot |s_t\right)\right)$ is the entropy of the policy $\pi$ at state $s_t$.

Every action taken by the agent will generate an immediate reward $r(s_t,a_t)$, and recording the reward value can form a reward space $R(t)=\{r_1,r_2...r_t\}$. Due to the natural imbalance of network traffic, normal traffic accounts for a much larger portion of the training dataset than anomalous traffic, and the portion of different classes of anomalous traffic varies. In the design of the reward function, the corresponding reward values assigned to the agent’s performance in different situations are given full consideration. The reward value design is shown in Eq. (8) as follows:

• (1)

  When the attacker agent selects an attack category sample and the defender agent determines it as an attack and the type judgment is accurate, the attacker agent’s basic reward value is set to −4 and the defender agent’s basic reward value is set to +4;

• (2)

  When the attacker agent selects an attack category sample and the defender agent determines it as an attack but misjudges the type, the attacker agent’s basic reward value is set to +2 and the defender agent’s basic reward value is set to +2;

• (3)

  When the attacker agent selects an attack category sample and the defender agent determines it as normal traffic, the attacker agent’s basic reward value is set to +4 and the defender agent’s basic reward value is set to −4;

• (4)

  When the attacker agent selects a normal sample and the defender agent determines it as normal traffic, the attacker agent’s basic reward value is set to −1 and the defender agent’s basic reward value is set to +1;

• (5)

  When the attacker agent selects a normal sample and the defender agent determines it as an attack, the attacker agent’s base reward is set to +2 and the defender agent’s base reward is set to −2.

(8) $$R_{Basic}=\{\begin{array}{cc}{R}_a=-4,R_d=+4\hfill & a_{a,t}\in attack,a_{d,t}\in attack,a_{d,t}=a_{a,t}\hfill \\ R_a=+2,R_d=+2\hfill & a_{a,t}\in attack,a_{d,t}\in attack,a_{d,t}\ne a_{a,t}\hfill \\ R_a=+4,R_d=-4\hfill & a_{a,t}\in attack,a_{d,t}\notin attack\hfill \\ R_a=-1,R_d=+1\hfill & a_{a,t}\notin attack,a_{d,t}\notin attack\hfill \\ R_a=+2,R_d=-2\hfill & a_{a,t}\notin attack,a_{d,t}\in attack\hfill \end{array}$$

The class-weighted reward scheme (e.g., +4 for detecting U2R attacks) directly addresses data imbalance in IDS, where rare attacks (0.04% of NSL-KDD training data) are often overlooked. At the same time, in order to enable the classifier to identify the minority category of abnormal traffic, this method assigns a higher reward value to the minority category of abnormal traffic, as shown in Eq. (9).

(9) $$R=\{\begin{array}{l}\begin{array}{cc}1\cdot R_{Basic}& \raisebox{1ex}{$a_t$}\!\left/ \!\raisebox{-1ex}{$Total$}\right.>3\%\end{array}\hfill \\ \begin{array}{cc}2\cdot R_{Basic}& 1\%<\raisebox{1ex}{$a_t$}\!\left/ \!\raisebox{-1ex}{$Total$}\right.\le 3\%\end{array}\hfill \\ \begin{array}{cc}4\cdot R_{Basic}& \raisebox{1ex}{$a_t$}\!\left/ \!\raisebox{-1ex}{$Total$}\right.\le 1\%\end{array}\hfill \end{array}.$$

Mini-Batch data generation module

Reinforcement learning algorithms usually deal with unsupervised learning problems; however, the NSL-KDD and CSE-CIC-IDS2018 datasets used in this article are supervised and come with labels. In order to adapt the supervised dataset into the reinforcement learning framework, we adopted the Mini-Batch strategy. In “Problem Mapping”, all features except the label are considered to be the current state $s_t$, the label is considered to be the current action $a_t$, and all features of the latter data except the label are considered to be the next state $s_{t+1}$. The Mini-Batch structure used is shown in Fig. 3.

The fundamental principle of the Mini-Batch strategy is predicated on two key concepts: randomness and iteration. In each training batch, Mini-Batch randomly samples a set of samples containing $[s_t,a_t,s_{t+1}]$ triplets from the original dataset to avoid repeated use of samples and ensure that the dataset is fully and evenly explored during iterative training. By using the Mini-Batch strategy in SA3C-ID, we adapt the supervised dataset to the reinforcement learning framework, enabling the intrusion detection model to be effectively trained based on the data.

SA3C adversarial intrusion detection model

In the agent construction process described in this article, a reinforcement learning agent based on the SAC algorithm is designed to optimize the decision-making strategy by interactive learning with network traffic data. As shown in Fig. 4, the SA3C agent includes an Actor network, two V Critic networks, and two Q Critic networks.

The Actor network is responsible for the output of the agent’s actions. Its input is the state $s_t$ of the current time step, and its output is the action probability distribution $\pi (a|s)$, with each action having a corresponding probability value. The agent’s behavior is typically characterized by its selection of actions with higher probabilities. However, the SAC algorithm’s emphasis on exploration leads the agent, at times, to opt for actions with lower probabilities to maintain the randomness and diversity of the strategy The exploration balance is usually achieved by adding an entropy term to the loss function. The Actor network loss function in this article is shown in Eq. (10).

(10) $$L_{\mathrm{A}\mathrm{c}\mathrm{t}\mathrm{o}\mathrm{r}}=-\frac{1}{|M|}\sum _{e\in M}\left({min}_{j=1,2}{Q}_j(s_t,a_t)-\alpha \log (\pi (a_t|s_t))\right)$$where $|M|$ denotes the number of experiences used in a training step, $Q_j(s_t,a_t)$ denotes the output value of the $j$th Q Critic network under state $s_t$ and action $a_t$. The minimum of the outputs of the two Q networks is taken to minimize overestimation. Additionally, $\alpha$ denotes the entropy regularization coefficient, which is used to encourage the Actor network to output more uncertain actions.

The V Critic network is utilized to estimate the value of a state. Its input is the state $s_t$ at the current time step, and the output is the state-value function $V(s_t)$, which represents the expected return that the agent can obtain when following the current policy in the state $s_t$. The loss function of the V Critic network is shown in Eq. (11).

(11) $$L_{\mathrm{V}-\mathrm{C}\mathrm{r}\mathrm{i}\mathrm{t}\mathrm{i}\mathrm{c}}={\displaystyle \frac{1}{|M|}\sum _{e\in M}{\left(Q(s_t,a_t)-E_{{a^{\prime }}_t\sim \pi (\cdot |s_t)}\left[{min}_{j=1,2}{Q}_j(s_t,{a^{\prime }}_t)-\alpha \log (\pi ({a^{\prime }}_t|s_t))\right]\right)}^2}$$where $E_{{a^{\prime }}_t\sim \pi (\cdot |s_t)}$ represents the expectation over the action $a_t^{\prime }$ sampled from the action probability distribution of the current policy $\pi$ in the state $s_t$. This article proposed the incorporation of a regularization term associated with policy entropy into the loss function of the V Critic network to enhance the model’s generalization ability.

The Q Critic network is employed to evaluate the values of different actions. Its input is the state $s_t$ at the current time step, and its output is the action-state value function $Q(s_t,a_t)$, which represents the expected return that the agent can obtain by following the current policy after choosing action $a_t$ in state $s_t$. In this article, the loss function of the Q Critic network is shown in Eq. (12).

(12) $$L_{\mathrm{Q}-\mathrm{C}\mathrm{r}\mathrm{i}\mathrm{t}\mathrm{i}\mathrm{c}}={\displaystyle \frac{1}{|M|}\sum _{e\in M}{\left(V(s_t)-\left(r_t+\gamma E_{{a^{\prime }}_{t+1}\sim \pi (\cdot |s_{t+1})}\left[{min}_{j=1,2}{Q}_j(s_{t+1},{a^{\prime }}_{t+1})\right]-\alpha \log (\pi ({a^{\prime }}_{t+1}|s_{t+1}))\right)\right)}^2}$$where $r_t$ represents the immediate return obtained after executing action $a_t$ in state $s_t$, while $\gamma$ is the discount factor, which controls the influence of future returns on the current value. The loss function of the Q Critic network optimizes the network parameters by minimizing the difference between the predicted value and the actual return.

As illustrated in Fig. 5, the architectures of the attacker agent and the defender agent are comprised of a shallow neural network, namely a multi-layer perceptron (MLP). The architecture of the attacker agent consists of two layers with 100 neurons in each layer, while the architecture of the defender agent comprises three layers with 150 neurons in each layer. As can be seen from Fig. 5, both the attacker agent and the defender agent take the features of the current sample as inputs, but produce different outputs depending on the differences in their respective networks. For the attacker agent in the NSL-KDD dataset context, the Actor network generates a probability distribution over 40 traffic types, encompassing normal and attack categories, via its policy function $\pi \left(a|s\right)$. This distribution guides the selection of a specific traffic type, after which a corresponding real sample is extracted from the dataset.

The attacker’s selection process is driven by the reward function defined in “Problem Mapping”, which assigns higher rewards for successfully evading the defender (e.g., when an attack is misclassified as normal traffic). Through iterative adversarial training, the attacker agent optimizes its policy to maximize cumulative rewards, prioritizing attack categories that challenge the defender’s current detection strategy. This adaptive mechanism allows the attacker to learn context-aware selection patterns, such as focusing on minority-class attacks.

For the defender agent, the output is a categorical prediction of the intrusion category, enabled by its three-layer MLP architecture. The defender’s policy is updated in parallel, leveraging the prioritized experience replay buffer to learn from high-impact interactions with the attacker. Together, the attacker’s strategic sample selection and the defender’s adaptive classification form a closed-loop adversarial system, where both agents refine their strategies through reinforcement learning to simulate realistic attack-defense dynamics.

Priority experience replay

SA3C-ID improves the training speed and stability of the algorithm by introducing the prioritized experience replay mechanism into the SAC algorithm. In the network intrusion detection of this article, the state at time $t$ is $s_t$, the action $a_t$ is selected based on the probability distribution policy $\pi \left(a|s_t\right)$ output by the Actor network. The action $a_t$ is input into the state $s_t$ to obtain the reward $r_t$ at the current time step, the next state $s_{t+1}$, and a boolean value $d_t$ indicating whether the interaction between the agent and the environment has reached the terminal state. Meanwhile, each piece of experience is associated with a weight parameter $w_t$ calculated by the temporal-difference error (TD error). The TD error calculation formula is shown in Eq. (13).

(13) $${\delta }_t=Q(s_t,a_t)-(r_t+\gamma (1-d_t)\cdot V(s_{t+1}))$$where ${\delta }_t$ represents the TD Error of the current time step, $Q(s_t,a_t)$ represents the expected return of taking action $a_t$ in state $s_t$, $\gamma$ represents the discount factor, and $V(s_{t+1})$ represents the expected return of the next state $s_{t+1}$. Each piece of experience is stored in the experience replay buffer in the form of a six-tuple $<s_t,a_t,r_t,s_{t+1},d_t,w_t>$. By assigning different priority weights to different experiences, the temporal correlation between samples is broken, and the utilization efficiency of samples is improved. During the agent training process, samples from the replay buffer are randomly selected according to their priorities to update the parameters of the neural network, aiming to enhance the stability and efficiency of learning.

Asynchronous training strategy

The asynchronous training mechanism is designed to tackle dynamic attack-defense dynamics and convergence delays in centralized RL frameworks. During the training process of the SA3C model, multiple worker threads are created. Each thread contains an independent attacker agent, a defender agent, and a shared environment copy. All worker threads execute training tasks concurrently. That is to say, each thread independently interacts with the environment, collects data, and updates the parameters of the policy network and value network of its respective agent. Subsequent to each update, the agent parameters in the worker threads are not immediately synchronized to the main network. Instead, the updated parameters are asynchronously transferred to the main network at an opportune time.

In the asynchronous update method, each worker thread is capable of exploring different parts of the environment independently, thus avoiding the issue of getting trapped in local optima. In this article, the aforementioned approach is used to achieve efficient synchronization of the parameters of the attacker agent and the defender agent, ensuring the stability and effectiveness of the entire training process. Meanwhile, the computing resources of multi-core CPUs are fully utilized to accelerate the training process.

The architecture of the SA3C algorithm is shown in Fig. 6. The integration of the adversarial training mechanisms of the attacker and defender agents enhances the model’s detection accuracy and adaptability in complex network environments. The adoption of an asynchronous update strategy maximizes the utilization of computing resources, significantly reduces the training cycle, and provides scientific decision-making support for real-time network protection.

SA3C-ID model algorithm is shown in Algorithm 1. First, methods such as one-hot encoding and standardization are used to integrate, clean, transform, and standardize the dataset. Second, the SABPIO algorithm is employed to simulate the pigeon flock migration behavior and search for the optimal feature subset in the feature space, eliminating redundant features to reduce the data dimension. Third, the Mini-Batch training strategy is utilized to re-encode the data after feature selection. Then, the re-encoded data is input into the SA3C adversarial intrusion detection module. The pre-processed data is converted into states, actions, and reward values. The prioritized experience replay buffer technology is used to stably and efficiently optimize the network parameters, and the computing resources are fully utilized through asynchronous updates. Finally, an adversarial training mechanism is introduced, where the attacker and defender agents are set up for iterative confrontation. The agents on both sides continually refine their classification strategies through the use of reinforcement learning, ultimately achieving effective detection of intrusion traffic.

Ablation experiments

In order to verify the effectiveness of each module of SA3C-ID model, we conducted ablation experiments based on the NSL-KDD dataset for four models: baseline SAC model, SABPIO +basic SAC model, asynchronous training + basic SAC model, and SA3C-ID model. The experimental results are shown in Table 7.

The ablation experiment based on the NSL-KDD dataset, as shown in Fig. 7, aimed to assess the influence of various components on model performance. The baseline model had an Acc of 0.8415, Pre of 0.8427, DR of 0.8415, F1-score of 0.8397, and a training time of 9,099 s. When the SABPIO module was added to the baseline, the Acc increased to 0.8864, Pre to 0.9573, DR slightly rose to 0.8427, F1-score became 0.8401, and the training time decreased significantly to 4,253 s, indicating that while it improved some metrics and reduced training time, the enhancement in DR and F1-score was limited. Incorporating asynchronous training led to an Acc of 0.9022, Pre of 0.9656, DR of 0.8631, F1-score of 0.8993, and a training time of 7,084 s, which was shorter than the baseline yet longer than with the SABPIO module alone, signifying an improvement in detection ability and performance balance. The SA3C-ID model, combining the baseline SAC model with other optimization modules, achieved the optimal performance with an Acc of 0.9192, Pre of 0.9741, DR of 0.8821, F1-score of 0.9258, and a training time of 3,817 s, thus fully demonstrating the effectiveness of this integrated optimization approach.

Although the SABPIO module did not improve model performance when used alone, its reduced training time suggests its potential in accelerating the training process. Asynchronous training performs well in improving model performance and accelerating the training process. SA3C-ID model achieved significant improvements in diagnosis rate, F1-score and training time compared to the baseline model.

SA3C-ID performance evaluation

This study used the training and test sets of the NSL-KDD dataset to verify the effectiveness of the proposed detection method, and comprehensively evaluated the performance through accuracy, precision, detection rate, and F1-score.

Figures 8A and 8B respectively show the changing trends of the training reward value and training loss of SA3C-ID model on the NSL-KDD dataset. From the experimental results in Fig. 8, it can be observed that as the number of training iterations increases, the training loss of SA3C-ID attack and defense agents continues to decrease and tends to converge after about 80 iterations.

Figures 9A and 9B show the confusion matrix of SA3C-ID model for attack classification in the NSL-KDD dataset. The horizontal axis is the predicted data and the vertical axis is the real data. The classification effect of each type of label is distinguished by the depth of color in the same row. The intrusion detection capability of SA3C-ID can be intuitively observed through the confusion matrix. For normal traffic, SA3C-ID misclassifies only 1.2% of samples as attacks, demonstrating its capability to reduce false alarms, which is a persistent challenge in IDS.

The ROC curve is a graph that illustrates the relationship between TPR and FPR, with a larger area under the curve (AUC) value indicating better performance. As illustrated in Fig. 10, the ROC curve of SA3C-ID model in the NSL-KDD dataset demonstrates that the AUC values of different categories are distinct. Specifically, the Normal class achieved an AUC of 0.977, reflecting strong recognition of normal traffic. For minority attack classes, the User-to-Root (U2R) class, which contains only 54 samples in the training set, achieved an AUC of 0.953; the Remote-to-Local (R2L) class, with 1,126 training samples, achieved an AUC of 0.968. While these values indicate relatively high detection potential for minority classes, the U2R class’s AUC is 2.4% lower than the Normal class, suggesting room for improvement in detecting extremely rare attacks. This gap highlights the challenge of imbalanced data, where models may underperform on classes with insufficient training examples.

Comparative experiment

In this section, we compare SA3C-ID model with other intrusion detection models on the NSL-KDD and CSE-CIC-IDS2018 datasets, including Logistic regression, RF, CNN, LSTM, DQN, DDQN (Lopez-Martin, Carro & Sanchez-Esguevillas, 2020), AE-RL (Caminero, Lopez-Martin & Carro, 2019), A3C (Grondman et al., 2012), AE-SAC (Li et al., 2023), HCRNN (Khan, 2021) and other models. It should be noted that since most models do not give their full training results on the CSE-CIC-IDS2018 dataset, we only compared the accuracy and F1-score in the CSE-CIC-IDS2018 dataset.

Figure 11 shows the comparison results of the SA3C-ID model with other models in the NSL-KDD dataset. The experimental results show that compared with the other nine methods, SA3C-ID model achieves the highest indicators, with accuracy, precision, detection rate, and F1-score reaching 0.9192, 0.9741, 0.8821, and 0.9258 respectively. The F1-score is improved by about 16.6% and 10.3% compared with AE-RL and AE-SAC respectively. SA3C-ID model starts from the problem of sample class imbalance in the intrusion detection dataset, solves the problem of low detection performance of most intrusion detection models for minority class samples, and solves the problem of feature redundancy based on the data itself, thus achieving the best results in the comparative experiment. The detection rate of SA3C-ID in Fig. 11 is lower than that of other indicators because the model reduces the degree of attention to normal samples to a certain extent in order to enhance the detection ability of minority class attack samples.

Figure 12 shows the comparison of the training time of the SA3C-ID model with other models in the NSL-KDD dataset. The results show that compared with general machine learning, deep learning, and reinforcement learning algorithms, the adversarial training algorithms AE-RL, AE-SAC, and SA3C-ID require longer training time. However, since network intrusion detection is trained offline, in actual application scenarios, as long as the training is completed before the model is deployed, it will not interfere with real-time network intrusion detection. At the same time, in practical applications, SA3C-ID only needs to deploy the Actor network, and the network parameters have been determined in the training phase. Therefore, the operating efficiency in practical applications is not affected by the training time. The prediction time of the SA3C-ID model in the NSL-KDD and CSE-CIC-IDS2018 data sets is as follows. In the NSL-KDD test set, with a sample size of 22,543, the prediction time is 0.36 s, which equates to a prediction time of 15.97 microseconds for a single sample. Notably, SA3C-ID’s single-sample inference time (15.97 μs) significantly outperforms traditional RL models (DQN: 58 μs, A3C: 42 μs), meeting the real-time detection requirements of IoT/5G devices (industry standard ≤20 μs). This efficiency stems from SABPIO’s dimensionality reduction and asynchronous parameter updates.

As illustrated in Fig. 13, the SA3C-ID model demonstrates superior performance in comparison to other models within the CSE-CIC-IDS2018 dataset. Compared with the other 7 methods, the SA3C-ID model has the highest accuracy and F1-score. Since most models do not give their full training results on the CSE-CIC-IDS2018 dataset, we only compare the accuracy and F1-score here. In the CSE-CIC-IDS2018 dataset, the accuracy, precision, detection rate, and F1-score of SA3C-ID reached 0.9796, 0.9953, 0.9799, and 0.9876 respectively, and the F1-score was improved by about 5.85% and 1.19% compared with XGBoost and HCRNN, respectively.