A novel network security situation assessment model based on multiple strategies whale optimization algorithm and bidirectional GRU

Establish a hierarchical structure model

A hierarchical structure model was set according to the four indicators (threat, vulnerability, reliability, and availability) established here. The structure chart is shown in Fig. 1.

Construct judgment matrix

Using the hierarchical structure, the relationship judgment matrix was constructed by comparing the importance of each indicator for the situation value. To put it simply, the indicators were judged in pairs. The Santy’s 1–9 scale method is usually used, and the scale method is shown in Table 1.

The relationship judgment matrix constructed is as follows: (1) $$A={\left(a_{ij}\right)}_{4\times 4},a_{ij}>0,a_{ij}={\displaystyle \frac{1}{a_{ji}},a_{ii}=1}$$

Hierarchical sorting

The hierarchical ranking was used to solve the weight of each indicator according to the judgment matrix formed above.

The power root method was used to calculate the weight:

1. Calculate the 1/m power of the product of each line to obtain an m-dimensional vector, M = 4 in this example, and the formula is as follows: (2) $${\omega }_i^{\mathrm{\prime }}=\sqrt[m]{\prod _{j=1}^m{a}_{ij}}$$

2. Normalize the vector to obtain the weight vector, and the formula is as follows: (3) $${\omega }_i={\displaystyle \frac{{\omega }_i^{\mathrm{\prime }}}{{\sum }_{i=1}^m{\omega }_i^{\mathrm{\prime }}}}$$

Calculate the maximum feature root and CI value

After the weight matrix was obtained, the maximum eigenvalue was calculated as follows: (4) $${\lambda }_{\mathrm{m}\mathrm{a}\mathrm{x}}={\displaystyle \frac{1}{n}\sum _{i=1}^n{\displaystyle \frac{{\left(AW\right)}_i}{W_i}}}$$where, $n$ is the dimension number. In this example, the dimension number should be 5.

After calculating the maximum eigenvalue, we calculated the $C.I.$ value. The formula is as follows: (5) $$C.I.={\displaystyle \frac{{\lambda }_{max}}{n-1}}$$

The consistency test

The significance of consistency test is to determine whether the constructed relational judgment matrix has logical problems. $R.I.$ values can be derived from Table 2.

The formula for the $C.R.$ value is as follows. (6) $$C.R.={\displaystyle \frac{C.I.}{R.I.}}$$

After calculating the $C.R.$ value, if $C.R.<0.1$, it means that the relationship judgment matrix $A$ has passed the consistency test. The calculated weight vector may then be applied.

Using this experiment as an example, the relevant factors first need to be identified, which in this experiment refers to determining the appropriate indicators. Then, a hierarchical structure model must be established. The hierarchical structure of this experiment is shown in Fig. 1. The judgment matrix was constructed and is shown in Eq. (7). The maximum eigenvalue of R was calculated as 4.060433616146346, the CI value was 0.020144538715448707, and the CR value was 0.022634313163425512. Because the CR value was less than 0.1, the judgment matrix passed the consistency test. Finally, the hierarchical ranking and consistency tests were conducted. Calculated $W=\left\{“W_T”,“W_V”,“W_R”,“W_U”\right\}$. The situation value was calculated using the calculated weight value, and the formula is shown in Eq. (7). $$R=\left[\begin{array}{cccc}1& 2& 2& 2\\ 1/2& 1& 2& 2\\ 1/2& 1/2& 1& 1\\ 1/2& 1/2& 1& 1\end{array}\right]$$ (7) $$SV=W_{T}T+W_{V}V+W_{R}R+W_{U}U$$

Indicator calculation process

By referring to the CVSS scoring standard and the established indicator system, as well as the indicator measurement method in this field, and after combining network management experience, the following descriptions and calculations are given in this article.

1) Threat: The threat level refers to various external factors that pose a threat to the network system itself. The most significant impact on network security is typically from external threats, leading to service interruptions or data loss. External threats usually encompass various forms of network attack activities and triggered network security incidents. Therefore, in this study, the main indicators selected under the primary indicator of threat level are the severity level of attack types, the number of alerts, and the likelihood of security incidents occurring. The formula is as follows: (8) $$T=\sum _{i=1}^n\sum _{j=1}^k{\displaystyle \frac{{10}^{D_{ij}}{Q}_i{C}_{ij}}{M}}$$where $T$ represents the threat score, $n$ is the total number of hosts in the network system, $k$ is the number of attack types, $M$ is the number of all attacks detected in a period, and $D_{ij}$ is the attack when the i-th host receives the j-th attack the hazard level corresponding to the type, $Q_i$ is the importance of the i-th host, and $C_{ij}$ is the number of times the i-th host receives the attack j.

2) Vulnerability: Vulnerability mainly focuses on describing the inherent security flaws of the network itself. These flaws are primarily manifested in vulnerabilities existing in the network topology, which often become entry points for network attacks. Vulnerability information provides detailed insights into potential security flaws in the network system, including known vulnerability types, potential attack paths, and possible threat levels. On the other hand, the importance of each host reflects its criticality within the network system, indicating how much impact an attack or exploitation of a vulnerability on a specific host could have on the entire system. By considering both factors comprehensively, we can more accurately assess the vulnerability of the network system and identify which vulnerabilities and hosts pose a greater threat to its security. Therefore, vulnerability information and host importance jointly and accurately reflect the network system’s vulnerability and serve as secondary indicators under the primary vulnerability indicator. The formula for this indicator is as follows: (9) $$V=\sum _{i=1}^n\sum _{j=1}^m{Q}_i{S}_{ij}$$where $V$ stands for vulnerability score, $n$ is the total number of hosts in the network system, $m$ is the number of vulnerability types, $D_{ij}$ is the hazard level corresponding to the attack type when the i-th host receives the j-th attack, $Q_i$ is the importance of the i-th host, $S_{ij}$ and is the score of vulnerability j on the i-th host. This value refers to the CVSS vulnerability score.

3) Reliability: Factors affecting reliability in the network primarily revolve around potential issues with the network system architecture and network devices. Firstly, a well-designed network topology can reduce the probability of network interruptions and enhance network reliability. Additionally, the network topology also influences load balancing and capacity planning, ensuring a reasonable allocation of network resources, thus improving the network system’s reliability. Secondly, the number of open ports on devices also has an impact on network system reliability. Open ports serve as communication channels for network devices with the external world, but they also pose potential security vulnerabilities. If too many ports are left open without proper security configuration and management, the network system becomes more vulnerable to malicious attacks and unauthorized access. This could lead to data breaches, service interruptions, or system crashes, thereby lowering the reliability of the network system. Therefore, the number of open device ports is closely related to network system reliability. In conclusion, a well-structured network topology and an appropriately managed number of open ports on devices can enhance the reliability of the network system. In this study, the network topology score and the number of open ports on devices are primarily used to reflect the status of network reliability. The formula is as follows: (10) $$R=W_{TP}TP+W_{PR}PR$$

$W_{TP}$ and $W_{PR}$ are the weights of evaluation factors obtained by AHP.

Network topology score: This index reflects the stability of the topology. The higher the index score, the worse the stability of the network system. The formula is as follows: $$t{p}_i=\{\begin{array}{c}1.0,\begin{array}{cc}& \end{array}0\le br<3\\ 0.5,\begin{array}{cc}& \end{array}3\le br\le 5\\ 0.1,\begin{array}{cc}& \end{array}br>5\end{array}$$ (11) $$TP=\sum _{i=1}^{n}t{p}_i$$where $TP$ represents the network topology score, $br$ represents the number of branch nodes corresponding to each node in the network system. $t{p}_i$ represents the topology score corresponding to the i-th node. Generally speaking, the more branches, the more vulnerable to attack, the lower the topology score, and $n$ represents the number of nodes.

The number of accessible device ports: This indicator reflects the number of open ports of all devices in the network. The higher the indicator score, the more open ports, and the worse the network’s reliability. The indicator formula is as follows:

(12) $$PR=\sum _{i=1}^{n}p{r}_i$$

$PR$ represents the number of open ports on the device, $p{r}_i$ represents the number of open ports on the i-th host, and $n$ represents the number of hosts in the network.

4) Availability: The availability of a network system refers to its ability to function properly and provide the required services during a specific period. In this study, the availability of the current network system was primarily reflected through three indicators: traffic fluctuation rate, data packet distribution, and the frequency of security incidents. Firstly, the traffic fluctuation rate represents the speed of changes in data traffic within the network. A low and stable traffic fluctuation rate indicates that the network load is relatively light and network resources are adequately utilized, thereby improving the network system’s availability. However, if the traffic fluctuation rate is significantly high or fluctuates greatly, it may lead to network congestion, increased latency, and a decline in service quality, affecting the network system’s availability. Secondly, while the methods of network attacks are complex, any form of network attack requires data packets for execution. Therefore, the variation in data packet distribution in the network can reveal whether the network is under attack or experiencing unusual conditions. A sudden increase in traffic from abnormal sources within a specific time period may indicate a potential network attack. Finally, the frequency of security incidents refers to how often security events occur within the network. Security incidents may result in service disruptions, data leaks, or malicious attacks, thereby reducing the network’s availability. In conclusion, the indicators of traffic fluctuation rate, data packet distribution, and the frequency of security incidents reflect different aspects of the current network system’s availability status. The index formula is as follows: (13) $$U=W_{fr}FR+W_{dp}DP+W_{sr}SR$$

$W_{fr}$, $W_{dp}$, and $W_{sr}$ are the weights of evaluation factors calculated by AHP.

Flow rate of change: The formula is as follows: (14) $$FR={\displaystyle \frac{f_t}{f_{t-1}}}$$

In it, $f_t$ represents the traffic size in the current period, and $f_{t-1}$ represents the traffic size in the previous period.

Packet distribution: The formula is as follows: (15) $$DP=ip\ast dtp$$

In the formula, $DP$ represents the distribution of data packets, $ip$ represents the number of different IP addresses, and $dtp$ represents the number of types of different protocols.

The frequency of security incidents: the formula is as follows: (16) $$SR={\displaystyle \frac{se}{e}}$$

$SR$ represents the frequency of security events, $se$ represents the number of security events, and $e$ represents the number of all events.

Opposition-based learning

Whale populations are typically randomly generated. Random generation means that the population may coalesce around the optimal solution, and the optimization algorithm will find the optimal solution very quickly. However, if the randomly generated population is very far from the optimal solution, it takes a long time to arrive at the optimal solution. The traditional population generation method will easily make the optimization algorithm fall into the optimal local solution if there are multiple locally optimal solutions in the search space. We introduced a method called opposition-based learning (Mahdavi, Rahnamayan & Deb, 2018) to prevent the algorithm from falling into an optimal local solution and to improve the optimization efficiency. Opposition-based learning has two primary purposes. One is to speed up the convergence speed of the optimization algorithm. The initial population is generated randomly. Therefore, we can introduce the reverse solution $x_i^{\ast }$ to get better results. If the reverse solution of $x_i$ works better than the initial solution, it should be replaced. This way, the reverse solution is 50% more likely to be closer to the optimal global solution than the current solution. Currently, most research on optimization algorithms is applied to reverse learning processes for initializing populations. However, we not only used opposition-based learning in the population initialization process but also applied it to the iterative process of the population. This method also avoids getting stuck in an optimal local solution. When the optimization algorithm reaches a certain stage, it may fall into a locally optimal solution. Replacing the current population with the reverse solution of the population, there is a certain probability of jumping out of the optimal local solution. Equation (24) is the formula for the multi-dimensional population to find the reverse solution.

(24) $$x_i^{\ast }=a_i+b_i-x_{i}i=1,...,n$$

Here, $x_i\in \left[a_i,b_i\right]$, $a_i$ and $b_i$ are the upper and lower boundaries of the search space, respectively.

After computing the reverse solution, we combine the reverse solution $x_i^{\ast }$ with the original solution $x_i$. Assuming that $fitness\left(x\right)$ is the fitness function in the application scenario, and the number of populations is $N$, combine the reverse solution $x_i^{\ast }$ with the initial solution $x_i$ to generate a combination of solution vectors. Calculate the fitness value of all population individuals and then sort the new population according to the size of the fitness value. In our scenario, we prioritize individuals with small fitness values, so we sort them in ascending order and then select the first N individuals with small fitness values as the new population.

Random adjustment control parameter strategy

In optimization algorithms, striking a good balance between local search and global search capabilities allows the algorithm to maintain a higher search efficiency during the search process (Cuong-Le et al., 2022). Simultaneously, it helps to avoid getting trapped in local optima or encountering premature convergence issues, thus enhancing the algorithm’s ability to find global optimal solutions. This balancing process typically requires adjusting the algorithm’s parameters or designing more sophisticated search strategies to achieve the desired optimization performance.

According to the principle of WOA, the parameter A changes with the change of the convergence factor a, so the value of the convergence factor a plays a vital role in balancing the algorithm’s local search and global search capabilities. In the original WOA algorithm, the convergence factor a in Eq. (17) decreases linearly as the number of iterations increases. This variation of the convergence factor tends to make the algorithm fall into a locally optimal solution. If the convergence factor a has more randomness, it will help the algorithm to jump out of the local optimum. This article proposes a strategy for random adjustment of control parameters, and the formula is as follows: (25) $$a(t)=a_{min}+\sigma \cdot randn$$where $a_{min}$ is the minimum value of the convergence factor $a$, $randn$ is a random number that obeys the normal distribution, and the variance $\sigma$ is used to measure the degree of deviation between the convergence factor $a$ and its mathematical expectation (mean value), which can control the error of the convergence factor $a$. Make it evolve in the direction of desired control parameters.

Levy flight strategy

Levy flight, named after the French mathematician Paul Levy, refers to a random walk in which the probability distribution of the step size is a heavy-tailed distribution (Yang & Wu, 2009). There is a relatively high probability of large strides being taken during random walks. Generally speaking, Levy flight is a search method that combines a small step size search and a large step size search. Here, the spiral update strategy of WOA is replaced by the Levy flight strategy. The Levy flight strategy can make the update position more random, jump out of the optimal local solution quickly, and improve the optimization efficiency of the algorithm. The Levy flight strategy has been widely applied to various optimization algorithms (Minh et al., 2023). The formula of Levy flight is shown in Eq. (26). (26) $$x_i^{t+1}=x_i^t+Levy\left(n\right)\oplus \alpha$$

Here, $\alpha =x_i^t-x_b$, $\oplus$ is the dot product, $x_i^t$ is the $i$ solution of the $t$ generation, $x_b$ is the current optimal solution, $Levy\left(s\right)$ conforms to the Levy distribution, satisfies $Levy\left(s\right)\sim {\left|s\right|}^{-1-\beta }0<\beta \le 2$. The mathematical model of Levy distribution is shown in Eq. (27). (27) $$L\left(s,\gamma ,u\right)=\{\begin{array}{c}\sqrt{{\displaystyle \frac{\gamma }{2\pi }}}exp\left(\left[-{\displaystyle \frac{\gamma }{2\left(s-u\right)}}\right]\right){\displaystyle \frac{1}{{(s-u)}^{3/2}},0<u<s<\mathrm{\infty }}\\ 0,\mathrm{o}\mathrm{t}\mathrm{h}\mathrm{e}\mathrm{r}\mathrm{w}\mathrm{i}\mathrm{s}\mathrm{e}\end{array}$$

In this equation, $\gamma$ and $u$ are greater than 0, $\gamma$ is a proportional parameter, $u$ is a displacement parameter, and the step length of the Levy flight can be expressed as the following formula: (28) $$Levy\left(x\right)=0.01\times {\displaystyle \frac{u\cdot \sigma }{{\left|v\right|}^{1/\beta }}}$$

In it, $u$ and $v$ are standard normal distribution random numbers in [0, 1], $\beta$ = 1.5, and the formula of $\sigma$ is as follows: (29) $$\sigma ={\left\{{\displaystyle \frac{\mathrm{\Gamma }\left(1+\beta \right)\cdot sin\left({\displaystyle \frac{\pi \beta }{2}}\right)}{\mathrm{\Gamma }\left({\displaystyle \frac{1+\beta }{2}}\right)\cdot \beta \cdot 2^{\left(\frac{\beta -1}{2}\right)}}}\right\}}^{{\displaystyle \frac{1}{\beta }}}$$where $\mathrm{\Gamma }\left(x\right)$ is the gamma function, namely $\mathrm{\Gamma }\left(x\right)=\left(x-1\right)!$.

Experimental operating environment and parameter settings

All experiments in this section were conducted using an Intel(R) Xeon(R) PC with CPU E5-2609, 32 GB memory, and 2T hard disk, and the Windows 10 operating system. The operating environment of the program was Python3.9 and pycharm 2021.3.

We used WOA, PSO (Kennedy & Eberhart, 1995), GWO (Mirjalili, Mirjalili & Lewis, 2014), SSA (Mirjalili et al., 2017), and GA (Whitley, 1994) algorithms for comparative experiments in order to verify the effect of the improved WOA optimization algorithm. In the experiment, 12 groups of benchmark functions were selected to verify the effect of the improved optimization algorithm. The 12 benchmark functions include six unimodal, five multimodal, and one fixed-dimensional multimodal function. Unimodal functions, multimodal functions and the fixed-dimensional multimodal functions are shown in Table 3. F1–F6 are unimodal functions. The unimodal function has only one global optimal solution and no optimal local solution that is used to test the algorithm’s convergence and exploration abilities. F7–F11 are multimodal functions with only one global optimal solution and multiple locally optimal solutions, primarily used to test the global search ability of the algorithm. F12 is a fixed-dimensional multipeak test function. The dimension of this function is small, so the optimization pressure is not great. However, the feature of this function is that it is not easy to find the optimal global value, which is mainly used to verify the optimization accuracy of the algorithm. To ensure that there will be no randomness in the experimental results, each optimization algorithm is independently run 30 times on each function. Then the population number of each optimization algorithm is 30, and each optimization algorithm iterates 1,000 times. Finally, the optimal value (Best), the average value (Ave), and the standard deviation (Sd) of each optimization algorithm on the benchmark function are obtained.

Optimal precision analysis

The optimization results of the improved algorithm proposed here and the optimization results of other classic optimization algorithms are shown in Table 4. The hybrid strategy improved WOA and has apparent advantages in convergence accuracy on the benchmark functions F1, F2, F3, F4, and F5, compared with different classical algorithms. The advantages are high convergence accuracy and high stability. On the benchmark function F6, the convergence accuracy of the MSWOA algorithm was significantly higher than that of GWO, GA, PSO, and SSA. However, the optimal value of the convergence accuracy was close to the optimal value of the WOA algorithm, indicating that the optimization potential of WOA on this function was close to the MSWOA algorithm. However, the average and standard deviation of the convergence precision of MSWOA was an order of magnitude lower than that of WOA, indicating that the convergence stability of MSWOA was higher than that of WOA. On the benchmark functions F10 and F11, the convergence accuracy of MSWOA was significantly higher than other optimization algorithms, and the stability was also significantly higher than other algorithms. In functions F7, F8, and F9, the optimal value of the convergence precision of MSWOA was close to the optimal value of other optimization algorithms, indicating that different algorithms can find the optimal global solution under certain conditions. However, the average and standard deviation of the convergence accuracy of MSWOA are smaller than other algorithms, indicating that MSWOA is more stable than other algorithms. In function F12, the optimal value, the average value, and the standard deviation of the convergence accuracy of MSWOA are better than other algorithms, indicating that the optimization performance of MSWOA is excellent.

Convergence speed analysis

First, we analyzed the convergence curves of the optimization algorithm for the functions F1, F2, F3, F4, F5, and F6. Figure 3 shows that the convergence speed of MSWOA in the F1 function was faster than that of GA and SSA, and the convergence speed of MSWOA was slightly faster than that of PSO, WOA, and GWO. On the F2 function, the convergence speed of MSWOA was faster than that of GA, WOA, and PSO but slightly slower than that of SSA. In functions F3 to F5, the convergence speed of MSWOA was faster than that of the other five optimization algorithms. In the F6 function, the convergence speed of MSWOA was faster than that of PSO, GA, and WOA and was close to that of SSA and GWO. The functions F1 to F6 are all unimodal functions. The unimodal function has only one global optimal value and no optimal local value. This shows that the algorithm’s performance on the unimodal function can reflect the convergence ability of the algorithm. In general, the convergence speed of MSWOA was significantly faster than other optimization algorithms for the unimodal function, indicating that the convergence performance of MSWOA is excellent. Then analyze the convergence curves of the optimization algorithm on the F7, F8, F9, F10, F11, and F12 functions. Figure 3 shows that on functions F7 to F12, the convergence speed of MSWOA was significantly faster than that of the other algorithms. F7 to F12 are multimodal functions. The multimodal function has an optimal global value and multiple local optimal values. The algorithm’s performance on the multimodal function may reflect the algorithm’s global search and local optimization capabilities. The analysis shows that MSWOA performed better than other global search and local optimization algorithms, which indicated that the ability of MSWOA’s to solve complex optimization problems was excellent.

Experiment I: comparison of classical machine learning algorithms in evaluation effect

The model proposed in this study is BiGRU model optimized by MSWOA. In order to show that the evaluation effect of the model was excellent, the performance of the basic model BiGRU needed to be tested first. Compared with the GRU model, the average MSE, MAE, RMSE, and R² of the BiGRU model increased by 29.57%, 30.79%, 16.07%, and 8.90%, respectively (Table 8). Compared with the LSTM model, the average MSE and MAE of the regular BiGRU model increased by 21.82%, 25.69%, RMSE increased by 11.57%, and R² increased by 5.74%, respectively. Hence, the evaluation effect of BiGRU was superior to other classical machine learning algorithms. The visual representation of the improvement degree is shown in Table 9.

Experiment II: comparison of evaluation effects of deep learning algorithms based on swarm intelligence optimization algorithms

To test whether the swarm intelligent optimization algorithms are effective for deep learning, experiment II is designed for verification. The model optimized by the optimization algorithm outperformed the normal BiGRU model (Table 10). Compared with the GRU model based on PSO optimization, the MSE, MAE, RMSE, and R² of the GRU model were improved by 19.45%, 3.38%, 10.26%, and 3.78%, respectively, and the MSE, MAE, RMSE, and R² of the GRU model based on WOA optimization were improved by 14.86%, 2.15%, 7.72%, 2.89%, and the MSE, MAE, RMSE, R² of the BiGRU model based on PSO optimization were improved by 19.17%, 5.07%, 10.10%, 3.73%, respectively. The MSE, MAE, RMSE, and R² of the BIGRU model based on WOA optimization increased by 32.09%, 9.17%, 17.59%, and 6.24%, respectively. The MSE, MAE, RMSE, and R² of the MSWOA-optimized BIGRU model increased by 44.98%, 18.25%, 25.83%, and 8.75%, respectively. Therefore, it can be concluded from experiment II that the performance of the deep learning algorithm optimized by the swarm intelligence optimization algorithm performs better than the unoptimized deep learning algorithm for cyber security situation assessment. The improvement percentage is shown in Table 11.

Experiment III: performance comparison between optimization algorithms

The purpose of designing this part of the experiment is to test the performance of different optimization algorithms and compare them. There are currently many popular optimization algorithms. Since many network parameters need to be debugged during the training process of the neural network, it is more suitable for the swarm intelligence optimization algorithm to optimize them. MSWOA was used in our study to optimize the suggested model. The representative PSO algorithm and WOA algorithm in the swarm intelligence optimization algorithm were selected for comparison, and the performance is shown in Table 12. Comparing the BIGRU model based on MSWOA optimization with the BIGRU model based on PSO optimization, Table 13 shows that MSE, MAE, RMSE, and R² increased by 31.93%, 13.88%, 17.50%, and 4.84%, respectively. Compared to the WOA-optimized BIGRU model, MSE, MAE, RMSE, and R² of the MSWOA-optimized BIGRU model increased by 18.98%, 10.00%, 10.00%, and 2.36%, respectively. The effect of the MSWOA optimization evaluation model was better than the effect of the PSO and WOA optimization evaluation models.

Experiment IV: comparison of NSSA effects

As shown in Fig. 10, the changing trend of the network security situation value calculated by the eight models is roughly the same as that of the actual network security situation value. However, a BiGRU model based on MSWOA optimization for evaluating the network security situation was proposed and the varying trend of the value was more consistent with the changing direction of the actual value. Figure 10 shows the following results:

The network security situation value calculated by the MSWOA-BiGRU model was almost in the same situation value range as the actual situation value. Except at the 18th sample point, the model result is medium, but the real situation is a high risk. The rest of the samples were all correctly evaluated.

The calculation results of the eight models at the 10th and 12th sample points were in the same range as the real situation value, but the model proposed in this article is closer to the real situation value.

To more intuitively compare the proximity between the situation value calculated by the model and the actual situation value, the evaluation results of some samples were randomly extracted from the test samples in this study. The comparison results are shown in Table 14 and the situation value calculated by the MSWOA-BiGRU model and the actual situation value are basically within the same evaluation level range. These results confirm that the proposed model in this study can accurately reflect the current network security situation.

Experiment V: comparison with existing baselines

We compared the proposed method, MSWOA-BiGRU, with several other works related to network security situation assessment. The baseline methods used for comparison are as follows:

AE-PMU (Tao, Liu & Yang, 2021): Utilizing the autoencoder (AE) for data dimensionality reduction, followed by the application of the PMU deep neural network to achieve accurate and efficient NSSA.

AEDNN (Yang et al., 2021): Using the deep autoencoder (DAE) for data dimensionality reduction, and then employing the deep neural network (DNN) to calculate the network security status value.

GA-BP (Wen & Wang, 2017): Calculating the network security status value using a BP neural network optimized by GA.

BiPMU (Liu et al., 2023): Using bidirectional PMU (BiPMU) to evaluate the network security situation value.

PSO-WNN (Zhao & Liu, 2018): Using Particle Swarm Optimization (PSO) to optimize the parameters of the Wavelet Neural Network (WNN), and then applying the wavelet neural network based on particle swarm optimization to calculate the situation value.

As shown in the Table 15, compared with AE-PMU, the MSE, RMSE, MAE, and R² values increased by 63.37%, 39.50%, 55.58%, and 23.61% respectively. Compared with AEDNN, the MSE, RMSE, MAE, and R² values increased by 64.90%, 40.73%, 56.65%, and 25.53% respectively. Compared with GA-BP, the MSE, RMSE, MAE, and R² values increased by 67.31%, 42.83%, 58.16%, and 29.37% respectively. Compared with BiPMU, the MSE, RMSE, MAE, and R² values increased by 59.47%, 36.31%, 53.04%, and 19.27% respectively. Compared with PSO-WNN, the MSE, RMSE, MAE, and R² values increased by 43.28%, 24.74%, 43.94%, and 9.22% respectively. In our experiments, the MSWOA-BiGRU model outperformed the baseline models significantly, especially showing a noticeable improvement in situation assessment performance. Specifically, this performance boost can be primarily attributed to the BiGRU’s fusion of traffic features with extracted temporal features, as well as the excellent optimization performance of MSWOA.

Experiment VI: robustness analysis

To ensure the robustness of our experimental results, we tested the proposed method for robustness from two perspectives. Firstly, we conducted three experiments on the model. The standard deviation of the results is summarized in the Table 16, and the results in the Table 16 indicate a significantly small standard deviation. Then, to further ensure the robustness of the experimental results, we added noise data to the dataset at proportions of 1%, 3%, 5%, 7%, and 10% of the original dataset to test the model’s resistance to interference. The standard deviation of the results is summarized in the Table 17. When adding 7% to 10% noise data, the model’s performance experienced a noticeable decline, but the standard deviation remained small. Through robustness testing of the proposed network security posture assessment method from two perspectives and analyzing the results, it is demonstrated that our model is robust, and the reported results can be considered reliable.

Experiment VII: statistical soundness analysis

To ensure the statistical soundness of our results, we employed the Kolmogorov-Smirnov test and the Spearman coefficient for testing.

Firstly, the Kolmogorov-Smirnov test can determine whether the samples of situation assessment values and the true situation values come from the same distribution. If the two sets of samples are from the same distribution, it preliminarily suggests that the members or observations of the two sets are homogenous to some extent, or they might be influenced by similar factors or conditions. The result of the KS test is 0.071429, with a p-value of 0.869191. The experimental results show that if the p-value is less than 0.05, we do not reject the null hypothesis, concluding that the two distributions come from the same distribution.

Next, we used the Spearman coefficient to test the consistency between the samples of situation assessment values and the samples of true situation values. The Spearman coefficient is 0.920400, with a p-value of 3.83536E−58. If the p-value is less than 0.05, we reject the null hypothesis, indicating a significant Spearman correlation between the two sets of samples. Viewing the samples of posture assessment values and the true situation values as two time series, a significant Spearman correlation suggests that the two time series have a strong association. This further validates the excellent assessment performance of the method we proposed.