RA-QoS: a robust autoencoder-based QoS predictor for highly accurate web service QoS prediction

Collaborative filtering-based QoS prediction

Collaborative filtering (CF) stands as a leading prediction method in recommendation systems and is extensively utilized in Web service QoS prediction (Zheng et al., 2020). CF leverages historical data on user behavior to detect similarities among users or items and forecast user preferences for items, incorporating quality assessments from known services to estimate for unseen users. Due to its effectiveness in capturing the dynamics of user and service characteristics, CF-based methods have enabled personalized QoS predictions for service users. Several influential models have emerged from this approach, including kernel least-mean-square (KLMS) by Luo et al. (2016), embedding and factorization methods by Wu et al. (2017), matrix factorization by Zheng & Lyu (2013), location-based factorization by Yang et al. (2018b), and tensor techniques within a five-dimensional QoS framework by Wang et al. (2016). These models demonstrate strengths in user-service dynamics but often lack comprehensive solutions to handle complex, nonlinear relationships and varying biases.

DNN-based QoS prediction

Deep neural networks (DNNs), with their strong nonlinear learning capabilities, have also gained traction for QoS prediction. Notable models include recurrent networks for classification and regression by Chen et al. (2019), a stacked denoising autoencoder for long-tail services by Bai et al. (2017), and the hybrid NRR model by Li et al. (2017). More recent work includes GNN-based CF models by Zhang & Chen (2019) and Ying et al. (2018), which exploit user-item graph preferences. While these methods improve QoS prediction accuracy, they fall short in addressing biases and outliers in QoS data comprehensively.

DNN-based models incorporating partial debiasing (e.g., Schnabel et al., 2016; Saito, 2020; Yuan, Luo & Shang, 2018) or combining L1 and L2 norms (e.g., Zhu et al., 2018; Raza & Ding, 2020; Wu et al., 2023; Huang et al., 2023) have been introduced, yet they lack robustness to diverse biases and outliers. Existing L1 and L2 norm-based methods address specific biases but struggle with the instability and non-uniqueness issues associated with the L1 norm.

In response to these limitations, this study proposes a novel RA-QoS predictor with a robust loss function combining preprocessing bias (PB), training bias (TB), and both L1 and L2 norms within an autoencoder framework. This approach enables comprehensive handling of biases and improved robustness against outliers, thereby enhancing prediction accuracy.

The comparison of related QoS prediction methods are summerized in Table 1.

Autoencoder input and output

The RA-QoS model (shown in Fig. 2) utilizes an autoencoder architecture designed to predict missing QoS values. The input to the autoencoder, denoted as $y^{(q)}$, represents observed QoS data from the user-service matrix, specifically response time or throughput values for a given user-service pair. This input data may be sparse due to the incomplete nature of user-service interactions.

The autoencoder’s objective is to learn a compressed representation of the input data through its hidden layers, which is then used to reconstruct or predict the output. The output of the autoencoder, denoted as ${\hat{y}}^{(q)}$, is the predicted QoS value corresponding to the input. The model is trained to minimize the difference between the actual QoS value (from the observed data) and the predicted value ${\hat{y}}^{(q)}$, allowing it to predict missing values effectively.

The architecture of the RA-QoS model is shown in Fig. 3. In the model, the input layer is on the left, denoted by $y^{(q)}$. The input data then passes through a series of hidden layers ( $l_1,l_2,\dots ,l_k$), where it undergoes transformations by applying a series of weights ( $w_k$) and biases ( $b_k$), as well as nonlinear activation function such as Sigmoid function. The result of each transformation is passed to the next layer until reaching the final layer ( $l_k$). From there, outputs are generated to make predictions, denoted as ${\hat{y}}^{(q)}$. Note the parameters in the $\varphi (TB)$ are trainable. We adopt the $\varphi (TB)$ and $\psi (PB)$ to represent TB and PB combinations.

Calculation of PB and TB

The preprocessing bias (PB) combinations denote to preprocess the observed user behavior data before training. They comprises three components: $Pa$, $P{i}^{(n)}$ and $P{u}^{(m)}$, representing the global average rating, service bias, and user bias, respectively. These combinations are varied using $z_1$, $z_2$ and $z_3$ to adjust $Pa$, $P{i}^{(n)}$ and $P{u}^{(m)}$ independently. The global average rating $Pa\in {\mathbb{N}}^{+}$ reflects the statistical properties of observed QoS ratings in the user-service matrixQ. It can be obtained as follows:

(4) $$Pa=z_1\cdot \frac{{\sum }_{mn\in \mathrm{\Gamma }}{q}_{mn}}{{\sum }_{mn\in \mathrm{\Gamma }}{d}_{mn}},$$where $\mathrm{\Gamma }\in {\mathbb{R}}^{|M|\times |N|}$ represents the training set that is a subset of the matrix Q. The preference of all users for a service $s$ can be estimated using $P{i}^{(n)}$ with the equation as follows:

(5) $$P{i}^{(n)}=z_2\cdot \frac{{\sum }_{m\in \mathrm{\Gamma }(n)}{d}_{mn}\cdot \left(q_{mn}-Pa\right)}{{\omega }_1+{\sum }_{m\in \mathrm{\Gamma }(n)}{d}_{mn}},$$where $\mathrm{\Gamma }(n)$ represents the $n$-th column of $\mathrm{\Gamma }$, and ${\omega }_1$ denotes the threshold constant associated with the average rating of service $s$. The preference of a user for different services $Pu$ can be noted as:

(6) $$P{u}^{(m)}=z_3\cdot \frac{{\sum }_{n\in \mathrm{\Gamma }(m)}{d}_{mn}\cdot \left(q_{mn}-Pa-P{i}^{(n)}\right)}{{\omega }_2+{\sum }_{n\in \mathrm{\Gamma }(m)}{d}_{mn}}.$$

Table 2 summarizes all the possible combinations of PB. Moreover, $Pi$ and $Pu$ denote the PB vectors for all services and all users, respectively.

The training bias (TB) operates in conjunction with the training process, unlike PB. TB consists of two components: $Ti(s)$ and $Tu(u)$, which extract user $u$’s and service $s$’s preferences, respectively, during training. Similarly, we employ $z_4$ and $z_5$ to modulate the values of $Ti(s)$ and $Tu(s)$ to create various TB combinations, and $Ti$ and $Tu$ represent the TB vectors for all services and all users, respectively. A summary of all possible TB combinations is provided in Table 3.

The combinations of PB and TB

The RA-QoS model adjusts the combination of PB and TB to balance the influence of bias. In this model, the $l_k$ is used to represent the $k$-th hidden layer of RA-QoS, $k\in \{1,2,\dots ,K\}$. The term $w_k$ represents the weighted term of $l_k$, and the term $b_k$ represents the weighted term of $l_k$. Therefore, RA-QoS’s prediction function can be expressed as follows:

(7) $$\begin{array}{c}\begin{array}{c}{\mathbf{y}}^{{\left(q\right)}^{\mathrm{\prime }}}={\mathbf{y}}^{\left(q\right)}-\psi \hfill \\ l_1=g\left(w_1\cdot {\mathbf{y}}^{{(q)}^{\mathrm{\prime }}}+b_1\right)\hfill \\ l_k=g\left(w_k\cdot l_{k-1}+b_k\right),k=2,\cdot \cdot \cdot ,K-1\hfill \\ {\hat{\mathbf{y}}}^{{(q)}^{\mathrm{\prime }}}=w_K\cdot l_{K-1}+b_K+\varphi \hfill \\ {\hat{\mathbf{y}}}^{(q)}=f\left({\mathbf{y}}^{(q)};\theta ,\varphi ,\psi \right)=h\left(l_K+\psi \right)\hfill \end{array}\hfill \\ \{\begin{array}{c}\mathrm{I}\text{-}\mathrm{R}\mathrm{A}\text{-}\mathrm{Q}\mathrm{o}\mathrm{S}:\{\begin{array}{c}{\mathbf{y}}^{(q)}={\mathbf{y}}^{(n)}=\left\{y_{1n},\cdots ,y_{|M|n}\right\}\hfill \\ \psi =Pa+P{i}^{(n)}+Pu\hfill \\ \varphi =T{i}^{(n)}+Tu\hfill \end{array}\hfill \\ \mathrm{U}\text{-}\mathrm{R}\mathrm{A}\text{-}\mathrm{Q}\mathrm{o}\mathrm{S}:\{\begin{array}{c}{\mathbf{y}}^{(q)}={\mathbf{y}}^{(m)}=\left\{y_{m1},\cdots ,y_{m|N|}\right\}\hfill \\ \psi =Pa+Pi+P{u}^{(m)}\hfill \\ \varphi =Ti+T{u}^{(m)}\hfill \end{array}\hfill \end{array}\end{array}.$$

The model is optimized by minimizing the following objective function:

(8) $$\{\begin{array}{cc}\mathrm{I}\text{-}\mathrm{R}\mathrm{A}\text{-}\mathrm{Q}\mathrm{o}\mathrm{S}:\{\begin{array}{cc}L(f)={\sum }_{\genfrac{}{}{0ex}{}{{\mathbf{y}}^{(n)}\in Y}{{\mathbf{d}}^{(n)}=D}}{\left(\left({\mathbf{y}}^{(n)}-{\hat{\mathbf{y}}}^{(n)}\right)\odot {\mathbf{d}}^{(n)}\right)}^2\hfill & \hfill \\ +\frac{{\lambda }_1}{2}\cdot \left({\sum }_{k=1}^K{\left(w_k\right)}^2\right)+\frac{{\lambda }_2}{2}\cdot {\left(T{i}^{(n)}+Tu\right)}^2\hfill & \hfill \end{array}\hfill & \hfill \\ \mathrm{U}\text{-}\mathrm{R}\mathrm{A}\text{-}\mathrm{Q}\mathrm{o}\mathrm{S}:\{\begin{array}{cc}L(f)={\sum }_{\genfrac{}{}{0ex}{}{{\mathbf{y}}^{(m)}\in Y}{{\mathbf{d}}^{(m)}=D}}{\left(\left({\mathbf{y}}^{(m)}-{\hat{\mathbf{y}}}^{(m)}\right)\odot {\mathbf{d}}^{(m)}\right)}^2\hfill & \hfill \\ +\frac{{\lambda }_1}{2}\cdot \left({\sum }_{k=1}^K{\left(w_k\right)}^2\right)+\frac{{\lambda }_2}{2}\cdot {\left(Ti+T{u}^{(m)}\right)}^2\hfill & \hfill \end{array}\hfill & \hfill \end{array}$$where ${\lambda }_1$, ${\lambda }_2$ are the regularization rate hyper-parameters. To minimize the function $L(f)$ and train the learnable parameters $\mathrm{\Theta }=\{w_1,b_1,\dots ,w_K,d_k,Tu,Ti\}$ above, we adopted Adam (Kingma & Ba, 2014) to adapt the learning rate automatically to train $\mathrm{\Theta }$.

Self loss function in RA-QoS

The L2-norm loss function is sensitive to outliers in QoS data for each user, while the L1-norm loss function lacks stability during training. To address this, we combine them into a self-adaptively weighted L1-and-L2-norm-oriented loss function in RA-QoS model. Thus, the objective function with L1-and-L2norm-oriented loss function of RA-QoS becomes:

(9) $$\{\begin{array}{c}\mathrm{I}\text{-}\mathrm{R}\mathrm{A}\text{-}\mathrm{Q}\mathrm{o}\mathrm{S}:\{\begin{array}{cc}L(f)={\sum }_{\genfrac{}{}{0ex}{}{{\mathbf{y}}^{(n)}\in \mathrm{Y}}{{\mathbf{d}}^{(\mathit{n})}=D}}{\gamma }_1\cdot |\left({\mathbf{y}}^{(n)}-{\hat{\mathbf{y}}}^{(n)}\right)\odot {\mathbf{d}}^{(n)}|+{\gamma }_2\cdot {\left(\left({\mathbf{y}}^{(n)}-{\hat{\mathbf{y}}}^{(n)}\right)\odot {\mathbf{d}}^{(n)}\right)}^2\hfill & \hfill \\ +\frac{{\lambda }_1}{2}\cdot \left({\sum }_{k=1}^K{(w_k)}^2\right)+\frac{{\lambda }_2}{2}\cdot {\left(T{i}^{(n)}+Tu\right)}^2\hfill & \hfill \end{array}\hfill \\ \mathrm{U}\text{-}\mathrm{R}\mathrm{A}\text{-}\mathrm{Q}\mathrm{o}\mathrm{S}:\{\begin{array}{cc}L(f)={\sum }_{\genfrac{}{}{0ex}{}{{\mathbf{y}}^{(m)}\in \mathrm{Y}}{{\mathbf{d}}^{(m)}=D}}{\gamma }_1\cdot |\left({\mathbf{y}}^{(m)}-{\hat{\mathbf{y}}}^{(m)}\right)\odot {\mathbf{d}}^{(m)}|+{\gamma }_2\cdot {\left(\left({\mathbf{y}}^{(m)}-{\hat{\mathbf{y}}}^{(m)}\right)\odot {\mathbf{d}}^{(m)}\right)}^2\hfill & \hfill \\ +\frac{{\lambda }_1}{2}\cdot \left({\sum }_{k=1}^K{(w_k)}^2\right)+\frac{{\lambda }_2}{2}\cdot {\left(Ti+T{u}^{(m)}\right)}^2\hfill & \hfill \end{array}\hfill \end{array}$$where ${\gamma }_1$ and ${\gamma }_2$ denote the aggregation weights. They self-adaptively regulate the influence of L1-norm and L2-norm, respectively. Note that to maintain the magnitude of loss, ${\gamma }_1+{\gamma }_2=1$, and ${\gamma }_1\cdot {\gamma }_2\ge 0$. The aggregation strategy has the following design strategy: increasing ${\gamma }_1$ if the partial loss of L1-norm is lower than L2-norm, otherwise increasing ${\gamma }_2$. In training process, let ${\delta }_1^{t}and{\delta }_2^t$ denote the partial losses of $L_1$-norm, $L_2$-norm and let ${\delta }_{1,2}^t={\delta }_1^t+{\delta }_2^t$. Let $A_1^t={\sum }_{j=1}^t{\delta }_1^j$, $A_2^t={\sum }_{j=1}^t{\delta }_2^j$ and $A_{1,2}^t={\sum }_{j=1}^t{\delta }_{1,2}^j$.

L1-L2 norm fusion mechanism and its contribution to robustness

The RA-QoS model incorporates a self-adaptive fusion mechanism between the L1-norm and L2-norm, which significantly contributes to its robustness in the presence of noise and irregular data. The aggregation weights ${\gamma }_1$ and ${\gamma }_2$ govern the influence of the L1-norm and L2-norm, respectively, with the constraint ${\gamma }_1+{\gamma }_2=1$ and ${\gamma }_1\cdot {\gamma }_2\ge 0$. This mechanism allows the model to dynamically adjust its focus between the two norms based on the current training stage.

The design strategy of this fusion mechanism is as follows: when the partial loss of L1-norm ${\delta }_1^t$ is lower than the L2-norm loss ${\delta }_2^t$, the model increases ${\gamma }_1$, favoring the L1-norm to better handle sparse or noisy data. Conversely, if the L1-norm loss is larger, the model increases ${\gamma }_2$, which places more emphasis on the L2-norm. This self-adaptive strategy allows the model to dynamically adjust its sensitivity to noise and outliers throughout the training process.

At each training iteration, the partial losses of the L1-norm ( ${\delta }_1^t$) and L2-norm ( ${\delta }_2^t$) are computed, and their cumulative sums are updated as follows:

$$A_1^t=\sum _{j=1}^t{\delta }_1^j,A_2^t=\sum _{j=1}^t{\delta }_2^j,A_{1,2}^t=\sum _{j=1}^t({\delta }_1^j+{\delta }_2^j).$$

This mechanism ensures that the model adapts its loss function dynamically during training. The aggregation of the L1-norm and L2-norm helps to balance the model’s sensitivity to sparse, noisy, or outlier data points while maintaining stability in learning.

As a result, this self-adaptive fusion mechanism makes the RA-QoS model particularly robust when confronted with noisy, sparse, or imbalanced data. By giving more weight to the L1-norm when dealing with outliers and noise, and adjusting to the L2-norm when the data is more consistent, the model improves its generalization and performance under challenging conditions. This enhanced robustness ensures that the RA-QoS model performs stably even when faced with imperfections in the data, making it an excellent choice for real-world QoS prediction tasks.

The detailed algorithm is shown in Algorithm 1.

Time and space complexity

Providing |M| inputs to the hidden units of I-RA-QoS requires |M| multiplications and $|M|+1$ additions to account for bias terms and activation functions. The computational demand of these operations is linear, represented as $O(|M|)$. If I-RA-QoS has $x$ hidden units in its first layer, each receiving |M| inputs, the complexity for this layer is calculated as $x\times O(|M|)$. With two hidden layers, the total time complexity of I-RA-QoS is $x\times O(|M|)+O(|M|)\times x\times O(|M|)$, simplifying to $O(|M{|}^2\times x)$. Assuming $x$ remains constant, the complexity stabilizes at $O(|M{|}^2)$. Similarly, the time complexity for U-RA-QoS can be deduced to be $O(|N{|}^2\times x)$ and $O(|N{|}^2)$, using the same computational framework.

RA-QoS only needs to keep a version of the input matrix and additional record information with similar dimensions. In contrast, I-RA-QoS requires $O(|M|\times K)$ space to store its parameters. Consequently, the space complexity for I-RA-QoS is $O(|M|\times |N|+|M|+K)$. Given that N typically far exceeds K, the space complexity for I-RA-QoS simplifies to $O(|M|\times |N|)$, a simplification that also holds true for U-RA-QoS.

Quality criteria for services

In this study, we focus on two commonly used quality of service (QoS) metrics: response time and throughput. Response time is defined as the time taken from when a user sends a request until the service responds, and it reflects the promptness of service delivery. Throughput is defined as the rate at which data is successfully processed by a service over a specific period, indicating the capacity of the service to handle user demands. Both metrics are obtained from the WS-Dream dataset (Zheng et al., 2010), where they were recorded by users interacting with various web services. To evaluate and compare the prediction accuracy of the proposed RA-QoS model, these metrics serve as target values in our prediction model. Errors between the predicted and actual values of these metrics are calculated using mean absolute error (MAE) and root mean squared error (RMSE), as detailed below.

Platform and environment

The experiments were conducted on a computing platform equipped with Intel(R) Core(TM) i7-10510U CPU @ 1.80 GHz 2.30 GHz, IVIDIA RTX3090 GPU and 16.0 GB RAM. The RA-QoS model and baseline algorithms were implemented using Python and several machine learning libraries, including tensorflow for deep learning functionality. Experiments were executed with additional packages for data processing and evaluation, such as NumPy. The source code for experiment is available at this URL (https://gitee.com/sonica/RA-QoS).

Datasets

In this work, the WS-Dream dataset (Zheng et al., 2010) was employed for model experimentation. This dataset is a real-world web service dataset that has been frequently utilized in prior studies. The experiment utilized two benchmark QoS datasets: Response Time (RT) and Throughput (TP). The Response Time dataset contains 1,873,838 records, while the Throughput dataset contains 1,831,253 records. Both datasets were generated by 339 users across 5,825 distinct services.

For each experiment, different ratios of training data were used, as indicated by the ID in the table. Specifically, the RT in the ID column refers to the Response Time dataset, and the TP in the ID column refers to the Throughput dataset. The number following the underscore in the ID (e.g., RT_05) denotes the ratio of training data used in that particular experiment set, where 05 corresponds to 5% of the dataset used for training.

Both datasets were partitioned into training and testing sets at four different ratios: 5%, 10%, 15%, and 20%. The exact number of samples used for training and testing at each ratio is provided in Table 4.

Evaluation of experiments

The task of QoS prediction involves forecasting missing data in sparse QoS matrices. Prediction, by its nature, cannot achieve 100% accuracy; thus, there always exists an error between predicted and actual values. Therefore, we utilize this error to evaluate the accuracy of QoS prediction on the test set. MAE and RMSE are usually used as the error evaluation metrics. MAE addresses the issue of error cancellation, accurately reflecting the magnitude of prediction errors. RMSE is sensitive to larger or smaller errors in the test set, effectively reflecting the robustness of predictions. Hence, this study selects RMSE and MAE as evaluation metrics for assessing predicted QoS values. The specific formulas are as follows:

(10) $$\mathrm{M}\mathrm{A}\mathrm{E}=\frac{{\sum }_{u,s}\left|Q_{u,s}-{\hat{Q}}_{u,s}\right|}{N}$$

(11) $$\mathrm{R}\mathrm{M}\mathrm{S}\mathrm{E}=\sqrt{\frac{{\sum }_{u,s}{(Q_{u,s}-{\hat{Q}}_{u,s})}^2}{N}}$$where $Q_{u,s}$ represents the actual values in the test set, and ${\hat{Q}}_{u,s}$ represents the predicted values given by the proposed RA-QoS model.

Baselines

To validate the performance of the RA-QoS model in web service quality prediction, this study compares four mainstream QoS prediction methods:

Robust Sparse Non-negative Matrix Factorization (RSNMF) (Peng et al., 2018): RSNMF is utilized for the decomposition of user-item rating matrices, facilitating personalized recommendations. By learning feature representations of users and items, RSNMF captures user interests and item attributes, thus achieving accurate recommendations.

Non-negative Alternating Matrix Factorization (NAMF) (Li et al., 2022): NAMF is also a matrix factorization technique characterized by its alternating update optimization process.

Data-Characteristic-Aware Latent Factor (DCALF) (Wu et al., 2022): DCALF is a latent factor model used in recommendation systems, considering the influence of data characteristics. The DCALF model enhances latent factor models by incorporating additional data features, thus better capturing the relationships between users and items.

Autoencoders Meet Collaborative Filtering (AutoRec) (Sedhain et al., 2015): AutoRec is a collaborative filtering model based on autoencoders, utilized for personalized recommendation tasks in recommendation systems. It employs autoencoders to learn the hidden structure within user-item rating matrices, thereby facilitating the learning and representation of user interests and item features.

Deep Learning-based Communication Adaptive Network (DLCANet) (Kim et al., 2024): DLCANet is a deep learning-based model specifically designed for QoS prediction in robotic communication networks. By integrating CNN, LSTM, GNN, and attention mechanisms, DLCANet captures spatial, temporal, and topological features of communication data, making it highly effective for dynamic and complex network environments. Through meticulous data preprocessing and a custom loss function that emphasizes critical QoS metrics like latency and bandwidth.

Results and discussion

In this work, we evaluated the performance of seven algorithms—RSNMF, NAMF, GeoMF, DCALF, AutoRec, DLCANet, and RA-QoS—on two benchmark QoS datasets: RT and TP. The experiments were conducted across varying training data ratios, and the evaluation metrics used were MAE and RMSE. The detailed results are summarized in Tables 5 and 6.

As introduced in “The Proposed Method: Ra-qos Model”, the RA-QoS framework supports two input modes: I-RA-QoS (item-based) and U-RA-QoS (user-based), each offering different perspectives of QoS matrix completion. To determine the most effective variant for practical deployment, we conduct a comprehensive comparison of both I-RA-QoS and U-RA-QoS across multiple datasets and under various combinations of Preprocessing Bias (PB) and Training Bias (TB). Experimental results consistently demonstrate that I-RA-QoS achieves better prediction accuracy, stronger robustness to sparsity, and faster convergence than U-RA-QoS. Therefore, we adopt I-RA-QoS with PB.2 and TB.2 as the default configuration of the final RA-QoS model, ensuring high accuracy and effective debiasing while preserving architectural simplicity.

Table 5 shows the MAE results, where RA-QoS consistently demonstrates strong performance, often achieving the lowest MAE values. For the RT dataset, RA-QoS outperforms all models in most cases, with the lowest MAE of 0.3835 at RT_20, demonstrating its ability to minimize errors. In the TP dataset, RA-QoS remains competitive, outperforming the other models except in a few instances, such as at TP_20 where DCALF slightly surpasses it (MAE = 13.5491 for DCALF vs. 14.5654 for RA-QoS). The Win/Loss analysis highlights RA-QoS’s effectiveness, achieving 40 wins against only eight losses, the second-best performance after DCALF (44 wins, four losses).

The RMSE results in Table 6 further validate the superiority of RA-QoS in both datasets. For RT, RA-QoS achieves an RMSE of 1.1573 at RT_20, which is marginally higher than GeoMF’s RMSE of 1.1528 but still ranks among the best-performing models. In the TP dataset, RA-QoS consistently delivers competitive results, achieving RMSE values close to DCALF, which leads in several scenarios (e.g., TP_20, RMSE = 41.2194 for DCALF vs. 42.5971 for RA-QoS). The overall win/loss analysis shows RA-QoS securing 33 wins and 15 losses, confirming its robustness compared to most other models.

The Friedman Rank analysis, presented in Tables 5 and 6, offers a comprehensive view of the models’ performance. RA-QoS ranks second for MAE (rank = 2.286) and third for RMSE (rank = 3.286), reflecting its consistent accuracy and reliability. In contrast, models like RSNMF and NAMF generally exhibit weaker performance, with ranks of 5.000 and 5.571 for MAE and 6.429 and 4.571 for RMSE, respectively.

The results highlight RA-QoS’s ability to achieve close-to-optimal predictions across varying data sparsity levels. While DCALF occasionally outperforms RA-QoS, especially in low-sparsity conditions, RA-QoS demonstrates competitive performance with minimal differences in accuracy. For instance, at TP_20, the MAE and RMSE values for RA-QoS are within a narrow margin of DCALF’s results, showcasing RA-QoS’s robustness.

Additionally, RA-QoS strikes a balance between accuracy and versatility, performing well across both RT and TP datasets, regardless of the training data ratio. This versatility makes it a strong candidate for real-world QoS prediction tasks, where data sparsity and distribution vary significantly. While DCALF emerges as the top-performing model in specific scenarios, RA-QoS delivers highly competitive results with a consistent edge over most other models. Its strong performance across different metrics, combined with its adaptability, positions RA-QoS as a reliable and effective model for QoS prediction tasks.

To provide a more intuitive comparison, Fig. 4 contrasts the experimental results of the AutoRec model and the proposed RA-QoS model. It can be seen that RA-QoS consistently exhibits lower RMSE and MAE compared to AutoRec across both datasets, indicating its superior predictive performance. This highlights the advantage of the RA-QoS approach in improving the prediction of web service quality, making it a valuable enhancement over the original AutoRec methodology.

Conclusion. Overall, the experimental results on both RT and TP datasets validate the effectiveness of RA-QoS. It consistently achieves lower MAE and RMSE than baseline models under various data sparsity settings. Although DCALF slightly surpasses RA-QoS in a few individual cases, RA-QoS offers more stable performance overall, especially in highly sparse settings. This demonstrates its superior generalization and robustness, largely attributed to the integration of bias correction and hybrid loss design.

Parameter sensitivity and convergence

In the experiment, achieving better results, i.e., lower RMSE and MAE, was not merely a matter of using the model directly. Instead, it involved numerous rounds of hyperparameter tuning, model adjustments, and comparisons with other factors affecting the prediction results. Therefore, this chapter will delve into the impact of various factors related to deep neural networks, including the network’s depth, the number of neurons in each layer, and certain model hyperparameters, on the experimental outcomes. Additionally, we will conduct an ablation study to validate the positive effects of the improvements made to the model. The ablation study encompasses two aspects: one is the combination of PB and TB, and the other is the fusion of L1 and L2 norms as part of the loss function.

Number of hidden layers. Directly using the RA-QoS model to predict the two datasets yielded unsatisfactory results. The obtained RMSE and MAE values were considerably larger. Upon observing and analyzing the datasets used in the experiment, it was found that they exhibited significant variance and high data sparsity. Consequently, this article proposes increasing the number of hidden layers in the model, specifically adding layers to both the encoder and decoder, to enhance the model’s learning capacity and enable it to handle such datasets more effectively. The comparison of experimental results under different numbers of hidden layers is illustrated in Fig. 5. It can be observed that the experimental results are optimal when the number of hidden layers is three, i.e., when an additional layer is added to both the encoder and decoder. However, as the number of hidden layers increases, both the efficiency and generalization ability of the model decline. This is because an excessive number of hidden layers increases the risk of overfitting, potentially leading to the model learning noise and details in the data excessively, thereby reducing the prediction accuracy of the model. Interestingly, for the dataset related to response time, directly using three hidden layers did not yield satisfactory results. Instead, achieving better results was observed by reducing one layer in the decoder. Therefore, in subsequent comparative experiments and ablation studies, a three-layer hidden structure was used, with slight variations in the decoder stage.

Number of iterations. The number of iterations (or training epochs) in deep neural networks has a significant impact on prediction results. With more iterations, the degree of fitting to the training data may increase, but this also raises the risk of overfitting and requires more computational time. Selecting the appropriate number of iterations involves balancing the trade-off between overfitting and underfitting, while also aiding in improving model generalization. Figure 6 illustrates the comparison of the impact of different numbers of iterations on experimental results. It can be noted that as the number of iterations increases, the results become increasingly favorable. However, after 500 iterations, the reduction in RMSE and MAE diminishes, and there may even be negative growth. Therefore, setting the number of iterations to 500 can yield a satisfactory prediction performance for the model.

Ablation study

As this article described, the experiment in this study involved the RA-QoS model, primarily manifested in two aspects: the incorporation of different combinations of PB and TB, and the modification of the loss function. However, it is necessary to verify whether these modifications have improved the accuracy of the prediction results. Therefore, this article conducts an ablation study using the method of controlling variables.

This study includes four scenarios: not including PB and TB combinations without modifying the loss function; including PB and TB combinations without modifying the loss function; not including PB and TB combinations while modifying the loss function; and including different combinations of PB and TB while modifying the loss function. For the convenience of visualization, we represent these four scenarios as A1, A2, A3, and A4, respectively. The details of configuration of A1, A2, A3, and A4 are shown in the Table 7.

The specific comparison results are illustrated in Fig. 7. It can be seen that the line corresponding to A4 is located at the bottom of the graph overall. This indicates that the prediction performance of scenario A4 is the best among the four scenarios, implying that the proposed model’s improvement is effective.

Robustness test

To evaluate the robustness of the RA-QoS model when facing noise, we designed a series of experiments that introduce different types and intensities of noise into the QoS data, and observe the model’s performance under these noisy environments. The main goal of the experiment is to verify whether the RA-QoS model can maintain prediction accuracy stably in the presence of noise, and to compare its performance with baseline methods.

Experimental procedure

Dataset selection: We used the WS-Dream dataset containing multiple services and user QoS metrics, including response time and throughput. The RA-QoS model was trained on both the original and noisy data, and evaluated using metrics such as MAE and RMSE. To evaluate the robustness of the RA-QoS model, we compared it with several baseline models that used in the former mentioned experiments.

We evaluated the robustness of the RA-QoS model under various noise levels. The RMSE values for different methods under different noise levels are shown in the table and the corresponding trend is illustrated in Fig. 8.

The results showed that as the noise intensity increased (e.g., from $\sigma =0$ to $\sigma =1.0$), the MAE and RMSE for the RA-QoS model remained relatively stable, with only a slight increase in error. Even under high noise levels (e.g., $\sigma =1.0$), the RA-QoS model maintained low error rates, demonstrating strong robustness.

In contrast, the baseline methods, such as matrix factorization based RSNMF, NAMF and Collaborative Filtering based AutoRec, showed more unstable performance under high noise conditions. Specifically, with the introduction of missing data and high Gaussian noise, their MAE and RMSE values increased significantly, and their performance degraded faster. This indicates that the RA-QoS model is more stable when facing noise compared to these baseline methods. Through experiments with varying noise levels, we observed that the RA-QoS model remains stable and maintains low error rates even in the presence of high noise and missing data. In comparison, the baseline methods exhibited significant performance degradation under noisy conditions. This demonstrates that the RA-QoS model is robust and performs stably when confronted with data noise, making it a reliable choice for QoS prediction in real-world applications with imperfect data.

As the noise intensity (i.e., the value of $\sigma$) increases, the RMSE values for all baseline methods increase, indicating a deterioration in performance as noise levels rise. Among the baseline methods, including RSNMF, NAMF, GeoMF, DCALF, AutoRec and DLCANet, RA-QoS exhibits the smallest increase in RMSE, demonstrating its superior robustness against noise. This result highlights the effectiveness of the RA-QoS model in handling noisy data, making it a more stable and reliable choice for real-world QoS prediction tasks.

RA-QoS consistently demonstrates the smallest increase in both RMSE and MAE as the noise level increases, across both datasets (RT_20 and TP_20). This indicates that RA-QoS is more robust to noise than other baseline methods. The model’s ability to maintain low error rates, even under noisy conditions, makes it a reliable choice for QoS prediction in real-world environments.

Methods like RSNMF, NAMF, and AutoRec show a higher sensitivity to noise, with their error metrics increasing more significantly as the noise level rises. These methods struggle to handle the noise and exhibit performance degradation that RA-QoS avoids.

The GeoMF and DCALF models perform reasonably well but still show more sensitivity to noise compared to RA-QoS. While these models demonstrate better robustness than RSNMF and NAMF, they still experience a noticeable increase in RMSE and MAE values as noise levels grow.

Conclusion. This section reveals that the performance of RA-QoS is highly sensitive to bias configuration and network structure. The integration of PB.2 and TB.2 significantly enhances prediction accuracy and training efficiency. Moreover, the L1-L2 hybrid loss contributes to stronger robustness against noisy and outlier data. These findings confirm that the design choices in RA-QoS are effective in addressing challenges such as data sparsity, outliers, and convergence speed in deep QoS prediction.