Graph neural networks for preference social recommendation

Social preference network representation

In most cases, friends will influence each other, resulting in similar preferences, but it is not absolute. If friends have completely different preferences, recommendations based on social relationships are unreliable. This means that social networks cannot necessarily be used directly for recommendation systems, which need to be adjusted beforehand.

We use user preference information of the U-I network to obtain social network friend influence. Analyzing the U-I network, let the item sets of users’ preferences be H and the common preference matrix of users be C, then the common preference number C_xy between user x and user y is C_xy = |H_x∩H_y|. By fully considering the preference relationship and preference difference between users, we propose the friend influence indicator T_xy: (1)${\displaystyle T_{xy}=\frac{C_{xy}}{\sqrt{\left|H_x\right|}\sqrt{\left|H_y\right|}}.}$

We use social network to constrain indicator results to ensure the validity of friend influence. That is, the indicator only calculates the friend influence among connected users in social network. The value range of T_xy is [0, 1]. If T_xy = 0, the mutual influence is 0, which means that there is no common preference between user x and user y. If T_xy = 1, the mutual influence reaches the maximum, which means that the preference is highly correlated between users. In particular, if all friend influence indicator values T_{x_} of user x are all 0, the user x only has U-I network information, but no social network information. Taking the friend influence indicator as the edge weight of the social network, and removing edges with weight value of 0, a new view is obtained. In this article, we name the view the Social Preference Network.

Next, we update user representation in the social preference network. We use friend influence as aggregate weight of neighbor nodes, and define the update method of the representation $p_{u_x}^k$ of user x of layer k as (2)${\displaystyle p_{u_x}^k=\sigma \left(\sum _{y\in {\Gamma }_x^S\cup x}{T}_{xy}{p}_{u_y}^{k-1}{W}_1\right)}$ where ${\Gamma }_x^S$ is the neighbors of user x in social preference network, σ is the tanh activation function, W₁ ∈ ℝ^d×d is weight matrix and d is the dimension of hidden layer. In particular, T_xy = 1 when y = x.

We use D_u = diag{D_u1, …, D_uN} ∈ ℝ^N×N as the diagonal degree matrix, where D_ux = |H_x| is the degree of user x in U-I network. Combining Eqs. (1) and (2), the matrix formulation of node update of layer k in social preference network is expressed as (3)${\displaystyle p_u^k=GN{N}_1\left(p_u^{k-1}\right)=\sigma \left(\left(C\odot D_u^{-\frac{1}{2}}{A}_S{D}_u^{-\frac{1}{2}}+I\right)p_u^{k-1}{W}_1\right)}$ where the weight matrix W₁ is shared by parameters in different layers.

U-I network representation

The U-I network can well reflect the user’s preference information, that can be directly used in recommendation system. We define the update method of the representation $q_{u_x}^k$ of user x of layer k as (4)${\displaystyle q_{u_x}^k=\sigma \left(\sum _{y\in {\Gamma }_x^I}\frac{1}{\sqrt{D_{u_x}{D}_{i_y}}}{q}_{i_y}^{k-1}{W}_2\right)}$ where ${\Gamma }_x^I$ is the neighbor item set of user x, and D_ux and D_iy represent the degree of user x and item y respectively in U-I network. In order to be consistent with the user representation of social preference network, we use the same activation function σ = tanh and the same dimension of the weight matrix W₂ ∈ ℝ^d×d.

In order to reduce the number of parameters and the computational complexity, we delete the weight matrix and nonlinear activation function used in item update. Therefore, we define the update method of the representation $q_{i_z}^k$ of item z of layer k as (5)${\displaystyle q_{i_z}^k=\sum _{y\in {\Gamma }_z^I}\frac{1}{\sqrt{D_{i_z}{D}_{u_y}}}{q}_{u_y}^{k-1}}$ where ${\Gamma }_z^I$ is neighbor user set of item z, and D_iz and D_uy represent the degree of item z and user y respectively in U-I network.

We use D = diag{D_u1, …, D_uN, D_i1, …, D_iM} ∈ ℝ^(N+M)×(N+M) as the diagonal degree matrix. The corresponding adjacency matrix A ∈ ℝ^(N+M)×(N+M) is (6)${\displaystyle \left(\begin{array}{cc}\hfill \mathbf{0}\hfill & \hfill A_I\hfill \\ \hfill A_I^T\hfill & \hfill \mathbf{0}.\hfill \end{array}\right)}$

Combining Eqs. (4), (5) and (6), the matrix formulation of node update of layer k in U-I network is expressed as (7)${\displaystyle \left(\begin{array}{c}\hfill q_u^k\hfill \\ \hfill q_i^k\hfill \end{array}\right)=\left(\begin{array}{c}\hfill GN{N}_2\left(q_u^{k-1}\right)\hfill \\ \hfill GN{N}_3\left(q_i^{k-1}\right)\hfill \end{array}\right)=f\left(D^{-\frac{1}{2}}A{D}^{-\frac{1}{2}}\left(\begin{array}{c}\hfill q_u^{k-1}{W}_2\hfill \\ \hfill q_i^{k-1}\hfill \end{array}\right)\right)}$ where f is activation function, the first N lines f(x) = σ(x), the last M lines f(x) = x, and the weight matrix W₂ is shared by parameters in different layers.

Two losses-based PSR model training

We use a full connected layer to process the user representation of the social preference network and the U-I network to obtain the final user representation h_u (8)${\displaystyle h_u=\frac{1}{l}\sum _{k=1}^l\left(p_u^k\left|\right|q_u^k\right)W_3}$ where || means concatenate, W₃ ∈ ℝ^2d×d is weight matrix. The final item representation h_i is (9)${\displaystyle h_i=\frac{1}{l}\sum _{k=1}^l{q}_i^k.}$

We use inner product y_xz = h_ux⋅h_iz as the rating of user x and item z. By minimizing Bayesian Personalized Ranking (BPR) loss (Rendle et al., 2009) and Global Orthogonal Regularization (GOR) loss (Zhang et al., 2017), the model is trained. BPR loss is used to preserve the original connection relationship, the calculation formula is (10)${\displaystyle L_{BPR}=\sum _{\left(x,z\right)\in E_I\cup \left(x,z^{\prime }\right)⁄\in E_I}-ln\left(\sigma \left(y_{xz}-y_{x{z}^{\prime }}\right)\right)+\lambda \left|\right|E^0|{|}_2^2}$ where $E^0=\left[h_u^0\left|\right|q_i^0\right]$, $h_u^0=p_u^0=q_u^0$ is randomly initialized user representation, $q_i^0$ is randomly initialized item representation. To keep consistent with the comparison algorithm (Liao et al., 2022), λ is set to 1e⁻⁴.

GOR loss is used to widen the distance between positive and negative samples, and the calculation formula is (11)${\displaystyle L_{GOR}={\left(\frac{1}{N^{\prime }}\sum _{\left(x,z^{\prime }\right)⁄\in E_I}{y}_{x{z}^{\prime }}\right)}^2+\varphi \left(\frac{1}{N^{\prime }}\sum _{\left(x,z^{\prime }\right)⁄\in E_I}{y}_{x{z}^{\prime }}^2-\frac{1}{d}\right)}$ where N′ is the number of negative samples. To make the activation function ϕ smoother, we use Softplus instead of RELU of the original article.

Through Eqs. (10) and (11), the final loss function is obtained as (12)${\displaystyle L=L_{BPR}+\alpha L_{GOR}}$ where α is hyperparameter used to balance the two loss.

Space complexity

In PSR, there are two parts of trainable parameters: (i) initial representation of the node, and (ii) weight matrix of neural network. For (i), the space complexity is (N + M)d, which is consistent with most neural network models (e.g., LightGCN and SocialLGN). For (ii), PSR uses three weight matrixes W₁ ∈ ℝ^d×d, W₂ ∈ ℝ^d×d and W₃ ∈ ℝ^2d×d. Since each weight matrix is parameter-shared among layers, the space complexity of PSR in this part is 4d². In summary, the total space complexity of PSR is (N + M + 4d)d, which is consistent with SocialLGN. Since min(N, M) ≫ d, 4d can be ignored, which means that the space complexity of PSR is also approximately equal to LightGCN.

Time complexity

Similar to most graph convolution kernels, the friend influence indicator can be calculated as preprocessing. For a single-layer neural network, considering the sparsity of the network, the time complexity of node aggregation of social network is $\mathcal{O}\left(\left|E_S\right|d\right)$, while U-I network is $\mathcal{O}\left(\left|E_I\right|d\right)$, and the time complexity of the graph diffusion operation through weight matrix is $\mathcal{O}\left(4N{d}^2\right)$. Therefore, the total time complexity of PSR is $\mathcal{O}\left(\left|E_S\right|dl+\left|E_I\right|dl+4N{d}^{2}l\right)$ which is linearly related to max(|E_S|, |E_I|, N). It is consistent with SocialLGN, which means that it is lower than most existing GNN-based social recommendation models (SocialLGN is a light GNN-based model).

Datasets

We use LastFM (Yu et al., 2021a; Yu et al., 2021b) and Ciao (Fan et al., 2019; Fan et al., 2020) to analyze the performance of model. These two datasets are real-world datasets that are often used in recommendation systems. As a music dataset, LastFM includes friend relationships and users’ music preferences. As an online shopping dataset, Ciao includes friend relationships and users’ shopping information. The dataset statistics are shown in Table 1.

Comparison algorithms

We compare PSR with some well-known methods to verify the performance of the model. The comparison algorithms are as follows.

- BPR (Rendle et al., 2009): A non-graph neural network model which ranks items by maximizing the posterior probability.

- SBPR (Zhao, McAuley & King, 2014): The first model to introduce social relationships into recommender systems, that sorts the items according to the user’s preference, the user’s friend’s preference, and the remaining preference.

- DiffNet (Wu et al., 2019b): It treats friends influence equally, and updates user and item information by accumulation.

- NGCF (Wang et al., 2019): A social recommendation model which only aggregates neighborhood information without aggregating central node information.

- LightGCN (He et al., 2020): It removes the weight matrix and nonlinear activation function in NGCF.

- SocialLGN (Liao et al., 2022): It designs a graph fusion component for user update, and removes the nonlinear activation function and weight matrix for item update.

Parameter settings

For better comparison, we keep aligned with the experimental settings of the current SOTA model (SocialLGN). We take 80% of the data as the training set. And we set the random seed to 2020, the representation dimension to 64, the number of neural network layers l to 3, λ to 1e⁻⁴, the initial learning rate to 1e⁻³ and Adam as the optimizer. For the new hyperparameter α in PSR, we choose from {0, 1, …, 10}. In order to reduce the influence of hyperparameters α, we choose α = 5 by default.

Evaluation indicators

We use three mainstream evaluation indicators (Wu et al., 2020; Liao et al., 2022), namely Precision@K, Recall@K, and NDCG@K, to evaluate the recommendation performance of top-K ranking.

Precision@K indicates the probability of correct prediction in the predicted positive sample set. (13)${\displaystyle Precision@K=\frac{TP@K}{TP@K+FP@K}}$

Recall@K indicates the probability of correct prediction in the real positive sample set. (14)${\displaystyle Recall@K=\frac{TP@K}{TP@K+FN@K}}$

For the connection relationship of the U-I network, TP is the number of predicted connected edges which are actually connected, FP is the number of predicted connected edges which are actually disconnected, FN is the number of predicted disconnected edges which are actually connected.

NDCG@K considers the ranking order of the prediction results on the basis of the above two indicators, and the formula is (15)${\displaystyle NDCG@K=\frac{\sum _{i=1}^K\frac{re{l}_i}{{log}_2\left(i+1\right)}}{\sum _{i=1}^{\left|REL\right|}\frac{re{l}_i}{{log}_2\left(i+1\right)}}}$ where rel_i is relevance score, |REL| is the ranking result under the similarity.