GCL-ALG: graph contrastive learning with adaptive learnable view generators

Contrastive learning

Contrastive learning (CL) has made promising achievements in image representation learning (Chen et al., 2020b; Zhu et al., 2021) thanks to its powerful self-supervised learning ability. CL learns robust representations by pulling in positive pairs and pushing out negative pairs, and utilizing the relative relationship between samples to determine semantic associations. Therefore, many researchers attempted to apply contrastive methods to graph domain in order to extract meaningful graph structure semantic relationships. However, the intrinsic structural complexity of graph data also poses greater challenges. Previous works have been improved from several perspectives such as different contrast angles, augmentation strategies, positive and negative sample selection, etc. DGI (Velickovic et al., 2018) is driven by the mutual information estimation proposed by MINE (Belghazi et al., 2018) to maximize the mutual information between the global graph embedding and the local node embedding. GMI (Peng et al., 2020) learns node representation vectors by maximizing the mutual information between the encoder input representation and the output hidden representation. MVGRL (Hassani & Khasahmadi, 2020) constructs node-level and graph-level augmented instances based on graph diffusion and first-order neighborhoods, and proposes a cross-view multi-scale contrastive learning approach. GRACE (Zhu et al., 2020) generates two augmented graphs by removing edges and masking attributes, focusing on maximizing the consistency of node embedding between broken graphs. Graph contrastive representation learning with adaptive augmentation (GCA) (Zhu et al., 2021) builds upon GRACE (Zhu et al., 2020) and proposes adaptive augmentation based on the topology and attributes of the graph, enabling it to dynamically adjust the augmentation strategy according to the specific features of the input graph. GraphCL (You et al., 2020) designs a data augmentation method with a combination of four approaches: node removal, edge perturbation, subgraph sampling and feature masking, revealing that the selection of augmentation strategies needs to be adapted to the specific traits of different tasks. GCP (Adjeisah et al., 2024) also adopts diverse augmentation strategies to generate multi-view graph representations, and leverages pre-training to automatically identify the optimal augmentation configuration. However, Xia et al. (2022) question the necessity of data augmentation by proposing SimGRACE, which perturbs the encoder to generate contrastive views directly from the original graph, thereby avoiding semantic bias introduced by manual augmentations while achieving competitive performance across multiple benchmark tasks. NCAGC (Wang et al., 2023) uses k-nearest neighbors (KNN) to construct intra-cluster and inter-cluster neighborhood contrastive relations to guide the learned feature representations towards clustering. SUGRL (Mo et al., 2022) exploits the complementary information between structural and neighbor information to generate positive samples, and designs a multi-objective contrastive loss to enforce a closer relation to positive samples and a further divergence from negative samples.

We note that the augmentation operations in the above approaches obtain augmentation instances based on simple predefined parameters violently destroying the original graph, which may cause severe semantic loss leading to unstable quality of the generated views. To tackle these concerns, GCL-ALG builds learnable view generators to automatically acquire valid augmentation instances and provide rich augmentation variance.

Automated data augmentation

Data augmentation enriches training samples while maintaining label consistency. As demonstrated by Adjeisah et al. (2023), augmentation strategies play a pivotal role in enhancing the robustness and generalization capabilities of graph neural networks, especially in data-scarce scenarios. However, most existing methods still rely on manually predefined augmentation operations, which are limited in their adaptability to diverse task requirements. In contrast, automated data augmentation has emerged as a promising alternative, enabling dynamic strategy optimization based on both input data characteristics and task objectives, thus garnering increasing research attention. In the computer vision field, AutoAugment (Cubuk et al., 2019) first proposes to utilize reinforcement learning to achieve automated data augmentation. Cubuk et al. (2020) propose the RandomAugment (Cubuk et al., 2020) method using a parameter-free procedure instead of learning strategies and probabilities, successfully balancing data augmentation diversity and search costs. BO-Aug (Zhang et al., 2023) proposes to explore optimal augmentation schemes in the space of predefined data augmentation strategies using Bayesian optimization techniques.

These automated image augmentation methods inspire graph contrastive learning. JOAO (You et al., 2021) learns the sampling distribution of predefined augmentation operations on the top of GraphCL (You et al., 2020), but the augmentation itself is not learnable. AD-GCL (Suresh et al., 2021) is motivated by adversarial learning and proposes a learnable edge-level data augmentation. AutoGCL (Yin et al., 2022) proposes learnable view generators driven by learning probabilities. RGCL (Li et al., 2022) utilizes a learnable rationale generator to identify and preserve discriminative node sets in the graph, combines complement sampling to construct non-discriminative structures, and optimizes augmented views through adversarial training. HAGCL (Chen, Ren & Yong, 2023) constructs dedicated view generators from feature and edge perspectives to achieve mixed data augmentation. GCC-LDA (Yang et al., 2022) proposes to automatically generate structure-based and attribute-based augmentation with attention mechanisms. NCLA (Shen et al., 2023) proposes a learnable augmentation based on multi-head attention mechanisms. However, the augmentation perspectives in previous studies are still not comprehensive, and the augmentation process lacks guidance signals. In this work, we performed fine-grained augmentation separately on the graph constituents- nodes and edges, introduce network centrality as guidance signals so that augmentation instances are oriented to key elements, and optimize the learnable augmentation parameters through an autoencoder to dynamically balance semantic preservation and view variability.

Notions and preliminaries

Given a graph $\mathcal{G}\in \{{\mathcal{G}}_m:m\in M\}$ from the dataset of M graphs. Define graph $\mathcal{G}=(\mathcal{V},\mathcal{E})$, considering $\mathcal{V}=\{v_1,v_2,\cdots ,v_N\}$ and $\mathcal{E}\subseteq \mathcal{V}\times \mathcal{V}$ represent the node set and edge set respectively. $\mathit{X}\in {\mathbb{R}}^{N\times F}$ and $\mathit{A}\in \{0,1{\}}^{N\times N}$ denote the feature matrix and the adjacency matrix, where $x_i\in \mathit{X}$ indicates the feature vector of node $v_i$ and $A_{ij}=1$ if $(v_i,v_j)\in \mathcal{E}$ otherwise $A_{ij}=0$. $\mathcal{N}(v_i)=\{v_j\mid j\ne i,A_{ij}=1\}$ indicates a set of 1-top neighbors of $v_i$. $\mathit{D}=diag(d_1,d_2,\dots ,d_N)\in {\mathbb{R}}^{N\times N}$ denotes the degree matrix, where $d_i={\sum }_{(v_i,v_j)\in E}{A}_{ij}$. Input graphs to two separate view generators generate views $\widetilde{G_1}$, $\widetilde{G_2}$, where the process base node features ${\mathit{h}}_i$ and edge features ${\mathit{h}}_{ij}$ count sampling to generate augmentation strategy matrices ${\mathit{S}}_1\in {\mathbb{R}}^N$, ${\mathit{S}}_2\in {\mathbb{R}}^N$. Through the graph encoder $\mathcal{F}(\cdot )$, both node-level representations ${\mathit{H}}_n^1\in {\mathbb{R}}^{N\times F^{\prime }}$, ${\mathit{H}}_n^2\in {\mathbb{R}}^{N\times F^{\prime }}$ and graph-level representations ${\mathit{H}}_g^1\in {\mathbb{R}}^{D\times F\prime \prime }$, ${\mathit{H}}_g^2\in {\mathbb{R}}^{D\times F\prime \prime }$ are produced. Detailed information is shown in Table 1.

• Graph neural networks. Real-world data structures are very complex and are gradually characterized by graph structure. Scarselli et al. (2008) proposed the GNN (Scarselli et al., 2008) as an effective method for the analysis of graph-structured data. In this work, we focus on GNN as the backbone network of the encoder. For a given graph $\mathcal{G}$, each node $v_i$ has a feature distribution ${\mathit{h}}_i$, initialized as ${\mathit{h}}_i^{(0)}={\mathit{x}}_i$, then $v_i$ at the k-th GNN message propagation can be stated as,

  (1) $${\mathit{a}}_i^{(k)}=AGGREATIO{N}^{(k)}(\{{\mathit{h}}_i^{(k-1)}:i\in \mathcal{N}(i)\}),$$ (2) $${\mathit{h}}_i^{(k)}=COMBIN{E}^{(k)}({\mathit{h}}_i^{(k-1)},{\mathit{a}}_i^{(k}).$$ where both $AGGREGATE(\cdot )$ and $COMBINE(\cdot )$ are trainable functions, serving the purpose of aggregating and updating messages respectively. After a specified number of iterations, the READOUT() function is used to aggregate embeddings from all nodes, generating a graph-level representation. Then, this representation is converted into an output suitable for downstream tasks through a multi-layer perceptron (MLP), denoted as $z_G$:

  (3) $$z_G=MLP(READOUT(\{{\mathit{h}}_i^{(k)},v_i\in V\})).$$

• Gumbel-Softmax. We employ Gumbel-Softmax (Jang, Gu & Poole, 2016) as our sampling function in this work, which can convert a discrete random variable into a differentiable continuous distribution via reparameterization, and enabling backpropagation in neural networks. Assume the network propagation process yields a collection of discrete feature distributions ${\mathbf{\text{π}}}_{\mathit{1}},{\mathbf{\text{π}}}_{\mathit{2}},\dots ,{\mathbf{\text{π}}}_{\mathit{n}}$, where ${\mathbf{\text{π}}}_{\mathbf{i}}$ denotes the i-th edge or node feature, ${\sum }_i^n\mathbf{\text{π}}=1$, then the sampling result $z_i$ can be represented as,

  (4) $$z_i=\frac{exp({\mathit{g}}_i+\log {\mathbf{\text{π}}}_{\mathit{i}})/\tau }{{\sum }_j^{n}exp({\mathit{g}}_j+\log {\mathbf{\text{π}}}_{\mathit{j}})/\tau },$$where $\tau$ represents temperature coefficient, $g_i$ is a pre-collected sample from the Gumbel(0,1) distribution. It can be observed that the higher the probability of a particular feature distribution, the more likely it is to be selected during Gumbel-Softmax sampling. For this purpose, we generate prior probabilities from the original graph to adjust the sampling distribution in order to maximize the retention of important information in the graph.

Learnable view generator

In this section, we provide a detailed description of the view generators proposed in GCL-ALG, which includes how to explore graph features, how to incorporate adaptive metrics, and how to implement feature sampling.

• Dual-layer feature extraction. Graph neural networks often struggle to simultaneously capture local connectivity patterns and global topological structures in complex graph tasks. To address this issue, we propose a dual-layer feature extraction framework that hierarchically models structural information at different granularities. By leveraging a local-global complementary feature extraction scheme, the framework enhances the representational coverage and structural sensitivity, thereby improving the model’s adaptability to diverse types of graph data.

  The view generators integrate a self-attention-based encoder ${\mathcal{F}}_A$ and an edge-based encoder ${\mathcal{F}}_E$, with the primary objective of collaboratively extracting critical structural information from the graph.

  Self-Attention-based GNN encoder. The ${\mathcal{F}}_A$ encoder integrates the strong structural modeling capacity of graph isomorphism network (GIN) with the selective feature extraction capability of the attention mechanism, enabling it to effectively capture salient node features within local neighborhoods. Assuming a total of K GIN layers are used, a new node feature matrix ${\mathit{E}}_n^k\in {\mathbb{R}}^{N\times F^k}$ is generated after each convolution layer. We denote [;] is the concatenation operator and the k-th iteration of the iterative aggregation update formula for node $v_i$ as:

  (5) $${\mathit{h}}_i^{(k)}=COMBIN{E}^{(k)}({\mathit{h}}_i^{(k-1)},{\mathit{a}}_i^{(k)}),$$ (6) $${\mathit{a}}_i^{(k)}=AGGREATIO{N}^{(k)}(\{{\mathit{h}}_i^{(k-1)}:i\in \mathcal{N}(i)\}),$$ (7) $${\mathit{E}}_n^k={[{\mathit{h}}_1^{(k)};{\mathit{h}}_2^{(k)};\dots ;{\mathit{h}}_N^{(k)}]}^T.$$

  Inspired by Transformer (Vaswani et al., 2017), we seamlessly integrate a self-attention network module after each GIN layer, consciously guiding the network propagation process to keenly capture more crucial node-level information, thereby elevating the model’s learning capability to new heights. The hidden layer features ${\mathit{E}}_n^k$ are mapped to three different vector spaces: $\mathit{Q}\in {\mathbb{R}}^N\times {\mathbb{F}}^k$, $\mathit{K}\in {\mathbb{R}}^{N\times F^k}$, $\mathit{V}\in {\mathbb{R}}^{N\times F^k}$, which respectively represent the query matrix, key matrix and value matrix.

  (8) $$\mathit{Q}={\mathit{W}}_q({\mathit{X}}^k{)}^T,$$ (9) $$\mathit{K}={\mathit{W}}_k({\mathit{X}}^k{)}^T,$$ (10) $$\mathit{V}={\mathit{W}}_v({\mathit{X}}^k{)}^T,$$where ${\mathit{W}}_q\in {\mathbb{R}}^{F^k\times F^k}$, ${\mathit{W}}_k\in {\mathbb{R}}^{F^k\times F^k}$, ${\mathit{W}}_v\in {\mathbb{R}}^{F^k\times F^k}$ are the initialized weight matrices, and the node embedding ${\mathit{E}}_n\in {\mathbb{R}}^{N\times F^k}$ after the attention mechanism can be expressed as:

  (11) $${\mathit{E}}_n=softmax\left(\frac{\mathit{Q}{\mathit{K}}^T}{\sqrt{F^k}}\right)\mathit{V}.$$

  Edge-based GNN encoder. To compensate for the ${\mathcal{F}}_A$ encoder’s limited understanding of the overall graph structure, the ${\mathcal{F}}_E$ encoder is introduced to incorporate higher-order structural information. ${\mathcal{F}}_E$ leverages edge weight to measure the importance of edges in global information flow, effectively capturing the macro structure and global dependencies of the graph. We first concatenate the node features to generate preliminary edge features $E_e$, and then employ an MLP to parameterize the edge features and extract weight information,

  (12) $$w_{ij}=MLP\left([{\mathit{h}}_i^{(K)},{\mathit{h}}_j^{(K)}]\right),$$where ${\mathit{h}}_i^{(K)}$, ${\mathit{h}}_i^{(K)}\in {\mathit{H}}_n$ denote the feature vectors of nodes $v_i$ and $v_j$ respectively after ${\mathcal{F}}_A$ encoder.

  To learn the edge weights, $w_e$ is normalized to a continuous variable between $[0,1]$. Specifically, we employ $p_{ij}=sigmoid((log\delta -log(1\text{-}\delta )+w_e)/\tau )$ to represent the weight of the edge that links nodes $v_i$ and $v_j$, where $\delta \sim Uniform(0,1)$ and $\tau$ is a temperature hyper-parameter. Eventually, ${\mathcal{F}}_E$ adjusts the aggregation process of node features using this weight information, giving greater emphasis to edges with higher weights, thus improving the model’s emphasis on the edges. In formal terms, the formula for aggregated edge weights can be denoted as:

  (13) $${\mathit{h}}_i^{(k)}=COMBIN{E}^{(k)}({\mathit{h}}_i^{(k-1)},{\mathit{a}}_i^{(k)}),$$ (14) $${\mathit{a}}_i^{(k)}=AGGREATIO{N}^{(k)}\left(\{p{\mathit{h}}_i^{(k-1)}:i\in \mathcal{N}(i)\}\right).$$

  Then we obtain the final node feature matrix ${\mathit{H}}_n={[{\mathit{h}}_1^{(K)};{\mathit{h}}_2^{(K)};\dots ;{\mathit{h}}_N^{(K)}]}^T\in {\mathbb{R}}^{D\times F^{\prime }}$, $F^{\prime }\ll F$. The raw edge feature ${\mathit{E}}_e$ passes through the projection layer to capture high-order crossed features, obtaining the final edge feature. He represented by the formula ${\mathit{H}}_e=g({\mathit{E}}_e)$, where $g(\cdot )$ is the projection function.

• Node sampling. With the above procedures, we have successfully generated graph embeddings that encompass adaptive and crucial features. Now, we need to further adaptively sample node features and edge features. To be more specific, we again cut from the perspective of the original graph, generate prior probabilities for nodes and edges based on the original graph, and incorporate these probabilities into the subsequent sampling process. The underlying logic of this strategy is to guide the sampling function to selectively favor nodes and edges in the original graph that are considered essential, thereby avoiding a completely random or uniform sampling of all elements.

  We leverage the Gumbel-Softmax (Jang, Gu & Poole, 2016) reparametrization trick to generate a differentiable probability distribution, thus enabling the optimization of the sampling operation through gradient descent. Additionally, the introduction of Gumbel noise maintains a certain level of randomness, which is beneficial in preventing the model from over-concentrating on specific features, thereby mitigating the risk of under-generalization. Figure 2 takes node sampling as an example to describe in detail the feature correction and sampling process. The sampling probability of node $v_i$ can be expressed as: (15) $$P_i=Gumbel\mathrm{\_}Softmax(Node\mathrm{\_}Cent(v_i){\mathit{h}}_i),$$where $Node\mathrm{\_}Cent(\cdot )$ is the node centrality measure formula, ${\mathit{h}}_i\in {\mathit{H}}_n$ is the feature vector of node $v_i$, and $Node\mathrm{\_}Cent(\cdot ){\mathit{h}}_i$ denotes the adaptive sampling probability. With a combination of efficiency and performance considerations, we adopt node degree centrality as the metric function, which is formulated as:

  (16) $$Node\mathrm{\_}Cent(v_i)=\frac{d_i}{N-1}.$$

  In this context, N denotes the total number of nodes in the network which node $v_i$ belongs. Degree centrality is the most straightforward and broadly applied measure of node centrality in network analysis. This metric posits that nodes with more associated nodes are considered more important. For instance, in social networks, nodes represent users and edges represent connections between users. Users with a higher degree centrality tend to have greater influence.

• Edge sampling. The edge sampling process is akin to node sampling, which aims to filter out edges with infrequent interactions between nodes. We utilize MLP to reduce the dimensionality of the generated edge features to prevent too much information from being lost in subsequent sampling, while obtaining a representative edge feature matrix ${\mathit{H}}_e$. Then, by performing a Hadamard transformation between ${\mathit{H}}_e$ and the learned edge importance distribution from the original graph, we obtain an adaptive distribution matrix for the edges. Finally, we use the Gumbel_Softmax function to sample from the adaptive probability distribution and select the edges to be retained. In this process, $e_{ij}$ represents the edge connecting nodes $v_i$ and $v_j$, and ${\mathit{h}}_{ij}\in {\mathit{H}}_e$ represents the feature vector of edge $e_{ij}$, then we have:

  (17) $$P_{ij}=Gumbel\mathrm{\_}Softmax(Edge\mathrm{\_}Cent(e_{ij}){\mathit{h}}_{ij}),$$where $Edge\mathrm{\_}Cent(\cdot )$ indicates edge centrality formula. We choose the average degree centrality of the two nodes linked by the edge as the edge centrality metric, which can better preserve the stability of the network structure.

  (18) $$Edge\mathrm{\_}Cent(e_{ij})=\frac{Node\mathrm{\_}Cent(v_i)+Node\mathrm{\_}Cent(v_j)}{2}.$$

  Through the aforementioned adaptive encoding and sampling strategies, we successfully preserve the key elements of the graph’s inherent attributes, which form the foundation for deep integration of the graph.

Architecture of contrastive learning

In this article, the proposed graph contrastive learning framework adheres to the well-known InfoMin (Tian et al., 2020) principle, aiming to maximize the mutual information between the augmented views and the input graph while minimizing the mutual information between generated views. Following the settings of AutoGCL (Yin et al., 2022), we adopt three loss functions to constrain the contrastive learning framework: contrastive loss, similarity loss and classification loss.

Drawing from the previous works (Chen et al., 2020b; You et al., 2020), we use the normalized temperature-scaled cross entropy loss (NT-Xent) (Sohn, 2016) as our contrastive loss function to force the positive sample pairs to draw closer while separating the negative samples. Randomly select a batch comprising N graphs and obtain $2N$ views via two independent view generators. Arbitrarily specify $z_{1,i}\in Z^{(1)},z_{2,j}\in Z^{(2)}$ as the generated views from different view generators respectively, and treat the two as a pair of positive samples. Negative samples are naturally defined as views from the other $2(N-1)$ views. Formally, we define the cosine similarity function $\theta (z_{1,i},z_{2,j})=\frac{z_{1,i}\cdot z_{2,j}}{\Vert z_{1,i}{\Vert }_2\cdot \Vert z_{2,j}{\Vert }_2}$. Set the pairwise objective for each positive pair $(z_{1,i},z_{2,j})$ as $\ell (i,j)$, then the overall objective function can be expressed as ${\mathcal{L}}_{cl}$, where $\tau$ denotes the temperature parameter.

(19) $$\ell (i,j)=-\log \frac{e^{\theta ({\mathit{z}}_{1,i},{\mathit{z}}_{2,j})/\tau }}{{\sum }_{\genfrac{}{}{0ex}{}{m,n=1}{m\le n}}^N{e}^{\theta ({\mathit{z}}_{1,i},{\mathit{z}}_{1,m})/\tau }+e^{\theta ({\mathit{z}}_{1,i},{\mathit{z}}_{2,n})/\tau }},$$

(20) $${\mathcal{L}}_{cl}=\frac{1}{2N}\sum _{k=1}^N[\ell (2k-1,2k)+\ell (2k,2k-1)]$$

In each iteration of GCL-ALG, the original graph G is first input into the view generators, which produce two adaptive augmentation strategy matrices (or sampling matrices) $S_1$ and $S_2$, along with their corresponding augmented views ${\tilde{G}}_1$ and $\tilde{G}2$. To avoid the issue of view collapse, we aim to ensure sufficient diversity between the two augmented views. To this end, we introduce a view similarity loss function $L_{sim}$ in the semi-supervised setting, defined as:

(21) $${\mathcal{L}}_{sim}=1-\Vert {\mathit{S}}_1-{\mathit{S}}_2{\Vert }_2^2.$$

The loss term penalizes the similarity between the augmentation strategies, explicitly encouraging the generation of diverse views. Since this mechanism significantly intervenes in the view generation process, it may disrupt the semantic consistency of the original graph structure in unsupervised tasks. Therefore, this loss is only used in the semi-supervised setting.

Finally, to further enhance the model’s ability to fit the known labels and improve the discriminative power of the learned representations, we introduce a graph classification loss function $L_{cls}$ under the semi-supervised setting. Specifically, the original graph G and its two augmented views ${\tilde{G}}_1$ and $\tilde{G}2$ are jointly fed into the classifier C, and the standard cross-entropy function ${\ell }_{cls}$ is applied to compute the prediction loss with respect to the corresponding label $y$. The loss is formally defined as follows:

(22) $${\mathcal{L}}_{cls}=\frac{1}{3}[{\ell }_{cls}(C(\widetilde{G_1}),\mathit{y})+{\ell }_{cls}(C(\widetilde{G_2}),\mathit{y})+{\ell }_{cls}(C(\tilde{G}),\mathit{y})].$$

The time complexity

The GCL-ALG algorithm consists of three main parts. The first part is view generator based on dual-level feature extraction, the first level of feature extraction is based on the self-concerned encoder corresponding time complexity is $\mathcal{O}(l_a(|E|F+N^{2}F))$, where $l_a$ denotes the number of hidden layers. The second level of feature extraction fuses the edge weight information mined by the MLP into the GIN message propagation process with time complexity of $\mathcal{O}(l_{e}N{H}^2)$, where H is the hidden layer dimension, and $l_e$ is the number of hidden layers. The second part is a graph encoder with a GIN backbone whose time complexity is compressed to $\mathcal{O}(l_g(|E|H+NH))$, where $l_g$ denotes the number of hidden layers of the graph encoder. The third part is the downstream classifier, the unsupervised representation learning uses SVM as the classifier with $\mathcal{O}(N{D}^2)$ time complexity, the semi-supervised task predicts the category to which the graph belongs based on the ResGCN network with $\mathcal{O}(l_s|E|D)$ time complexity, and the migration learning generates the classification results with the GIN network with $\mathcal{O}(l_{t}E{D}^2)$, where $l_s$, $l_t$ represent the number of hidden layers for semi-supervised and migration learning classifiers, respectively. Therefore, the total time complexity of unsupervised learning, semi-supervised learning and transfer learning are compressed as: $\mathcal{O}(|E|(F+H)+N(F^2+H^2+D^2))$, $\mathcal{O}(|E|(F+H)+N(F^2+H^2)+l_s|E|D)$, $\mathcal{O}(|E|(F+H)+N(F^2+H^2)+l_t|E|D)$, respectively. The detailed pseudo-code description of GCL-ALG is given in Algorithm 1.

Datasets description

Our experiments include generalized datasets from multiple domains. For supervised and semi-supervised learning, we use the TUDataset (Morris et al., 2020), which provides a wide range of graph datasets, including but not limited to protein datasets and social network datasets. In addition, we predict chemical molecule properties and biological protein functions through transfer learning, in which we adopt the ChEMBL dataset (Mayr et al., 2018; Gaulton et al., 2011) containing 456K complex biochemical molecules as the pre-training dataset, fine-tuning the small molecule dataset of the MoleculeNet benchmark (Wu et al., 2018). We perform statistics based on dataset categories as shown in Table 2. The TuDataset download link is https://chrsmrrs.github.io/datasets/docs/datasets, the ChEMBL dataset can be found at https://www.ebi.ac.uk/chembl, and the MoleculeNet dataset can be downloaded from https://moleculenet.org/datasets-1. The code for our implementation can be found in https://github.com/lincyli/GCL-ALG.

Baselines

The effectiveness of the proposed method is verified by comparing it with the following 13 baseline approaches on three tasks: unsupervised learning, semi-supervised learning, and transfer learning.

• InfoGraph (Sun et al., 2019): A method for learning graph representations based on maximizing mutual information.

• GraphCL (You et al., 2020): A method that explores diverse features through combinatorial graph augmentations.

• GCA (Zhu et al., 2021): The method leverages network centrality and adaptive data augmentation to retain important elements.

• JOAO (You et al., 2021): The method constructs an augmentation-aware projection head to find the optimal augmentation strategy from a predefined pool of augmentations.

• AD-GCL (Suresh et al., 2021): An augmentation method that employs adversarial learning to delete edges, based on a max-min mutual information framework.

• AutoGCL (Yin et al., 2022): A method that uses data-driven active learning to adjust node dropout and feature masking probabilities in order to maximize retention of the original semantics.

• SimGRACE (Xia et al., 2022): An Augmentation-Free graph contrastive learning framework via Gaussian-Perturbed Encoders.

• RGCL (Li et al., 2022): A rationale-aware graph contrastive learning framework grounded in the principle of invariance.

• GPA (Zhang et al., 2024): A graph contrastive learning method with personalized augmentation selector for graph-level adaptive augmentation.

• Infomax (Velickovic et al., 2018): An approach for learning node representations by maximizing local-global mutual information.

• EdgePred, AtrrMasking, and ContextPred: Node-level pre-training methods proposed by Pretrain-GNN (Hu et al., 2019), involving original edge prediction, learning node/edge attributes, and predicting surrounding graph structures with subgraphs for training GNN to explore deep representations.

Experiment settings

To ensure fair comparison, GCL-ALG reports the average results over five runs with different random seeds. The model is implemented using the PyTorch deep learning framework and trained on NVIDIA RTX 3090 (24 GB) and NVIDIA A40 (48 GB) GPUs.

In order to evaluate the proposed method, the following three evaluation criteria are introduced, where for all metrics higher values indicate better performance:

• (1)

  Accuracy (ACC) refers to the proportion of correctly classified samples to the total number of samples, which can effectively evaluate datasets with balanced class distributions. We adopt ACC to assess the graph classification performance in both unsupervised and semi-supervised learning tasks.

• (2)

  F1-score is the harmonic mean of precision and recall, which is more suitable for datasets with imbalanced class distributions. Introducing F1-score into unsupervised learning and semi-supervised learning tasks can more comprehensively evaluate the model’s performance on different classes.

• (3)

  Receiver operating characteristic area under the curve (ROC-AUC) score measures the ability of a model to distinguish between positive and negative classes at all classification thresholds. Unlike metrics such as accuracy, it is relatively robust to class imbalance, making it particularly suitable for evaluating model performance on imbalanced or distribution-shifted target domains in transfer learning scenarios.

All experiments in GCL-ALG are conducted according to the settings in Table 3. We reproduce the experimental results of automated data augmentation methods (You et al., 2021; Suresh et al., 2021; Yin et al., 2022), the classical contrastive frameworks (You et al., 2020; Sun et al., 2019), and the Pretrain-GNN (Hu et al., 2019) method, where AutoGCL (Yin et al., 2022) reports on joint training strategies in a semi-supervised task. For AD-GCL (Suresh et al., 2021), we use the model with dynamically adjusted regularization parameters, i.e., AD-GCL-OPT. And for GraphCL (You et al., 2020), we adopt the default augmentation strategy, random4.

Experimental results and analysis

Detailed analysis of the encoder mechanism

To systematically evaluate the effectiveness of the dual-layer feature extraction framework and the synergy between its constituent modules, we conduct ablation experiments focusing on the independent contributions and interactive effects of structural modeling strategies based on self-attention mechanisms and edge load information. The experiments employ an SVM classifier, with training repeated 10 times under different random seeds. In each run, the unsupervised graph representation learning process is iterated for 100 epochs to ensure the stability and reliability of the results. Figure 3 reports the maximum, minimum, and average performance of GCL-ALG and its structural variants on two representative graph classification datasets, IMDB-B and PROTEINS. The variants include GCL-ALG-Attention, which contains only the self-attention-based GNN encoder; GCL-ALG-Weight, which includes only the edge-based GNN encoder; and GCL-ALG-None, which excludes both components. The results yield the following insights:

• Synergistic enhancement effect. From Fig. 3, it can be observed that for methods GCL-ALG-None, GCL-ALG-Weight, GCL-ALG-Attention, and GCL-ALG, their classification performances in both datasets show a progressive improvement. The GCL-ALG model, which jointly integrates both components, consistently outperforms all other variants in terms of maximum, minimum, and average performance. Specifically, GCL-ALG achieves average improvements of 1.39% on IMDB-B and 0.47% on PROTEINS compared to the GCL-ALG-None baseline. This not only validates the individual effectiveness of each component, but also highlights the positive complementarity between them.

• Differential module contributions. The experimental results reveal that GCL-ALG-Attention yields average gains of 0.83% and 0.29% on IMDB-B and PROTEINS, respectively, while GCL-ALG-Weight achieves only 0.48% and 0.15%. This discrepancy indicates that, in the context of unsupervised graph representation learning, the dynamically adaptive attention mechanism offers superior discriminative capacity compared to static structural measures such as edge load, enabling more effective identification of local structural features relevant to the task.

Hyperparameter analysis

In this section, we analyze the sensitivity of the Gumbel-Softmax temperature parameter $\tau$ in the GCL-ALG algorithm. $\tau$ controls the sharpness of the edge sampling probability distribution in the learnable view generator, directly affecting the balance between the diversity and the information completeness of the generated views. To explore the impact of $\tau$ on different types of graph data, we conduct experiments with $\tau \in 0.1,0.5,1.0,2.0,5.0$, using two representative datasets: IMDB-B and PROTEINS. As shown in Fig. 4, the IMDB-B dataset achieves the best performance of 74.37% when $\tau =0.5$, after which the performance gradually decreases as $\tau$ increases. The PROTEINS dataset reaches its peak performance of 76.54% at $\tau =1.0$ and maintains relatively stable performance within the range $\tau \in [0.1,1.0]$, but shows a significant performance drop when $\tau \ge 2.0$. The analysis indicates that IMDB-B performs best at a moderate $\tau$ value, suggesting that an appropriate degree of randomness helps alleviate the issue of local structure dependency. As a biologically meaningful graph dataset rich in features, PROTEINS maintains stable performance within the range $\tau \in [0.1,1.0]$, but exhibits a decline when $\tau \ge 2.0$, indicating that excessive randomness disrupts its inherent feature-structure correlations. It is worth noting that both datasets show a performance drop when $\tau \ge 2.0$, confirming that an overly high temperature introduces excessive noise, which weakens the effective gradient signal and affects the learning of the view generator. Overall, except for the performance degradation at higher temperature values, the model exhibits limited sensitivity to $\tau$ and good robustness, with the regularization and fully connected structures also contributing to mitigating performance fluctuations. Based on the above experimental results, we recommend setting the temperature parameter $\tau$ within the range $[0.5,1.0]$.