Label-Guided relation prototype generation for Continual Relation Extraction

Problem formulation

In continual relation extraction, there is a series of K tasks $\left\{T_1,T_2,\dots ,T_k\right\}$, where the k-th task has its own training set D_k and relation set R_k. The relation sets of different tasks are disjoint. Each instance in the training set D_k is represented as (x_i, r_i), where x_i is the input text, and r_i is the relation label. The objective of continual relation learning is to avoid catastrophic forgetting of previous learning tasks while learning new tasks, ensuring that satisfactory performance can still be achieved within the relation set ${\hat{R}}_k$, where ${\hat{R}}_k={\bigcup }_{i=1}^k{R}_i$ is the relation set already observed till the k-th task.

Additionally, an episodic memory space ${\hat{M}}_k={\bigcup }_{i=1}^k{M}_i$ will be used to store some typical samples of previously learned data for memory replay. Here, M_i is selected from the training set D_i of the i-th task. Constrained by memory size and resource consumption, storing all observed data is impractical, so typically, only the most representative m instances are stored.

Framework

The overall framework is shown in Fig. 1. For a new task T_k, our proposed method mainly includes four stages, with the specific training process shown in Algorithm 1 . (1) New task training and label knowledge injection (Line 2∼10): The purpose of this stage is to learn the newly emerging relations R_k in T_k. Relation labels will be inserted into the text at this stage to capture specific descriptions of the relationships in the text. Additionally, for each r in R_k, the K-means algorithm will be used to select representative instances from D_k to be stored in memory ${\hat{M}}_k$. (2) Relation prototype generation (Line 11): The embeddings of relation label r and the instances stored in memory are used as inputs for multi-head attention to generate relationship prototypes h_r for each observable relationship type. (3) Memory replay (Line 12∼16): Contrastive learning is performed using the classic instances and the generated relationship prototypes to maintain the knowledge previously learned. (4) Multi-similarity distillation (Line 17∼21): We conduct distillation learning using the embedding space based on the outputs of the encoder from the previous stage and the current encoder.

New task training and label knowledge Infusion

For a new task T_k, we train on its training set D_k. Given an input sentence x_i and two entityes within the sentence, we insert special markers [E11]/[E12] and [E21]/[E22] at the beginning and end of the head and tail entities, respectively, to indicate their positions in the text.

In this stage, we aim to enrich the semantic information within the relation labels. Specifically, we concatenate the relation label with the sentence using the token [SEP]. Formally, for an instance x_i and its corresponding relation label r_i, the sequence fed into the encoder takes the form: $\left[\mathrm{CLS}\right]r_i\left[\mathrm{SEP}\right]x_i\left[\mathrm{SEP}\right]$. Although the model’s objective is to predict the semantic relation between given entity pairs, the knowledge gained during this process shapes the representations of label tokens. Consequently, the model learns to generate embeddings for label tokens that capture semantic information based on their contextual usage in a sentence. To prevent the model from solely relying on the label and neglecting the context sentence, we employ a random inserting approach. This means that the relation label will be inserted into a sentence with a certain probability.

When learning a new task, for each training instance, we use the embeddings of [E11] and [E21] as representations for the head and tail entities. The input of the classifier is calculated using: (1)${\displaystyle \begin{array}{c}\hfill h=W_1\left[h^{11};h^{21}\right]+b\hfill \end{array}}$

where h¹¹, h²¹ ∈ ℝ^d are the hidden representations of the tokens [E11], [E21] respectively. d is the dimension of the hidden embeddings output from encoder. [;] denotes the concatenation operation. W₁ ∈ ℝ^d×2d and b ∈ ℝ^d are two trainable parameters.

We calculate the probability distribution of the current instance being classified into each relation using the softmax function: (2)${\displaystyle \begin{array}{c}\hfill P\left(x;{\theta }_k\right)=softmax\left(W_{2}h\right)\hfill \end{array}.}$

Here, $W_2\in {\mathbb{R}}^{\left|{\tilde{R}}_k\right|\times d}$ is the trainable parameter in the classifier. The loss function for the current learning task is computed as follows: (3)${\displaystyle \begin{array}{c}\hfill L_1=-\frac{1}{\left|D_k\right|}\sum _{x_i\in D_k}\sum _{r\in R_k}{\delta }_{r_i,r}logP\left(r|x_i;{\theta }_k\right)\hfill \end{array}.}$

$P\left(r|x_i;{\theta }_k\right)$ represents the probability that the current model θ_kclassifies the input x_i as relation r. r_i is the label of x_i such that if r = r_i, δ_ri,r = 1, and 0 otherwise.

Prototype generation

To fully leverage the semantic knowledge embedded in relation labels, we use the labels to guide the generation of relation prototypes. Specifically, for a particular relation type r, its hidden embedding representation h_r is obtained by averaging the encoder outputs of each word in the relation label. For instance, for the relation label “Organization founded”, the embedding representation is calculated as follows: $h_{org:founded}=\frac{1}{2}\left(h_{Organization}+h_{founded}\right)$.

In contrast to the previous approach, which simply averaged the embedding representations of instances stored in memory to obtain relation prototypes, we aim to make relation prototypes attentive to different patterns within instances. Inspired by KIP-framework (Zhang et al., 2022), we utilize the hidden representation of the relation label as the query vector and employ a multi-head attention mechanism to form relation prototypes.

Specifically, we first obtain the hidden embedding representations of instances in memory with relation class r through encoder: (4)${\displaystyle \begin{array}{c}\hfill H_{x_r}=\left\{h_x|\forall x\in M_r\right\}\hfill \end{array}}$

where M_r represents all instances in memory with relation class r, and h_x ∈ ℝ^d is the embedding representation of instance x. The output of the i-th attention head in the multi-head operation is denoted as p_i ∈ ℝ^d and is computed as follows: (5)${\displaystyle \begin{array}{c}\hfill Attention\left(Q,K,V\right)=softmax\left(\frac{Q{K}^T}{\sqrt{d}}\right)V\hfill \end{array}}$ (6)${\displaystyle \begin{array}{c}\hfill p_i=Attention\left(h_r{W}_i^Q,H_{x_r}{W}_i^K,H_{x_r}{W}_i^V\right)\hfill \end{array}}$

where h_r is the hidden embedding corresponding to the relationship label r, and $W_i^Q,W_i^K,W_i^V\in {\mathbb{R}}^{d\times d_h}$, where d_h = d/N, with N being the number of attention heads.

At last, the prototype of relation r is calculated by (6): (7)${\displaystyle \begin{array}{c}\hfill p_r=LN\left(W_3\left[h_1;h_2;\dots ;h_N\right]\right)\hfill \end{array}}$

where LN(⋅) is the operation of layer normalization, [;] denotes the concatenation operation. W₃ ∈ ℝ^d×Nd_h is a trainable matrix. N is the number of the attention head. p_r is the prototype of relation r.

Contrastive replay

After learning a new task, to mitigate the problem of catastrophic forgetting caused by changes in model parameters, we need to replay the stored typical instances in memory. In this phase, we employ the contrastive learning for memory replay. Specifically, we use the InfoNCE loss to train the model. For a specific instance x_i, the loss is computed as follows: (8)${\displaystyle \begin{array}{c}\hfill L_c=-\frac{1}{\left|\widehat{M_k}\right|}\sum _{x_i\in \widehat{M_k}}log\frac{exp\left(h_{x_i}\cdot p_{r_i}/{\tau }_1\right)}{\sum _{r\in \widehat{R_k}}exp\left(h_{x_i}\cdot p_r/{\tau }_1\right)}\hfill \end{array}}$

where r_i is the relation label of instance x_i, p_ri is the corresponding relation prototype, and τ₁ is the temperature parameter. Utilizing contrastive learning methods provides a more effective classification space, preserving previously learned knowledge while incorporating new knowledge from the current task. However, contrastive replay may alter the distribution of instance’s representation. To address this drawback, we propose a multi-similarity distillation method.

Multi-similarity distillation

In certain previous works, distillation is employed during the memory replay process to preserve prior knowledge. For instance, CRL use the similarity metric between relations in memory as memory knowledge. However, knowledge distillation based on relation prototype may lead to the loss of specific samples of individual instances. In this step, our goal is to ensure that the current model produces features similar to those of the previous model while keeping the model’s classification space as clear as possible to avoid confusion.

Considering a specific sample x_i, with the current model and the previous model’s output hidden representations denoted as $h_{x_i}^l$ and $h_{x_i}^{l-1}$, respectively, we aim to make these two representations as similar as possible. Then, the first term of distillation loss function is computed as follows: (9)${\displaystyle \begin{array}{c}\hfill L_{nd}=-\frac{1}{\left|\widehat{M_k}\right|}\sum _{x_i\in {\hat{M}}_k}\left(1-cos\left(h_x^l,h_x^{l-1}\right)\right)\hfill \end{array}}$

where cos(⋅) represents the cosine similarity between two vectors.

Furthermore, to enhance the discriminability of hidden representations of instances of different classes, we employ a contrastive learning approach. Inspired by the multi-similarity loss (Wang et al., 2019a), we introduce a multi-similarity distillation method. We define the similarity between instance x_i and x_j as S_ij: (10)${\displaystyle \begin{array}{c}\hfill S_{ij}=h_{x_i}^T{h}_{x_j}\hfill \end{array}.}$

Subsequently, we leverage this similarity to mine hard positive and negative samples. The selection criteria for negative sample pairs $\left\{x_i,x_j^{-}\right\}$ is: (11)${\displaystyle \begin{array}{c}\hfill S_{ij}^{-}>{min}_{r_k=r_i}{S}_{ik}-\varepsilon \hfill \end{array}.}$

Similarly, the selection criteria for positive sample pairs $\left\{x_i,x_j^{+}\right\}$ is: (12)${\displaystyle \begin{array}{c}\hfill S_{ij}^{+}>{min}_{r_k\ne r_i}{S}_{ik}+\varepsilon \hfill \end{array}.}$

Here, ϵ is a hyperparameter. The set containing all selected positive sample pairs is denoted as N_i, and the set of negative pairs is denoted as P_i. By doing so, we filter out sample pairs with less informative content, thus improving the efficiency of the model. Then we apply a soft weighting to each sample pair based on their similarity. The calculation of weights of positive pairs $w_{ij}^{+}$ and negative pairs $w_{ij}^{-}$ is as follows: (13)${\displaystyle \begin{array}{c}\hfill w_{ij}^{-}=\frac{1}{e^{\beta \left(\lambda -S_{ij}\right)}+\sum _{k\in N_i}{e}^{\beta \left(S_{ik}-S_{ij}\right)}}\hfill \end{array}}$ (14)${\displaystyle \begin{array}{c}\hfill w_{ij}^{+}=\frac{1}{e^{-\alpha \left(\lambda -S_{ij}\right)}+\sum _{K\in P_i}{e}^{-\alpha \left(S_{ik}-S_{ij}\right)}}\hfill \end{array}}$

where α, β, λ are hyper-parameters At last, the second term of distillation loss function is computed as: (15)${\displaystyle \begin{array}{c}\hfill L_{ms}=-\frac{1}{\left|\widehat{M_k}\right|}\sum _{x_i\in \widehat{M_k}}\left(\begin{array}{c}\hfill \frac{1}{\alpha }log\left[1+\sum _{k\in P_i}{e}^{-\alpha \left(S_{ik}-\lambda \right)}\right]\hfill \\ \hfill +\frac{1}{\beta }log\left[1+\sum _{k\in N_i}{e}^{\beta \left(S_{ik}-\lambda \right)}\right]\hfill \end{array}\right)\hfill \end{array}.}$

Compared to methods that randomly select sample pairs or choose the most challenging sample pairs, the multi-similarity distillation approach, when weighting sample pairs, considers not only the similarity of the sample pairs themselves but also tasks into account other sample pairs. This assigns more weight to sample pairs that carry more information, making the method more robust. Through this approach, the output of the current model not only aligns with that of the previous model but also leads to the aggregation of instances of the same category in the embedding space, making it easier to distinguish instances of different categories.

Overall, the distillation loss function is defined as: (16)${\displaystyle \begin{array}{c}\hfill L_d=L_{nd}+L_{ms}.\hfill \end{array}}$

Datasets

In our experiment, we utilized two benchmark datasets, FewREL (https://github.com/thunlp/FewRel) and TACRED (https://catalog.ldc.upenn.edu/LDC2018T24).

The FewREL (Han et al., 2018) datasets is a dataset for few-shot relation classification. To maintain consistency with previous benchmark models, we employed its version with 80 relations, each consisting of 700 instances.

The TACRED (Zhang et al., 2017) dataset is a large-scale relation extraction dataset comprising 42 different relations and a total of 106,264 instances. To ensure a balanced distribution of instances for each relation category, consistent with previous benchmark models, the number of training samples for each relation is restricted to 320, and the number of test samples for each relation is limited to 40.

Experimental setup and evaluation metrics

For a fair comparison, we employ the same setting and obtain the divided data from the open-source code of [5], [6] to guarantee exactly the same task sequence. We report the average accuracy of five different sampling task sequences. The number of stored instances in the memory for each relation is 10 for all methods.

Baseline

We compared the experimental results with the baseline models listed below.

Embedding aligned EMR (EA-EMR) (Wang et al., 2019b) enhances adaptability and knowledge retention amidst changing data and relations by anchoring sentence embeddings and using an explicit alignment model.

Episodic memory activation and reconsolidation (EMAR) (Han et al., 2020) activates episodic memory and reconsolidate with relation prototypes during both new and memorized data learning.

Curriculum-meta learning (CML) (Wu et al., 2021) employs a curriculum-meta learning approach to swiftly adapt to new tasks and mitigate interference from previous work.

Relation prototypes CRE (RP-CRE) (Cui et al., 2021) employs relation prototypes and multi-head attention mechanisms to address the issue of the hidden representation space being disrupted after learning new tasks.

Consistent representation learning (CRL) (Zhao et al., 2022) introduces supervised contrastive learning and knowledge distillation to maintain previously learned knowledge.

Continual relation extraction framework with contrastive learning (CRECL) (Hu et al., 2022) consists of a classification network and a contrastive network, decouple the representation and classification process

Classifier decomposition (CDec) (Xia et al., 2023) introduces a classifier decomposition framework to address biases in CRE, promoting the learning of robust representations.

Main results

The overall performance of our proposed model on two datasets is shown in Table 1. All baseline model results mentioned in the table are cited from their respective papers. From the analysis of Table 1, the following conclusions can be drawn:

• Our model achieves competitive results compared to existing baseline models, obtaining the best results in most tasks. This demonstrates the effectiveness of our proposed method in CRE.

• Our model did not achieve the best results when evaluating Task 1 on both datasets. This is because, in the initialization phase, there is no need to mitigate catastrophic forgetting through relation prototypes and knowledge distillation. As tasks continue to arrive, maintaining previously learned knowledge becomes more crucial, leading to better performance for our model.

• All models perform better on the FewRel dataset compared to the TACRED dataset. This could be attributed to the imbalanced distribution of relation types and the relatively longer sentences in the TACRED dataset, which increases the search space and may lead to performance degradation.

Ablation study

To validate the impact of the Label Knowledge Infusion (LKI) module and the Multi-Similarity Distillation (MSD) module on the model’s performance, we conducted an ablation study, with the results shown in Table 2. Specifically, for w/o LKI (w/o context), We do not randomly insert relation labels into the text context when training on new tasks. For w/o LKI (special token), we replaced relation labels with special tokens. In other words, during the training of a new task, a special token was randomly inserted into each training instance for each relation, and this special token was used to generate relation prototypes through multi-head attention operations. For w/o MSD, we did not employ multi-similarity distillation during the memory replay stage to maintain previously learned knowledge. As for avg. prototype, we obtained relation prototypes by averaging the hidden representations of typical samples. The results from Table 2 indicate that removing specific modules leads to a decrease in model performance, demonstrating the effectiveness of each module in our model. Not using MSD for knowledge distillation during the memory replay phase disrupts the distribution of relation embeddings in the embedding space, resulting in poorer performance on previously learned relation categories.

To figure out how the label knowledge infusion module work, we conducted the following experiment. Firstly, we employed t-SNE to visualize the distribution of relation labels in the embedding space at different epochs during the learning of new tasks. This aimed to confirm whether relation labels could acquire more semantic knowledge through training via textual context. The results are depicted in Fig. 2. During training new task, with an increase in the number of iterations, the embedding of relation labels moves from the boundary of the embedding space toward the center. This indicates that, through self-attention mechanism (Vaswani et al., 2017) and gradient propagation, relation labels can capture different patterns in training instances. If this process is not conducted, directly using relation label lacking contextual semantics to generate relation prototypes will result in a significant performance drop. Subsequently, replacing relation labels with special tokens also led to a decrease in model performance. We believe that, compared to using special tokens, relation labels provide a highly generalized semantic understanding of the corresponding relationship. This plays a crucial role both in injecting semantic information into tokens representing relation labels and in generating relation prototypes.

Analysis of memory size

To mitigate catastrophic forgetting, when new data arrives, we need to select a certain number of instances for memory replay using the k-means algorithm. The selection is limited by the memory size. Usually, memory size is a significant factor influencing model performance. Due to resource constraints and computational limitations, only a small number of representative instances are stored. This requires that the model is minimally affected by changes in memory size. To investigate the performance of our model under different memory sizes, we conducted experiments with memory sizes set to 2, 5, and 15 instances, respectively. We compared the results with baseline models CRL and CRECL. The experimental results are shown in the Fig. 3.

When the memory size is set to 2 and 5, the performance of our proposed model is better than the baseline models. We believe this phenomenon can be attributed to two reasons: (1) During the new task training phase, we inject rich semantic information into the relation labels, acting as a knowledge base. (2) We employ a multi-attention mechanism to generate relation prototypes, making full use of the limited information contained in the memory.

When the memory size is set to 15, our model did not consistently achieve the best performance in some cases. We believe that with a memory size of 15, the instances stored in memory already provide sufficient information for the model to retain prior knowledge. This results in our proposed method having limited impact on performance improvement.

Effectiveness of multi-loss distillation

The main reason for catastrophic forgetting in CRE is the presence of similar relations in the learning data stream. Due to their similar semantics, instances of these relations tend to cluster together in the embedding space, leading to classification difficulties. Table 3 shows the performance of CRL and CRECL on different groups of similar relations, where it can be observed that the more similar the learned relations are, the greater the performance decline of the model. Therefore, to improve model performance, it needs to have the ability to distinguish similar relations. Additionally, the proposed model demonstrates a smaller performance decline after learning similar relations compared to the baseline models, proving its stronger capability to distinguish similar relations.

To investigate the impact of multi-loss distillation on the distribution of instances of different relations in the embedding space, we selected five highly similar relations from the TACRED dataset and visualized the distribution of their corresponding samples using t-SNE. From Fig. 4, it can be seen that through multi-loss distillation, instances with the same relation label tend to cluster together in the embedding space, while there is a clear boundary between instances of different relations.