Heterogeneous text graph for comprehensive multilingual sentiment analysis: capturing short- and long-distance semantics

Multilingual sentiment analysis

Given the great importance of multilingual sentiment analysis in today’s interconnected world, several studies have been directed towards advancing methodologies, and enhancing accuracy to better perform sentiment analysis across languages, as previously introduced in Mercha & Benbrahim (2023). These studies aim to build language-agnostic systems that can learn predictive representations directly from multilingual content to predict the sentiments. The approaches outlined in the literature can be categorized into two approaches, depending on the methodologies used for text representation: word-based and character-based.

Previous research studies on word-based approaches investigated several classical machine learning classifiers with various handcrafted features, such as n-grams and bag-of-words (Narr, Hulfenhaus & Albayrak, 2012; Kincl et al., 2015; Abudawood, Alraqibah & Alsanie, 2018). In the study (Narr, Hulfenhaus & Albayrak, 2012), a suggestion was made to train naïve Bayes classifier on the generated multilingual data by a semi-supervised heuristic labeling method. The trained classifier was then tested on human annotated tweets using different n-gram features. The outcomes obtained from these experiments indicated that training a classifier using a mixture of multiple languages remained feasible, despite a slight decrease in performance. Kincl et al. (2015) introduced an approach that enhances the predictive capabilities of the support vector machine classifier by integrating the sentiment from local and chronological surrounding context. The achieved results across diverse datasets covering various languages and domains demonstrated the effectiveness of the proposed approach. Abudawood, Alraqibah & Alsanie (2018) suggested language-independent sentiment analysis approach which relies on word statistics and structural co-occurrence. The achieved results suggest that the proposed approach is better suited for online sentiment. Pribán & Balahur (2019) introduced and assessed three sentiment analysis systems designed to address three distinct sentiment analysis tasks within a multilingual context. The results of the experiments proved the proficiency of these systems within the context of multilingual capabilities.

Recently, the development of deep learning techniques has led to the widespread use of CNN and LSTM in MSA. Several studies, such as Medrouk & Pappa (2017), Deriu et al. (2017) and Attia et al. (2018), designed architectures based on CNN and randomly initialized word embeddings which learn directly from multilingual data. Medrouk & Pappa (2018) investigated CNN and LSTM in the multilingual context without any special feature considerations. The experimental results proved the effectiveness of the proposed architectures in both monolingual and multilingual settings, all without adopting any linguistic resources. Shakeel et al. (2020) proposed a hybrid architecture combining CNN and LSTM models to capture both n-gram features and long-term dependencies. The results indicated the model’s resilience against class label distribution skewness and its ability to maintain robust performance across various mixed languages. Manias et al. (2023) investigated four multilingual Transformer-based classifiers and zero-shot classification approach in MSA task. The experimental results revealed that Multilingual BERT-based classifiers achieved high performance and transfer inference when trained on multilingual data.

While word-based approaches are widely employed for handling MSA, numerous studies have introduced character-based approaches as an alternative. Most character-based methods leverage convolutional neural networks to construct efficient architectures that can extract valuable insights from character features (Becker et al., 2017; Zhang, Zhang & Chan, 2017a; Wehrmann et al., 2017; Wehrmann, Becker & Barros, 2018). In Becker et al. (2017), a translation-free deep neural approach for MSA was introduced, where three different neural architectures were compared using tweets from four distinct languages. The obtained results demonstrated that the suggested architecture was competitive with the baseline models, offering a notable improvement by decreasing the number of parameters and memory usage. Similarly, Zhang, Zhang & Chan (2017a) proposed a language-agnostic model, namely unicode character convolutional neural network (UniCNN), designed for sentiment classification. UniCNN was assessed on six Twitter datasets from distinct languages, and the outcomes demonstrated its superior performance over state-of-the-art models. In the study conducted in Wehrmann et al. (2017), a series of experiments were carried out to investigate the relationship between the performance of their proposed character-based CNN model and various architectural parameters. The findings underscored the efficiency of these models in multilingual contexts. Wehrmann, Becker & Barros (2018) described a multilingual character-based approach that simultaneously addresses language detection and sentiment analysis. Extensive experiments were conducted on tweets from various languages, and the results highlighted the superiority of the proposed approach compared to the baseline models. In a departure from these studies, Zhang, Zhang & Chan (2017b) proposed an approach that combines word and character features. The experimental results demonstrated the effectiveness of fusing deep features from both character and word levels within a CNN architecture.

Graph neural networks

Graph neural networks have gained considerable attention because of its convincing performance. GNNs were widely explored in various areas of natural language processing, including text classification (Yao, Mao & Luo, 2019), sentiment analysis (Li et al., 2022), information extraction (Li et al., 2023), and knowledge graph (Xia et al., 2023).

Earlier studies in the field of graph analysis typically rely on classical machine learning algorithms and hand-crafted features which are time-consuming and performance-limited. Motivated by the success of the methods developed for representation learning (Mikolov et al., 2013a, 2013b), graph representation learning methods have been invented to overcome the limitations of the traditional approaches (Zhang et al., 2018; Cai, Zheng & Chang, 2018). The goal of graph representation learning methods is to develop deep learning architectures that can combine information from the graph topologies and attributes into low-dimensional embeddings (Hamilton, Ying & Leskovec, 2017b). Several graph representation learning algorithms have been developed to explore deep semantic connections in the graph, such as GAT (Veličković et al., 2018), GraphSage (Hamilton, Ying & Leskovec, 2017a), GCN (Kipf & Welling, 2016), and GGNN (Li et al., 2016).

The majority of the suggested approaches in existing literature for multilingual sentiment analysis predominantly stem from either traditional machine learning or deep learning methods. Traditional machine learning approaches depend on statistical representation and employ various feature selection methods, but they suffer from a deficiency in semantic information, posing a significant challenge for sentiment analysis. On the other hand, deep learning-based methods rely on randomly initialized word embeddings or sparse character representations in conjunction with deep learning architectures. These approaches prioritize capturing short-distance semantics within local consecutive word sequences, neglecting the global co-occurrence of words throughout the entire corpus, which is crucial for conveying long-distance semantics.

In contrast to the aforementioned studies, in this work, we propose a novel approach to multilingual sentiment analysis that takes into account both short- and long-distance semantics to enrich the analytical process with deeper insights. This is accomplished by constructing a unified heterogeneous text graph for the entire multilingual corpus, integrating statistical, sequential, and semantic information. Subsequently, a slightly deep graph convolutional network is employed to acquire predictive representations by encouraging the transfer learning across languages.

Heterogeneous text graph construction

To represent the complete multilingual corpus, we create a unified heterogeneous text graph that incorporates both words and documents as nodes. The graph consists of two types of edges: word-document edges and word-word edges, connecting the respective nodes together. The word-document edges are constructed using statistical information. The weight of an edge between a word node and a document node is calculated using the term frequency-inverse document frequency (tf-idf) measure, which indicates the significance of a word within a document in the corpus. Therefore, the weight of edge connecting a word and a document is determined by the following formula:

(1) $$tf-idf(w,d,C)=tf(w,d)\times idf(w,C)$$where $tf(w,d)$ and $idf(w,C)$ denote respectively the relative frequency of word $w$ within document $d$ and the logarithmically scaled inverse fraction of documents of the corpus C containing the word $w$.

In addition, the word-word edges are established by considering two different aspects of language information: sequential and semantic properties. For capturing sequential relationships, we employ a sliding window approach to gather statistics on word co-occurrence across the entire corpus. The weight between two word nodes is then determined using the measure of positive pointwise mutual information (PPMI) (Church & Hanks, 1990). Mathematically, the weight of a word pair $w_i$, $w_j$ is calculated as follows:

(2) $$PPMI(w_i,w_j)=max\left(log\frac{p(w_i,w_j)}{p(w_i)p(w_j)},0\right)$$

(3) $$p(w_i,w_j)=\frac{\mathrm{\#}W(w_i,w_j)}{\mathrm{\#}W}$$

(4) $$p(w_i)=\frac{\mathrm{\#}W(w_i)}{\mathrm{\#}W}$$where $\mathrm{\#}W(w_i,w_j)$ is the number of sliding windows in which both words $w_i$ and $w_j$ co-occurred, $\mathrm{\#}W(w_i)$ is the number of sliding windows containing the word $w_i$, and #W is the total number of the sliding windows in the whole corpus.

Relying just on sequential information to establish word-word relationships may lead to a disconnected graph structure. To address this, we incorporate semantic information to connect languages and create a unified, interconnected graph. As the proposed MSA-GCN approach is language-agnostic, we adopt the method proposed by Conneau et al. (2017) to align monolingual word embedding spaces in an unsupervised manner. The method utilizes adversarial training to learn a linear mapping $W^{\ast }$ between two languages and it consists of two steps. Initially, a discriminator is trained to differentiate between word embeddings that have been mapped from the two languages, while the mapping is trained to prevent the discriminator from making correct predictions. In the second step, a synthetic dictionary is derived from the resulting shared embedding space, which is then used to refine the mapping through the closed-form Procrustes solution (Schönemann, 1966). For more details, let us assume that we have two languages, $L_1$ and $L_2$, each one is associated with a set of word embedding, $X=\{x_1,...,x_n\}$ and $Y=\{y_1,...,y_m\}$ respectively, trained independently on monolingual data. The discriminator is trained to distinguish between elements randomly sampled from $W^{\ast }X=\{W^{\ast }{x}_1,...,W^{\ast }{x}_n\}$ and Y. However, $W^{\ast }$ is trained to make $W^{\ast }X$ and Y similar as possible. The discriminator loss ${\mathcal{L}}_D$ and the mapping loss ${\mathcal{L}}_W$ are defined as follows:

(5) $${\mathcal{L}}_D({\theta }_D|W^{\ast })=-\frac{1}{n}{\sum }_{i=1}^n\log P_{{\theta }_D}(L_1|W^{\ast }{x}_i)-\frac{1}{m}{\sum }_{i=1}^m\log P_{{\theta }_D}(L_2|y_i)$$

(6) $${\mathcal{L}}_{W^{\ast }}(W^{\ast }|{\theta }_D)=-\frac{1}{n}{\sum }_{i=1}^n\log P_{{\theta }_D}(L_2|W^{\ast }{x}_i)-\frac{1}{m}{\sum }_{i=1}^m\log P_{{\theta }_D}(L_1|y_i)$$where ${\theta }_D$ is the discriminator parameters and $P_{{\theta }_D}(L_i|z)$ denotes the probability that a vector $z$ is the mapping of word embedding from $L_i$. During the training process, the discriminator and the mapping are learned to respectively minimize ${\mathcal{L}}_D$ and ${\mathcal{L}}_{W^{\ast }}$ using the training procedure of deep adversarial networks (Goodfellow et al., 2014). The learned mapping $W^{\ast }$ is then used to align monolingual word embeddings in a single space and the semantic edge weights are obtained based on the similarity of aligned multilingual word embeddings. Figure 2 illustrates the process of creating and weighting semantic relationships. Nevertheless, initial experiments indicate that directly combining sequential and semantic edge weights lead to worse results, due to the fact that the edge weights come from different distributions. Therefore, instead of directly using the cosine similarity, we propose to use the exponential of a scaled cosine similarity, which significantly increases the values of semantic weights and enhances the importance of transfer learning across languages. Formally, the weight of semantic edge of a word pair $w_i$, $w_j$ is calculated as follows:

(7) $$weigh{t}_{semantic}=\exp (\alpha .cosine\mathrm{\_}sim(w_i,w_j))$$

(8) $$cosine\mathrm{\_}sim(w_i,w_j)=\frac{x_i\cdot x_j}{\parallel x_i\parallel \parallel x_j\parallel }$$where $\alpha$ is a parameter for scaling the similarity between a pair of words, and $x_i$, $x_j$ denote respectively the aligned word embeddings of the words $w_i$ and $w_j$. To reduce the complexity, we use bilingual dictionaries (Conneau et al., 2017) to generate a set of linkages across languages rather than computing the similarity between the vocabulary of different languages to select the most similar words.

Slightly deep graph convolutional network

The vanilla graph convolutional network (GCN) (Kipf & Welling, 2016) is a graph neural network which follows a message-passing mechanism. GCN operates directly on a graph to learn node representations by aggregating the feature information from their topological neighbors. Formally, consider an attributed graph $\mathcal{G}=(\mathcal{V},\mathcal{E},A)$ where $\mathcal{V}$ is the set of $n=|\mathcal{V}|$ nodes, $\mathcal{E}$ is the set of edges, and $A\in {ℝ}^{n\times n}$ is the adjacency matrix of $\mathcal{G}$. For each connected pair of words $w_i$, $w_j$, $A(i,j)$ is the weight of the edge; otherwise, $A(i,j)=0$. In addition, each node is assumed to be connected to itself, i.e., $\mathrm{\forall }{w}_i\in \mathcal{V};(w_i,w_i)\in \mathcal{E}$ and $A(i,i)=1$. The normalized symmetric adjacency matrix of A is $\tilde{A}=D^{-\frac{1}{2}}A{D}^{-\frac{1}{2}}$, where D denotes the degree matrix of A, and $D_{ii}={\mathrm{\Sigma }}_j{A}_{ij}$. Let $X\in {ℝ}^{n\times d}$ be a feature matrix containing the embedding vectors of all nodes of $\mathcal{G}$, where $d$ denotes the dimension of the embedding vectors and each row $x_i$ of the matrix represents the embedding vector associated to the node $i$. The vanilla GCN model $f(X,A)$ learns the nodes embeddings based on two-layers as follows:

(9) $$f(X,A)=softmax(\tilde{A}ReLU(\tilde{A}X{W}^{(0)})W^{(1)})$$here $W^{(0)}\in {ℝ}^{d\times h}$ and $W^{(1)}\in {ℝ}^{h\times c}$ are the weight matrices of the first and second layer, respectively, with $h$ denotes the dimension of the hidden representation, and $c$ is the number of classes. The activation functions are defined as follows:

(10) $$ReLU(x)=max(x,0)$$

(11) $$softmax(x_i)=\frac{\exp (x_i)}{{\mathrm{\Sigma }}_j\exp (x_j)}$$

Inspired by the success of the vanilla GCN, we propose a slightly deep graph convolutional network which learns deeper and better node representations. We reformulate the vanilla GCN by increasing the depth of the architecture and adding a single feed forward neural network layer to aggregate information from beyond local neighborhoods and to learn long-range dependencies. Regarding the activation function, we adopt $tanh$ instead of ReLU, because it makes the learning easier, as it is zero centered and its values lie between $-1$ to $1$. However, the limitation faced by ReLU is the dying ReLU problem which decreases the learning ability of the model. Formally, our forward slightly deep graph convolutional network model is described as follows:

(12) $$H^{(1)}=tanh(\tilde{A}{H}^{(0)}{W}^{(0)})$$

(13) $$H^{(2)}=tanh(\tilde{A}{H}^{(1)}{W}^{(1)})$$

(14) $$H^{(3)}=softmax(H^{(2)}{W}^{(2)}+b)$$where $W^{(0)}\in {ℝ}^{d\times h}$ and $W^{(1)}\in {ℝ}^{h\times h}$ denote the weights of the first and second layer, respectively, $W^{(2)}\in {ℝ}^{h\times c}$ is the weight matrix of the dense layer and $H^{(0)}=X$. $H^{(i)}$ is the learned representations of nodes in the layer $i$. The $tanh$ activation function, is defined as:

(15) $$tanh(x)=\frac{\exp (x)-\exp (-x)}{\exp (x)+\exp (-x)}$$

To measure the classification performance of the slightly deep graph convolutional network, we adopt the cross-entropy loss across all labeled documents:

(16) $$\mathcal{L}=-{\sum }_{d\in {\mathcal{Y}}_{\mathcal{D}}}{\sum }_{c=1}^C{Y}_{dc}ln{H}_{dc}^{(3)}$$where ${\mathcal{Y}}_{\mathcal{D}}$ is the set of labeled document indices, and C is the number of classes.

Datasets

Throughout the experimental analysis, we adopt four datasets to evaluate the effectiveness of the proposed approach. These datasets include Multilingual Amazon Reviews Corpus (MARC) (Keung et al., 2020), Internet Movie Database (IMDB) (Maas et al., 2011), Allociné (Théophile Blard, 2020), and Muchocine (Mata et al., 2008).

• MARC is an extensive and diverse collection of Amazon reviews for multilingual text classification. The corpus comprises reviews from six languages: English, Japanese, German, French, Chinese, and Spanish. These reviews were gathered precisely during the period from 2015 to 2019. For each language, the training, development, and test sets consist of 200,000, 5,000, and 5,000 reviews, respectively.

• IMDB is a movie reviews dataset. It consists of reviews derived from the IMDB, a popular online database of movies, television shows, and related content. It is a binary sentiment classification dataset written in English. It is made up of 25,000 highly polar movie reviews for training and another 25,000 for testing.

• Allociné is a binary French-language sentiment analysis dataset which consists of movie reviews authored by Allociné website users, spanning from 2006 to 2020. These reviews cover various films and are categorized into positive and negative sentiments. In total, there are 100,000 positive and 100,000 negative reviews, which are further divided into three subsets: 160,000 for training, 20,000 for validation, and another 20,000 for testing.

• Muchocine is Spanish-language sentiment analysis dataset. The dataset comprises movie reviews written by Muchocine website users about various movies. Each movie in the dataset is associated with a longform movie review, the summary review, and the rating on a 1–5 scale. The dataset contains a total of 3,872 movie reviews.

Due to the syntactic structural disparities across the six languages, we consider reviews from four languages namely, English, German, French, and Spanish. Furthermore, we reduce the problem to binary classification. So, we assigned one and two stars to the negative class, four and five stars to the positive class, and three stars to the neutral class. Then, we construct 10 datasets for Amazon reviews based on different language combinations (EN-FR, EN-ES, EN-DE, FR-ES, FR-DE, ES-DE, EN-FR-ES, EN-FR-DE, EN-ES-DE, FR-ES-DE), and one for movie reviews by combining the three movie reviews datasets, IMDB, Allociné, and Muchocine.

Tables 1 and 2 show respectively the number of documents in training, validation, and test sets and the summary statistics of movie reviews dataset and Amazon reviews datasets. In addition, Tables 3 and 4 illustrate respectively a sample of Multilingual Amazon Reviews Corpus (MARC) and the constructed movie reviews dataset.

Baseline models

We compare the proposed approach against several baselines and state-of-the-art methods. The baselines include:

• Bi-LSTM Emb-Non-Static (Bi-LSTM-NS): a bi-directional LSTM (Liu, Qiu & Huang, 2016) with trainable word embeddings. Bi-LSTM represents the whole text with the last hidden state. The embeddings are initialized randomly and fine tuned during the training process.

• CNN Emb-Non-Static (CNN-NS): a convolutional neural network (Kim, 2014) with trainable word embeddings. The convolution is applied over the randomly initialized word embeddings to learn the word and text representations.

• Bi-LSTM Emb-Static (Bi-LSTM-S): a bi-directional LSTM (Liu, Qiu & Huang, 2016) with randomly initialized word embeddings. The word embeddings are not optimized during the training process.

• CNN Emb-Static (CNN-S): a convolutional neural network (Kim, 2014) with randomly initialized word embeddings. The word embeddings are not updated throughout the training process.

• Char-CNN: a small version of character-level convolutional network for text classification (Zhang, Zhao & LeCun, 2015). The designed architecture extracts efficient representation based on the character features.

• fastText: a simple baseline method for text classification (Joulin et al., 2017). The method averages the word features to construct efficient text representation that will be used for learning a linear classifier.

• Text level GCN (TL-GCN): text level graph neural network for text classification (Huang et al., 2019). The method uses a message passing mechanism to learn representation for each input text modeled with a graph.

• Graph-GCN: the vanilla GCN (Kipf & Welling, 2016), applied to the constructed heterogeneous text graph.

Experiment settings

In our experiments, we employed some common language-agnostic preprocessing techniques. These techniques involved removing URLs, eliminating special characters, converting text to lowercase, and removing HTML tags. The text was then tokenized using single space.

To optimize the proposed MSA-GCN approach, we conducted a series of experiments to determine the best hyperparameters. These experiments involve defining a grid of hyperparameter values and searching through all possible combinations of these values to find the best set of hyperparameters. Based on the experimental results we set the hyperparameters as follows:

• Low frequency terms: we remove terms that appeared less than five times.

• Size of the sliding window: 25. We investigate different sizes of sliding window (see Fig. 3) and we judge that the best sliding window size is 25, as the small window size may not provide enough global word co-occurrence information, whereas large window size could introduce extra edges between words that are not closely connected. As observed in Yao, Mao & Luo (2019), our initial experiments demonstrate that minor adjustments to the window size have minimal impact on the results.

• Word embeddings initialization: identity matrix. That is to say that each node is represented with a one-hot vector.

• $\alpha$: 7. We explored various values of $\alpha$ (refer to Eq. (7)) and determined that the most suitable value is 7 (refer to Fig. 4).

• Node embedding dimension: 200 for both layers of the slightly deep graph convolutional network.

• maximum of training epochs: 200.

• Optimizer: Adam optimizer (Kingma & Ba, 2015).

• Learning rate: 0.002.

• Stop training criteria: if the validation loss does not decrease for five consecutive epochs.

For the baseline models, we use the default hyper-parameter for fastText, TL-GCN, and Char-CNN as described in their original articles. These hyper-parameters were chosen because they consistently delivered optimal performance across diverse datasets in their original research (Zhang, Zhao & LeCun, 2015; Joulin et al., 2017; Huang et al., 2019). However, we performed hyper-parameter tuning to choose the best hyper-parameter for Bi-LSTM-NS and CNN-NS. So, the results were 100 for the embedding dimension and number of units for Bi-LSTM-NS, 0.5 for dropout, and $L_2$ for the layer weight regularizer with a weight of 0.01. Also, 100 for the embedding dimension and number of filters for CNN-NS, while the kernel size is 3. We use the same settings for Bi-LSTM-S, CNN-S as Bi-LSTM-NS and CNN-NS respectively. All four models are trained using the Adam optimizer with a learning rate of 0.001. For all baseline models we utilize randomly initialized word embedding rather than pre-trained word embedding because we explore these models in multilingual settings.

The proposed approach is implemented based on Python and PyTorch 1.11.0+cpu. The experiments are conducted on a machine with an Intel(R) Xeon(R) Platinum 8270 CPU @ 2.70 GHz and 200 GB RAM.

Results analysis

A comprehensive experiment is conducted on several datasets. We evaluate each model 30 times based on the accuracy, and we report the mean $\pm$ standard deviation. Tables 5 and 6 show the results of the MSA-GCN approach against diverse baseline models. The obtained results clearly demonstrate that the suggested approach significantly outperforms all the baselines (p-value < 0.05 based on student t-test) on eight datasets over 11, which prove the effectiveness of the proposed approach.

For more detailed performance analysis, we notice that the proposed approach consistently outperforms all baseline models in almost all datasets. On the other end of the spectrum, Char-CNN achieved the poorest results across almost all datasets. This discrepancy can be attributed to the limitation of encoding multilingual text using just character features, which fails to capture meaningful information due to syntactic variations between languages. The Bi-LSTM-S and CNN-S models achieved moderate results. This can be attributed to the strong interdependence between the learned document representation and word embeddings. Conversely, random word embeddings lack the ability to capture syntactic or semantic information, rendering the task of representation learning exceedingly challenging. Furthermore, our analysis uncovers that FastText’s performance is consistently modest across all datasets. This observation underscores the limitations of forming insightful document representations for MSA through a simple averaging of word embeddings. Remarkably, CNN-S outperforms Bi-LSTM-S by a considerable margin due to its aptitude for extracting local and position-invariant features, which are critical for sentiment analysis. Also, we observe that Bi-LSTM-NS and CNN-NS obviously beat Bi-LSTM-S and CNN-S, respectively, since they can learn valuable word representations in a supervised manner.

The graph-based methods TL-GCN and Graph-GCN perform quite well, and show competitive performances, indicating the efficacy of the graph-based approaches. This suggests the efficiency of modeling the entire multilingual corpus with a single graph and its ability to capture the relations across words and documents. Also, modeling each document with a graph allows to capture the relations between words by learning more expressive edges. The proposed approach consistently outperforms TL-GCN in all datasets, indicating the effectiveness of the way proposed for constructing the graph from the entire multilingual corpus. We also observed that the proposed approach outperforms Graph-GCN, which proves the performance of the slightly deep graph convolutional network to learn efficient representation for words and documents better than the vanilla GCN.

Ablation study

We conduct several ablation studies to further investigate the contribution of each component of MSA-GCN approach, and the results are shown in Figs. 5–8.

Discussion

The results of the experiments indicate that the MSA-GCN approach is proficient in acquiring predictive representations for both words and documents. Additionally, it demonstrates a high level of performance in multilingual sentiment analysis. Two primary factors account for the superiority of MSA-GCN. Firstly, the method we introduce involves modeling the entire corpus through a unified heterogeneous graph, incorporating semantic, sequential, and statistical information. This method allows the construction of a graph that captures the global information about word relations in the whole corpus and provides a structure that enables the transfer learning across languages. In addition, it facilitates the propagation of sentimental information among documents through the connections established by words, which helps predict sentiment of documents with low-resource languages and documents with scarce words. Furthermore, it bridges the linguistic gap by leveraging semantic connections between languages. Secondly, the slightly deep graph convolutional network enables the learning of deep predictive embeddings for both words and documents by propagating feature information from neighboring nodes. This propagation allows each node to gather information not only from its direct neighbors but also from nodes that are further away in the graph. This enables the network to capture both local and global context which considerably increases the multilingual sentiment analysis performance.

Although the proposed approach was tested on the problem of multilingual sentiment analysis, it will serve as a powerful approach for tackle other natural language processing such as named entity recognition in a multilingual settings.

Even though the MSA-GCN produced efficient results and outperformed the state-of-the-art models, it has limitations. On one hand, MSA-GCN builds a single graph for the entire corpus, meaning that the size of the graph scales with the corpus size, resulting in excessive memory usage in big corpora. On the other hand, MSA-GCN is intrinsically transductive. The transductive property implies that the model’s applicability is restricted to the specific corpus on which it was initially trained. Therefore, MSA-GCN cannot support the online testing, as the structural composition of the graph and its associated parameters are intricately related to the characteristics of the original training corpus. This restriction restrains the model’s ability to generalize effectively to new and unseen data, limiting its utility in dynamic and evolving environments where adaptability is crucial. Another significant limitation of the MSA-GCN is that it does not consider the aspect of word ordering, either on the graph construction or the development of the slightly deep graph convolutional network, which brings more insights to the analysis.