Clustering assessment in weighted networks

Basic definitions

Let G(V, E) be an undirected graph of order n = |V| and size m = |E|. In the case of a weighted graph¹ $\tilde{G}\left(V,\tilde{E}\right)$, we will denote $\tilde{m}={\sum }_{e\in \tilde{E}}w\left(e\right)$ the sum of all edge weights. Given S ⊂ G a subset of vertices of the graph, we have n_S = |S|, m_S = |{(u, v) ∈ E:u ∈ S, v ∈ S}|, and in the weighted case ${\tilde{m}}_S={\sum }_{\left(u,v\right)\in \tilde{E}:u,v\in S}w\left(\left(u,v\right)\right)$. We use w_uv instead of w((u, v)). Note that if we treat an unweighted graph as a weighted graph with weights 0 and 1 (1 if two vertices are connected by an edge, 0 otherwise), then $m=\tilde{m}$ and $m_S={\tilde{m}}_S$ for all S ⊂ V. Associated to G there is its adjacency matrix A(G) = (A_ij)_1≤i,j≤n where A_ij = 1 if (i, j) ∈ E, 0 otherwise. We insist that A(G) only take binary values 0 or 1 to indicate existence of edges, even in the case of weighted graphs. For the weights we will always use the weight function w((i, j)) = w_ij.

The following definitions will also be needed later on:

• c_S = |{(u, v) ∈ E:u ∈ S, v⁄ ∈ S}| is the number of edges connecting S to the rest of the graph.

• ${\tilde{c}}_S={\sum }_{\left(u,v\right)\in E:u\in S,v⁄\in S}{w}_{uv}$ is the natural extension of c_S to weighted graphs; the sum of weights of all edges connecting S to G∖S.

• $\tilde{d}\left(u\right)={\sum }_{v⁄=u}{w}_{uv}$ is the natural extension of the vertex degree d(u) to weighted graphs; the sum of weights of edges incident to u.

• d_S(u) = |{v ∈ S:(u, v) ∈ E}| and ${\tilde{d}}_S\left(u\right)={\sum }_{v\in S}{w}_{uv}$ are the (unweighted and weighted, respectively) degrees² restricted to the subgraph S.

• d_m and ${\tilde{d}}_m$ are the median values of d(u), u ∈ V³.

The left column in Table 1 shows the community scoring functions for unweighted networks defined in Yang & Leskovec (2015). These functions characterize some of the properties that are expected in networks with a strong community structure, with more ties between nodes in the same community than connecting them to the exterior. There are scoring functions based on internal connectivity (internal density, edges inside, average degree), external connectivity (expansion, cut ratio) or a combination of both (conductance, normalized cut, and maximum and average out degree fractions). Uparrow (respectively, downarrow) indicates the higher (resp., lower) the scoring function value the stronger the clustering.

On the right column we propose generalizations to the scoring functions which are suitable for weighted graphs while most closely resembling their unweighted counterparts. Note that for graphs which only have weights 0 and 1 (essentially unweighted graphs) each pair of functions is equivalent (any definition that did not satisfy this would not be a generalization at all).

• Internal Density, Edges Inside, Average Degree: These definitions are easily and naturally extended by replacing the number of edges by the sum of their weights.

• Expansion: Average number of edges connected to the outside of the community, per node. For weighted graphs, average sum of edges connected to the outside, per node.

• Cut Ratio: Fraction of edges leaving the cluster, over all possible edges. The proposed generalization is reasonable because edge weights are upper bounded by 1 and therefore relate easily to the unweighted case. In more general weigthed networks, however, this could take values well over 1 while lacking many “potential” edges (as edges with higher weights would distort the measure). In general bounded networks (with bound other than 1) it would be reasonable to divide the result by the bound, which would result in the function taking values between 0 and 1 (0 with all possible edges being 0 and 1 when all possible edges reached the bound).

• Conductance and Normalized Cut: Again, these definitions are easily extended using the methods described above.

• Maximum and Average Out Degree Fraction: Maximum and average fractions of edges leaving the cluster over the degree of the node. Again, in the weighted case the number of edges is replaced by the sum of edge weights.

Some of the introduced functions (internal density, edges inside, average degree, clustering coefficient) take higher values the stronger the clusterings are, while the others (expansion, cut ratio, conductance, normalized cut, out degree fraction) do the opposite.

Clustering coefficient

Another possible scoring function for communities is the clustering coefficient or transitivity: the fraction of closed triplets over the number of connected triplets of vertices. A high internal clustering coefficient (computed on the graph induced by the vertices of a community) matches the intuition of a well connected and cohesive community inside a network, but its generalization to weighted networks is not trivial.

There have been several attempts to come up with a definition of the clustering coefficient for weighted networks. One is proposed in Barrat et al. (2004) and is given by $c_i=\frac{1}{\tilde{d}\left(i\right)\left(d\left(i\right)-1\right)}{\sum }_{j,h}\frac{w_{ij}+w_{ih}}{2}{A}_{ij}{A}_{jh}{A}_{ih}$. Note that this gives a local(i.e. defined for each vertex) clustering coefficient.

While this may work well on some weighted networks, in the case of complete networks (e.g., such as those built from correlation of time series as in Renedo & Arratia (2016), we obtain (1)${\displaystyle c_i=\frac{1}{\tilde{d}\left(i\right)\left(d\left(i\right)-1\right)}\sum _{j,h}\frac{w_{ij}+w_{ih}}{2}=\frac{\sum _{jh}{w}_{ij}+\sum _{jh}{w}_{ih}}{\tilde{d}\left(i\right)\left(n-2\right)\cdot 2}=\frac{\left(n-2\right)\sum _j{w}_{ij}+\left(n-2\right)\sum _h{w}_{ih}}{\tilde{d}\left(i\right)\left(n-2\right)\cdot 2}=\frac{\left(n-2\right)\tilde{d}\left(i\right)+\left(n-2\right)\tilde{d}\left(i\right)}{\tilde{d}\left(i\right)\left(n-2\right)\cdot 2}=1,}$ which does not give any information about the network.

An alternative was proposed in McAssey & Bijma (2015) with complete weighted networks in mind(with weights in the interval [0, 1]), which makes it more adequate for our case.

• For t ∈ [0, 1] let A_t be the adjacency matrix with elements $A_{ij}^t=1$ if w_ij ≥ t and 0 otherwise.

• Let C_t the clustering coefficient of the graph defined by A_t.

• The resulting weighted clustering coefficient is defined as (2)${\displaystyle \tilde{C}={\int }_0^1{C}_{t}dt}$

For networks where the weights are either not bounded or bounded into a different interval than [0, 1], the most natural approach is to simply take (3)${\displaystyle \tilde{C}=\frac{1}{\bar{w}}{\int }_0^{\bar{w}}{C}_{t}dt,}$ where $\bar{w}$ can be either the upper bound or, in the case of networks with no natural bound, the maximum edge weight. The computation of this integral, which can be expressed as a sum of the values of C_t (a finite amount) is detailed in the “Weighted clustering coefficient”.

It is a desirable property that the output of scoring functions remain invariant under uniform scaling, that is, if we multiply all edge weights by a constant ϕ > 0, as the community structure of the network would be the same. This holds for all of the measures of the third group, which combine the notions of internal and external connectivity.

This means that these scores will be less biased in favour of networks with high overall weight (for the internal connectivity based scores) or low overall weight (for the external connectivity ones). It is particularly interesting for networks with weights that are not naturally upper bounded by one, and facilitates comparisons between networks with completely different weight distributions. When we compare each network’s scores to those of a randomized counterpart generated by the switching model, though, the total weight is kept constant, so even scores without this property could still give valuable information.

Let ${\tilde{G}}_{\varphi }\left(V,{\tilde{E}}_{\varphi }\right)$ be the weigthed graph obtained by multiplying all edge weights in $\tilde{E}$ by real positive number ϕ. In this case, n_Sϕ = n_S, m_Sϕ = ϕm_S, c_Sϕ = ϕc_S, d_Sϕ(u) = ϕd_S(u). This means that the internal density, edges inside, average degree, expansion and cut ratio behave linearly (with respect to their edge weights). Conductance, normalized cut and maximum and average out degree fractions, on the other hand, remain constant under these transformations. Since the notion of community structure is generally considered in relation to the rest of the network (a subset of vertices belong to the same community because they are more connected among themselves than to vertices outside of the community), it seems reasonable to consider that the same partitions on two graphs whose weights are the same up to a multiplicative positive constant factor have the same scores. This makes the scores in the third group, the only ones for which this property holds, more adequate in principle.

For the chosen definition of clustering coefficient this property also holds, as all terms in the integral in equation (3) behave linearly (the proof is immediate with a change of variables), and that linear factor cancels out.

Modularity

As for the modularity (Newman, 2006b), it is defined as: (4)${\displaystyle Q=\frac{1}{2\tilde{m}}\sum _{ij}\left[w_{ij}-\frac{\tilde{d}\left(i\right)\tilde{d}\left(j\right)}{2\tilde{m}}\right]\delta \left(c_i,c_j\right).}$ Then, by multiplying the edges by a constant ϕ > 0, we get the graph ${\tilde{G}}_{\varphi }\left(V,{\tilde{E}}_{\varphi }\right)$ of modularity: (5)${\displaystyle \begin{array}{c}\hfill Q_{\varphi }=\frac{1}{2\tilde{m}\varphi }\sum _{ij}\left[\varphi w_{ij}-\frac{{\varphi }^2\tilde{d}\left(i\right)\tilde{d}\left(j\right)}{2\tilde{m}\varphi }\right]\delta \left(c_i,c_j\right)\\ \hfill =\frac{1}{2\tilde{m}\varphi }\varphi \sum _{ij}\left[w_{ij}-\frac{\tilde{d}\left(i\right)\tilde{d}\left(j\right)}{2\tilde{m}}\right]\delta \left(c_i,c_j\right)=Q,\end{array}}$ which means that modularity is also invariant under uniform scaling.

Selection of $\bar{w}$

We distinguish between two types of weighted networks: those with an upper bound on the possible values of their edge weights given by the nature of the data (usually 1, such as in the Forex correlation network –see ‘Data’ below), and those without (such as social networks where edge weights count the number of interactions between nodes). Networks with negative weights have not been studied here, so 0 will be a lower bound in all cases.

However, in the case of networks which are upper and lower bounded, this results in a very large number of edge weights attaining the bounds, which might be undesirable (particularly networks like the Forex network, in which very few edges, if any, have weights 0 or 1) and give new randomized graphs that look nothing like the original data.

The value of $\bar{w}$ that most closely translates the essence of the switching method for unweighted graphs would perhaps be the maximum that still keeps all edges within their set bounds. This method seems particularly suited to sparse graphs with no upper bound, because it eliminates (by reducing its weight to zero) at least an edge per iteration. Other methods without this property could dramatically increase the edge density of the graph, constantly adding edges by transferring weight to them, while rarely removing them.

However, in the case of very dense graphs such as the Forex correlation network (or any other graph similarly constructed from a correlation measure), this method results in a large number of edge weights attaining the bounds (and in the case of the lower bound 0, removing the edge), which can reduce this density dramatically.

As an alternative, to produce a new set of edges with a similar distribution to those of the original network, we can impose the sample variance (i.e., $\frac{1}{n-1}{\sum }_{i,j=1}^n{\left(w_{ij}-m\right)}^2$, where m is the mean) to remain constant after applying the transformation, and find the appropriate value of $\bar{w}$. The variance remains constant if and only if the following equality holds: ${\displaystyle {\left(w_{AC}-m\right)}^2+{\left(w_{BD}-m\right)}^2+{\left(w_{AD}-m\right)}^2+{\left(w_{BC}-m\right)}^2}$ (6)${\displaystyle ={\left(w_{AC}-\bar{w}-m\right)}^2+{\left(w_{BD}-\bar{w}-m\right)}^2+{\left(w_{AD}+\bar{w}-m\right)}^2+{\left(w_{BC}+\bar{w}-m\right)}^2}$ ${\displaystyle ⟺2{\bar{w}}^2+\bar{w}\left(-w_{AC}-w_{BD}+w_{AD}+w_{BC}\right)=0.}$ The solutions to this equation are $\bar{w}=0$ (which is trivial and corresponds to not applying any transformation to the edge weights) and $\bar{w}=\frac{w_{AC}+w_{BD}-w_{AD}-w_{BC}}{2}$.

While this alternative can result in some weights falling outside of the bounds, in the networks we studied it is very rare, so it is enough to discard these few steps to obtain the desired results.

Figure 1 shows how the graph size decreases as the algorithm iterates with the maximum weight method, which also produces a dramatic increase in the variance. The constant variance method on the other hand does not remove any edges and the the size stays constant (as well as the variance, which is constant by definition, so the their corresponding lines coincide at 1).

However, applying the constant variance method on networks that are sparsely connected (such as most reasonably big social networks) results in a big increase in the graph size, to the point of actually becoming complete weighted graphs (see Fig. 2). Meanwhile, the maximum weight method does not significantly alter the size of the graph.

Therefore, we will use the constant variance method only for very densely connected networks, such as correlation networks, which are in fact complete weighted graphs. For sparse networks, the maximum weight method will be the preferred choice.

Note that if all edge weights are either 0 or 1, in both cases this algorithm is equivalent to the original switching algorithm for discrete graphs, as in every step the transferred weight will be one if the switch can be made without creating double edges, or zero otherwise (which corresponds to the case in which the switch cannot be made).

Number of iterations

To determine the number Tm of iterations for the algorithm to sufficiently “shuffle” the network (where m is the size of the graph, and T a parameter we select), we study the variation of information (Meilă, 2007) of the resulting clustering (in this case using the Louvain algorithm, though other clustering algorithms could be used instead) with respect to the initial one. (In “Clustering similarity measures” we discuss variation of information, and other clustering similarity metrics that we use in this work, and in “Clustering algorithms” we detail all the clustering algorithms that we put to test.) Figs. 1 and 2 show a plateau where the variation of information stops increasing after around T = 1 (which corresponds to one iteration per edge of the initial graph). This is consistent with the results for the original algorithm in Milo et al. (2003) for unweighted graphs, and we can also select T = 100 as a value that is by far high enough to obtain a sufficiently mixed graph.

Variation of information

The variation of information between two clusterings, a criterion introduced in Meilă (2007), is defined as follows.

Intuitively, the mutual information measures how much knowing the membership of an element of the set in partition P reduces the uncertainty of its membership in P′. This is consistent with the fact that the mutual information is bounded between zero and the individual partition entropies (17)${\displaystyle 0\le I\left(r;s\right)\le min\left\{\mathscr{H}\left(r\right),\mathscr{H}\left(s\right)\right\},}$ and the right side equality holds if and only if one of the partitions is a refinement of the other.

Consequently, the variation of information will be 0 if and only if the partitions are equal (up to permutations of indices of the parts), and will get bigger the more the partitions differ. It also satisfies the triangle inequality, so it is a metric in the space of partitions of any given set.

Reduced mutual information

The mutual information (Cover & Thomas, 1991), often in its normalized form is one of the most widely used measures to compare graph partitions in cluster analysis. More recently (Newman, Cantwell & Young, 2020) proposed the Reduced Mutual Information (RMI), an improved version which corrects the high mutual information values given to quite dissimilar partitions in some cases. For instance, if one of the partition is the trivial one splitting the network into n clusters of one element each, the standard mutual information will always take the maximal value (1, in the case of the normalized mutual information), even if the other is completely different. More generally, any partitions will always have maximal mutual information with all of their filtrations. This is crucial when comparing clustering algorithms, as some algorithms will output trivial partitions into single-element clusters when they fail to find a clustering structure. Therefore, it would not be possible to reliably measure the stability of these clustering methods with the standard mutual information.

Given r and s two labelings of a set of n elements, the Reduced Mutual Information is defined as: (18)${\displaystyle \text{RMI}\left(r;s\right)=I\left(r;s\right)-\frac{1}{n}log\Omega \left(a,b\right)}$ where Ω(a, b) is an integer equal to the number of R × S non-negative integer matrices with row sums a = {a_r} and column sums b = {b_s}. Details on how to compute or approximate Ω(a, b) are given in the “Methods for counting contingency tables”.

The Reduced Mutual Information can also be defined in a normalized form (NRMI), in the same way the standard mutual information is, by dividing it by the average of the values of the reduced mutual information of labelings a and b with themselves: (19)${\displaystyle \begin{array}{c}\hfill \text{NRMI}\left(r;s\right)=\frac{\text{RMI}\left(r;s\right)}{\frac{1}{2}\left[\text{RMI}\left(r;r\right)+\text{RMI}\left(s;s\right)\right]}\\ \hfill =\frac{I\left(r;s\right)-\frac{1}{n}log\Omega \left(a,b\right)}{\frac{1}{2}\left[H\left(r\right)+H\left(s\right)-\frac{1}{n}\left(log\Omega \left(a,a\right)+log\Omega \left(b,b\right)\right)\right]}\end{array}}$

We will use this normalized form to be able to compare more easily the results of networks with different number of nodes, as well as to compare them to other similarity measures.

Rand index

The Rand Index (RI) and the different measures derived from it (Hubert & Arabie, 1985) are based on the idea of counting pairs of elements that are classified similarly and dissimilarly across the two partitions P and P′. There are four types of pairs of elements:

• type I: elements are in the same class both in P and P′

• type II: elements are in different classes both in P and P′

• type III: elements are in different classes in P and in the same class in P′.

• type IV: elements are in the same class in P and different classes in P′.

Then, similar partitions would have many pairs of elements of types I and II (agreements) and few of type III and IV (disagreements). The Rand index is defined as the ratio of agreements over the total number of pairs of elements.

Using the terms of the contingency table (Table 2), the Rand index is given by (20)${\displaystyle \text{RI}\left(r,s\right)=\left(\genfrac{}{}{0.0pt}{}{n}{2}\right)+2\sum _{r}s\left(\genfrac{}{}{0.0pt}{}{n_{rs}}{2}\right)-\left[\sum _{r=1}^R\left(\genfrac{}{}{0.0pt}{}{a_r}{2}\right)+\sum _{s=1}^S\left(\genfrac{}{}{0.0pt}{}{b_s}{2}\right)\right]}$

An adjusted form of the Rand index (Hubert & Arabie, 1985) introduces a correction to account for all the pairings that match on both partitions because of random chance. The Adjusted Rand Index (ARI) is defined as: (21)${\displaystyle \text{ARI}\left(r;s\right)=\frac{\text{Index}-\text{Expected Index}}{\text{Maximum Index}-\text{Expected Index}},}$

which in terms of the contingency table (Table 2) can be expressed as: (22)${\displaystyle \text{ARI}\left(r;s\right)=\frac{\sum _{rs}\left(\genfrac{}{}{0.0pt}{}{c_{rs}}{2}\right)-\left[\sum _r\left(\genfrac{}{}{0.0pt}{}{a_r}{2}\right)\sum _s\left(\genfrac{}{}{0.0pt}{}{b_s}{2}\right)\right]∕\left(\genfrac{}{}{0.0pt}{}{n}{2}\right)}{\frac{1}{2}\left[\sum _r\left(\genfrac{}{}{0.0pt}{}{a_r}{2}\right)+\sum _s\left(\genfrac{}{}{0.0pt}{}{b_s}{2}\right)\right]-\left[\sum _r\left(\genfrac{}{}{0.0pt}{}{a_r}{2}\right)\sum _s\left(\genfrac{}{}{0.0pt}{}{b_s}{2}\right)\right]∕\left(\genfrac{}{}{0.0pt}{}{n}{2}\right)}}$

Weighted clustering coefficient

Let Γ be the number of connected triplets in the graph and γ the number of closed triplets (i.e., 3 times the number of triangles). As before Γ(t) and γ(t) are their respective values when only edges with weight greater or equal than t are considered. Then, the clustering coefficient or transitivity is defined as: (23)${\displaystyle \tilde{C}=\frac{1}{\bar{w}}{\int }_{t\ge 0}\frac{\gamma \left(t\right)}{\Gamma \left(t\right)}dt,}$ This is an integral of a step function that takes a finite number of values (bounded by the number of different edge weights) which we will compute as follows:

1. Construct a hash table of all edges with their corresponding weights to be able to search if there is an edge between any two edges (and obtain it’s weight) in constant time. Complexity: O(m)

2. Construct a hash table for each vertex containing all its neighbors. Can be done by iterating once over the edges and updating the corresponding tables at each step. This will be used to iterate over the connected triplets incident to each vertex. Complexity: O(m)

3. Construct a sorted list containing the edge weights at which either a connected triplet or a triangle appears (i.e., the maximum edge weight of that triangle or triplet), and an associated variable for each indicating whether it corresponds to a triangle or a triplet. For this, we iterate over the connected triplets using the hash tables from step 2, and for each, we check if it also forms a triangle by checking the hash table from step 1 (which allows each iteration to be done in constant amortized time). This step has complexity O(ΓlogΓ), as the list has Γ + γ elements, and (Γ + γ) ∈ O(Γ).

4. We iterate the list from step 3 and compute the cumulative sums of connected triplets and closed connected triplets (which correspond to γ(t) and Γ(t) for increasing values of t in the list). This gives us all values of $\frac{\gamma \left(t\right)}{\Gamma \left(t\right)}$, from which we compute the integral (Eq. (23)). This involves O(Γ) steps of constant complexity.

Therefore, the overall complexity of the algorithm is O(m + ΓlogΓ). Because Γ is bounded by m², we can also express the complexity only in terms of m (which will then be O(m²)), but that bound is not tight in most graphs.

Other scoring functions

Computing $\tilde{m}$, ${\tilde{m}}_S$, ${\tilde{c}}_S$ (for all values of S), as well as all vertex degrees and out degrees has complexity O(m), as it can be done sequentially by reading the edge list and updating the appropriate values as necessary. This means that all scoring functions except for the clustering coefficient, which are derived from these values, can be computed very efficiently.