Multimodal multi-objective optimization algorithm based on hierarchical environment selection strategy

Definitions for MMOPs

MMOPs are ubiquitous in real life, and these types of problems are also a special type of MOPs. When defining MMOPs, the following situations must occur:

1. Firstly, there are some global PSs;

2. secondly, scenario is that there is at least one local PS in the decision space.

However, the connection between these PSs and the objective values in the objective space is not a one-to-one mapping, but rather multiple PSs corresponding to a single objective value. Compared with general MOPs, algorithms need to spend more resources and time searching for more different optimal equivalent solutions when solving MMOPs.

Evaluation indexes for MMOPs

The traditional evaluation index of the MOPs only focuses on the performance of the population; the evaluation index of multimodal multi-objective optimization (MMO) optimization also needs to focus on its decision space. Common metrics for MMO algorithms are Pareto solutions proximity (PSP) (Liang et al., 2018), Hypervolume (HV) (Zitzler & Thiele, 1999), and Inverted Generational Distance (IGDX) (Zhou, Zhang & Jin, 2009), IGDX index can measure the performance of the PS; the PSP index can reflect both the convergence and diversity of the solution set obtained by the algorithm and can also measure the coverage of the PS on the real PS, PSP is more comprehensive and reasonable evaluation. The convergence and diversity of HV and PF are directly proportional. In the experimental stage, this article uses the PSP and HV metrics to evaluate the effectiveness of MMO algorithm, the equations for HV and PSP are as follows:

(2)${\displaystyle PSP=\frac{CR}{IGDX}}$ (3)${\displaystyle IGDX\left(PS,P{S}^{\ast }\right)=\frac{\sum _{a\in PS}d\left(a,P{S}^{\ast }\right)}{\left|PS\right|}}$ (4)${\displaystyle HV\left(PS,P\right)=volume\bigcup _{x\in PF}v\left(x,P\right)}$

In the above equation, CR is the coverage rate of the PSs, where P is the reference point, which represents the hyperbolic volume surrounded by the solutions and the reference point. The PSP measures the diversity and convergence and the larger PSP, the more equivalent solutions the algorithm obtains. The effect shown by the algorithm is proportional to the HV.

Differential evolution

Differential evolution algorithm (DE) is an efficient and straightforward evolution algorithm proposed (Das & Suganthan, 2011; Qin, Huang & Suganthan, 2009). Among many heuristic algorithms, the advantages of DE are more obvious. Firstly, compared to others, DE has a shorter running time; secondly, the algorithm framework of DE is simpler, the algorithm process is easy to understand, and researchers can quickly and deeply understand it; finally, DE converges faster compared to other evolutionary algorithms. Therefore, DE is more effective in many standard test functions and real-world problems. The process is consistent with many similar algorithms, and DE has four processes: firstly, randomly generating a population; using mutation equations to induce population variation; crossing populations and selecting suitable individuals; finally, environmental selection is the process of selecting outstanding individuals to form offspring populations.

Specialized crowding distances

Crowding distances (CD) is to measure the degree of crowding of individuals in a population, for calculating the crowding distances of individuals, and the algorithms use crowding distance metrics to enhance the population’s diversity (Ghorbanpour, Jin & Han, 2022). In MMOPs, the algorithm uses special crowding distances (SCD) to measure the degree of crowding of population individuals (Yue et al., 2021). The non-dominated sorting method using special crowding distances can improve the performance and efficiency of MO algorithms. The MMODE_ES uses the special crowding distance in the non-dominated sorting of individuals to solve the problem of selecting individuals from a dense set of solutions.

Differential variation strategies

Many different differential mutation strategies are designed according to other optimization problems give the searchability of the DE a boost. In this article, the algorithm adopts the differential variation strategies of DE/best/2 and DE/rand/2 to iterate.

To improve the searchability of the differential evolution algorithm, many different differential mutation strategies are designed according to other optimization problems, and the common ones are DE/rand/1, DE/best/1, DE/best/2, and DE/rand/2. In this article, the algorithm adopts the differential variation strategies of DE/best/2 and DE/rand/2 to iterate. The equation for DE/best/2 strategy is as follows: (5)${\displaystyle V_i=X_{bset}+F\cdot \left[\left(X_{r_1}-X_{r_2}\right)+\left(X_{r_3}-X_{r_4}\right)\right]}$

In this equation, X_best is the optimal individual in the current population, X_r1, X_r2, X_r3 and X_r4 are random individual in the population.

The equation for DE/rand/2 strategy is as follows: (6)${\displaystyle V_i=X_{r_1}+F\cdot \left[\left(X_{r_2}-X_{r_3}\right)+\left(X_{r_4}-X_{r_5}\right)\right]}$

In this equation, X_r1, X_r2, X _r3, X_r4, and X_r5 are random individuals.

The DE/rand/2 strategy enhances the diversity of the population because it randomly selects individuals for evolution so that the algorithm can explore more extensively in the search space. DE/best/2 focuses more on global search because it uses the best-performing individuals as a reference, which drives the algorithm to search for more promising regions, which facilitates the improvement of the convergence of the population.

The algorithm uses the DE/rand/2 strategy at the beginning of the evolution to ensure the diversity of the population obtaining more evenly distributed PSs. In the subsequent iterations, after the population is non-dominated sorted, the variety and convergence of the population need to be investigated at this time; if the current non-dominated rank is only 1 level, and the algorithm tends to converge at this time, it is necessary to improve the diversity of the population. In the next generation, the DE/rand/2 strategy can help population to evolve. Suppose there are multiple levels of population non-dominance rank. In that case, it is necessary to explore PSs closer to the actual PSs to improve the algorithm’s convergence, and DE/best/2 strategy can help the population’s evolution.

Neighborhood variation strategy

The primary process of the DE is the differential strategy; the algorithm selects appropriate individuals for mutation, which can ensure the population’s diversity and distribution and improve the algorithm’s searchability. The traditional differential mutation strategy randomly selects five individuals in the current population for cross-mutation, and the random selection of individuals cannot guarantee the population’s diversity and the algorithm’s convergence. To ensure the diversity and the variety, this article proposes a neighborhood variation strategy.

Neighborhood variation strategy adopts the idea of small populations. Select the congestion distance that is large in the first 20 individuals to form a small population according to the crowding distance of the current individual, and in the small population to choose the most considerable congestion distance of an individual as the base vector, and then finally randomly select individuals, and ultimately enter the mutation stage, which can improve the diversity. Based on the crowding distance for objective space in the current individual, form these population of the top 20 individuals with larger crowded distances. Using the individual with the largest crowding distance in these population as the basis vector, and then randomly selected individuals are used, and ultimately enter the mutation stage, which can improve the diversity in the objective space.

CD is calculated by weighted Euclidean average distance to individuals in the neighborhood in decision space. The equation for its calculation: (7)${\displaystyle C{D}_{i,x}=\sum _{j=1}^{k_1}\left(k_1-j+1\right)d_{i,j}}$

d_i,j indicated the Euclidean distance, which is the distance between i^th and j^th. The special congestion distance (SCD) equation is as follows: (8)${\displaystyle SCD=\left\{\begin{array}{c}max\left(C{D}_{i,x},\frac{C{D}_{i,F}}{Rank}\right),C{D}_{i,x}>C{D}_{agv,x}orC{D}_{i,f}>C{D}_{agv,f}\hfill \\ min\left(C{D}_{i,x},C{D}_{i,f}\right),otherwise\hfill \end{array}\right.}$

In Eq. (8), CD_avg,x denotes the average crowding distance, CD_i,f denotes the crowding distance of an individual, CD_avg,f denotes the average crowding distance, and Rank denotes the current individual’s nondominant rank.

First of all, when the population starts to evolve, the mutation strategy randomly selected one individual as the base vector, and randomly select the individual with a higher probability in the whole population to improve the exploration ability and prevent premature algorithm stagnation. To improve search efficiency, researchers not only need to get the uniformly distributed Pareto solution set but also need to get the uniformly distributed Pareto frontier surface; the mutation strategy needs to consider the distributivity of the population in both spaces at the same time, simultaneously, the population should be two mutation strategies guiding the evolution.

The reason for adopting the above strategy is the adaptive balance of exploration and exploitation capacity, where both decision-making and objective spatial diversity are improved. In the later stages of evolution, MMODE_ES will select the neighbors to improve search capabilities. Selecting adjacent individuals for evolution in two different spaces can enhance diversity in various spaces. MMODE_ES framework is as follows:

Layered environment selection strategy

To further improve performance on the objective space so that MMODE_ES obtains a more uniformly distributed PSs, the article proposes a hierarchical environment selection strategy, which selects the nondominated solutions of each layer in different proportions in different evolutionary stages. In the pre-evolutionary step, the algorithm selects suitable individuals for the next phase of evolution in a certain proportion of the population after non-dominated sorting with the special crowding distance, and individuals with high non-dominated rank and large special crowding distances have a higher probability of being selected. In the early stage of individual evolution, selecting individuals proportionally, instead of selecting all individuals with high dominance rank, can prevent the algorithm from converging prematurely and give the individuals with lower non-dominance rank in the early stage a chance to explore to obtain more Pareto solution sets, and at the same time, let the algorithm no longer spend more time on inefficient searching for the individuals with lower non-dominance rank, and in the early phase, the selection of vast majority of individuals are those in the first three nondominant ranks. The individuals in the first rank are greater than the number of individuals in the second rank and more significant than those in the third rank. In the late phase, the algorithm gradually converges, and the Pareto solution set and Pareto frontier surface progressively perfect. At this time, the algorithm enters the second search stage, where the environment selects only the individuals with higher non-dominated ranks.

The proportion Ra, Rb, Rc of hierarchical selection is calculated as follows:

(9)${\displaystyle Ra=\left\{\begin{array}{c}0.6+0.8\cdot \left(gc-1\right),1<gc<G\hfill \\ 1,G<gc<Maxgen\hfill \end{array}\right.}$ (10)${\displaystyle Rb=0.8\cdot Ra,Ra<1}$ (11)${\displaystyle Rc=0.6\cdot Ra,Ra<1}$

In the evolution of the population, the top non-dominated ranked individuals are not necessarily optimal, and the proportional selection of the top-ranked individuals is to remove the pre-evolutionary high non-dominated ranked but undesirable solutions and to give the low non-dominated ranked but potential individuals a chance to be explored. From Eqs. (9) to (11), we can see that Ra > Rb > Rc, so the hierarchical selection of individuals is mainly selected from the first three levels of individuals; the selection of individuals with a high non-dominated rank tends to allow the algorithm to search toward the potential region, but only selecting the first level of individuals as a child will make the algorithm fall into the local optimum prematurely. The selection of individuals at each level will allow the algorithm to search too much for the individuals who do not have the potential. The algorithm will require more work to converge. Therefore, the algorithm mainly selects the top three individuals in the non-dominated rank and selects the low-rank individuals with small probability, which can help the algorithm to search towards the potential area and also give the lower level of individuals a chance to be searched, which not only improves the diversity of the population but also improves the closeness of the population to the actual population. The framework of the algorithm is as follows:

Framework of MMODE_ES

The algorithm in this article is called MMODE_ES, it has the following steps:

Experiment

This section provides experimental validation of the proposed algorithm. It describes the relevant parameter settings and the test set used for the experiment and finally analyzes and summarizes on the experimental results.

Experimental settings

All algorithm parameters are set according to the original author’s suggestions, so that the algorithm can be proved to be efficient, which can allow individual algorithms to be compared at one level. The algorithm stops running when the fitness reaches its maximum, the setting of the relevant parameters is related to dimensions of the decision vector (Nvar) in test problems, the maximum fitness is set to 10000 and the population size to 100*Nvar. The maximum number of generations is set to 50. The CR is 0.5 and the F is 0.5. The relevant settings are described in Table 1. The running environment of this experiment is Windows 11 operating system, AMD Ryzen 7 4800H with Radeon Graphics 2.90 GHz and 16.0 GB memory. The software uses Matlab2021.

Comparison of algorithms

To illustrate MMODE_ES’s effectiveness in this article, five excellent MMO algorithms are selected for comparison test, which are MO_Ring_PSO_SCD ((Yue, Qu & Liang, 2018), DN-NSGAII (Liang, Yue & Qu, 2016), Omni-optimizer (Deb & Tiwari, 2005), TriMOEA-TA&R (Liu, Yen & Gong, 2018), MMODE_ICD (Yue et al., 2021) these algorithms. MO_Ring_PSO_SCD and TriMOEA-TA&R are often MMO algorithms that are compared and used as improvements by many researchers. Omni-optimizer and DN-NSGAII are classical MMO algorithms. The MO_Ring_PSO_SCD algorithm is the best algorithm in the CEC2019 multimodal optimization problem competition. MMODE_ES in this article is compared with these MMO algorithms.

Test functions

This article uses 13 test functions on the CEC2019 test problems were selected to test the search ability of the proposed method for demonstrating whether the MMODE_ES in this article is feasible, including functions: MMF1 to MMF10 (Yue et al., 2019); SYM-PART simple and SYM-PART simple rotated (Rudolph, Naujoks & Preuss, 2007); omni-test (Deb & Tiwari, 2008). These benchmark test problems contain Pareto fronts of different shapes and Pareto solution sets. Some of the real PSs and PF in these testing questions have complex situations, with overlapping, numerous, multi-dimensional, and complex shapes that pose great challenges to MMODE_ES.

Experimental results and analysis

The PSP metric can evaluate search ability, in other words, it can measure the degree of closeness between the obtained PSs and the actual PSs. HV can measure the convergence and diversity in the objective space. The article uses rPSP and rHV to facilitate the analysis. rPSP is the inverse of PSP, and rHV is the inverse of HV. Therefore, the smaller these two metrics are, the better the performance. MMODE_ES and several other comparative algorithms were tested 30 times on 13 test functions. This article summarizes the results of 30 tests and calculates the mean and standard deviation. In the experiment, the Wilson test was used to detect whether there was a significant difference in the results. The symbol “+” in the table indicates that MMODE-ES performs better than the comparison algorithm. Conversely, the symbol “-” indicates that MMODE_ES performs worse than the comparison algorithm, while the symbol “ =” indicates that there is no difference in performance between MMOD_ES and the comparison algorithm. Meanwhile, the Mean in the table represents the average of 30 results, which can indirectly reflect the overall level of each algorithm, while Std represents the standard deviation of these 30 results, which indirectly represents the stability of the algorithm. Therefore, the smaller the Mean and Std of the algorithm’s results, the better the algorithm. Therefore, to demonstrate the excellence of MMODE_ES, some data has been marked in bold, indicating that the results of MMODE_ES are better.

As the results in Tables 2 and 3 show that the algorithm MMODE_ES in this article performs well on most of the test functions. By comparing the rPSP and rHV metrics in the table, the algorithm in this article, and comparing it with many excellent algorithms, the operation of the algorithm in this article is effective. In the rPSP index: except for the DN-NSGAII algorithm in the SYM-PART simple function, MMODE_ES is at a disadvantage in the rest of the test function obtained better results, which shows that MMODE_ES searched for the PSs has a better distribution in the MMOPs is more competitive. For the rHV: MMODE_ES gives excellent results on 13 test problems, and experiments show that this article’s algorithm also obtains solutions with better convergence and distribution. Combining the results in Tables 2 and 3, MMODE_ES shows overwhelmingly satisfactory results than a lot comparative MMO algorithms, so MMODE_ES’s operation in this article is effective.

The optimization results of this method are compared with several comparative algorithms on the MMF3 to show the effectiveness of this algorithm more intuitively, and the results are shown in the form of scatter plots to show the results. In MMF3, the distribution of the decision space solutions of MMODE_ES is good, with very few missing solution sets. In contrast, several other algorithms have significantly more missing solution sets than the present algorithm. The evolved distribution of the solution sets is, to a great extent, able to reach the overlap with the real solution set. In the final Pareto front, the Pareto front obtained in this article is complete and close to the real show to a large extent. The Pareto front obtained by other algorithms can also be close to the real front to a large extent. Therefore, MMODE_ES has the best performance.

The PSs for all algorithms on the MMF3 in the decision function are as Fig. 1, Fig. 1 shows the real PSs of this test problem. The results of all algorithms on the objective space over the MMF3 function are as Fig. 2, Fig. 2 shows the real results of this test problem. From the results shown on the listed test problems, MMODE_ES has better results. In this article, MMODE-ES searched for more and distributed PSs, which approaches the actual situation; the distribution of the PSs is uniform, and missing PSs are less. In the objective space, PF of this algorithm near to the real front and better distributed than the comparison algorithm. Combined with the tabular rPSP and rHV value this article’s algorithm effectively solves MMOPs.

Overall, MMODE_ES is effective in solving MMOPs, which is attributed to the fact that the algorithm’s performance in decision space and objective space are considered simultaneously in the process of searching for PSs. Firstly, the designed variation strategy endeavors to combine the study of the distributivity and diversity of PSs in the process of evolution, which improves the performance of MMODE_ES in the decision space and improves the diversity of PSs; secondly, the performance of the decision space is fully considered, and such consideration improves the diversity and distributivity of PF; lastly, the hierarchical environmental selection scheme allows the excellent individuals to be retained, and such design allows the distributivity and diversity of PSs to be enhanced. After comparing with these excellent algorithms, it can be seen that although some of these algorithms make efforts to search for PSs in the decision space, these search strategies still need to be further improved; meanwhile, the traditional PS selection scheme selects individuals with high nondominant rank, which does not retain the dominant individuals well, and then leads to a part of PSs being excluded from the real PSs, and the final searched PF and PSs have a missing; at the same time part of the algorithm can consider the performance of the algorithm in the objective space, but the consideration is not comprehensive enough. In contrast, MMODE_ES is an excellent algorithm for solving MMOPs