New hybrid segmentation algorithm: UNet-GOA

U-Net architecture

As you can see in Fig. 1, the U-Net network consists of two parts, contraction and expansion. The expion part is very similar to the general operation of convolutional neural networks (Fukushima & Miyake, 1982), which also includes a convolution layer, and the purpose of this part is to repeatedly apply a 3 * 3 convolutional filter on the input image. After each convolution layer there is a ReLU layer (Agarap, 2018) or linear activator. The ReLU layer applies an activation function to each neuron, such as max(0, x) (which sets the threshold to zero, in other words, it considers negative values to be zero). The main advantage of using ReLU is that it has a fixed derivative for all inputs larger than zero. This fixed derivative accelerates network learning. After this layer, there is a 2 * 2 maxpooling layer (Ciresan et al., 2011) with two strides. The function of this layer is to reduce the spatial size of the image due to the reduction of the number of parameters and calculations within the network, which causes overfitting control. Together, these three layers form a sampling layer or in other words, downsampling, the purpose of which is better in terms of image content. Conversely, in the expansion section, in each layer, there is an incremental sampler or in other words, upsampling, along with a 2 * 2 convolutional layer, which is responsible for reducing specific channels. This section receives the data to be added to the image from its similar class in the contraction section. Then there is a 3 * 3 convolutional layer with a ReLU layer. The purpose of this section is to assist in the process of accurately locating objects. In the last layer, there is a 1 * 1 convolution layer that maps property vectors to a specific class (Ronneberger, Fischer & Brox, 2015).

Below you can see the steps of the U-Net architecture:

1. Define U-Net architecture with encoder and decoder parts

2. Input image to the encoder part and downsample it

3. Pass downsampled feature map to the decoder part

4. Upsample feature map and concatenate it with feature map from corresponding encoder layer

5. Repeat step 4 until original image size is reached

6. Apply convolutional layers to concatenated feature maps

7. Output segmented image.

Initializing U-Net

In the U-Net architecture, the network input consists of images and their fragmented parts into separate files. Also, the network optimization algorithm is a stochastic gradient descent (SGD) (Bottou & Bousquet, 2007), which is in fact an iterative method for optimizing a derivative function, which is also a random approximation of the classical gradient descent method. The amount of motion or momentum applied to the grid should be a maximum of about 0.99. The goal is for the network to make the most of the images it has already seen in the updates. The energy function is calculated by a Softmax function (Goodfellow, Bengio & Courville, 2016) at the pixel level, which is applied to the output map in combination with the cross entropy cost function (Cybenko, O’Leary & Rissanen, 1998). Equation (1) is used to calculate the energy function (Ronneberger, Fischer & Brox, 2015): (1)${\displaystyle E=\sum w\left(x\right)\mathit{log}\left(P_{k\left(x\right)}\left(X\right)\right)}$ where P_k is the pixel-wise SoftMax function applied over the final feature map, defined as Eq. (2) (Ronneberger, Fischer & Brox, 2015): (2)${\displaystyle P_k=exp\left(a_k\left(X\right)\right)/\sum _{k^{{}^{\prime }}=1}^{k}exp\left(a_k{\left(X\right)}^{{}^{\prime }}\right)}$

One of the most important issues in deep neural networks with many layers is the appropriate initialization to the network. If the initial value to the network is not correct, parts of the network will be over-activated and other parts will not be involved in the calculations. Theoretically, the appropriate initial value for each neuron in the network is the value for which each property in the network has approximately a unit variance. To calculate the initial value of each neuron, we use the Eq. (3) in which N is the number of neuron inputs (He et al., 2015): (3)${\displaystyle \text{Initial Weight}=\sqrt{2/N}}$

Evaluation U-Net

Collecting, analyzing as well as interpreting information from the desired data is called evaluation. Evaluation actually examines the effectiveness and efficiency of a system according to the data collected (Thorpe, 1988). We use the confusion matrix (error matrix) to evaluate a model. The confusion matrix is a summary table used to evaluate the performance of a model. In this method, the number of correct and incorrect predictions is summed with counting values and analyzed based on each class (Matthews, 1975).

In this article, we evaluate the model based on Accuracy (ACC) (Csurka et al., 2013; Hossin & Sulaiman, 2015), Boundary F1 (BF) (Csurka et al., 2013) and Intersection over Union (IoU) (Csurka et al., 2013) criteria. The description and how to obtain each of the criteria are specified in Table 1.

Why U-Net?

In this article, we will explore the utilization of the U-Net architecture, a popular choice among recent segmentation architectures, for image segmentation purposes. The U-Net architecture is a popular choice for image segmentation tasks, particularly for biomedical image analysis, because as the authors of this article (Ronneberger, Fischer & Brox, 2015) say it has demonstrated superior performance in comparison to other architectures for this specific task.

The reason for selecting the U-Net architecture in this article is multi-faceted. Firstly, the U-Net architecture possesses a notable strength in handling complex image segmentation tasks even with a limited number of training samples. This advantage stems from its unique design, incorporating both a contracting path to capture the overall context of the image and an expanding path to facilitate precise localization of the object of interest. Secondly, the U-Net architecture employs skip connections that establish connections between the contracting and expanding paths. This strategic utilization of skip connections enables the model to retain fine-grained details that might otherwise be lost during the downsampling process. As a result, the U-Net model excels particularly in segmenting small objects or objects characterized by intricate shapes, enhancing its effectiveness and versatility in various segmentation scenarios (Siddique et al., 2021).

Overview of the GOA

Grasshoppers are among the insects that are known as a dangerous pest for agricultural lands, which are very effective in not producing a quality product and also damage agricultural products (Ewees et al., 2020). The life cycle of grasshoppers consists of two stages. One is called the nymph and the other is in adulthood. In the nymph stage, immature grasshoppers move in small and slow steps, which is very similar to the extraction stage in optimization algorithms, and the adulthood stage is known for its sudden movements and long steps, which are also very similar to the exploration stage is in optimization algorithms. Due to the life structure of grasshoppers as well as the mass movement of grasshoppers, the grasshopper optimization algorithm was proposed in 2017 (Saremi, Mirjalili & Lewis, 2017). This algorithm is one of the swarming algorithms (Beni & Wang, 1993) and therefore mimics the natural search behaviors of grasshoppers and their swarming. The grasshopper swarm behavior in mathematics is modeled as Eq. (5) (Saremi, Mirjalili & Lewis, 2017):

(5)${\displaystyle X_i\left(t\right)=S_i\left(t\right)+G_i\left(t\right)+A_i\left(t\right)}$ ${\displaystyle \left\{\begin{array}{c}i=1,2,\dots ,nPop\hfill \\ t=1,2,\dots ,tMax\hfill \end{array}\right.}$

Where $X_i\left(t\right)$ indicates the position of the grasshopper i in the iteration of the t, $S_i\left(t\right)$ is the social interaction of the grasshopper i in the iteration of the t, $G_i\left(t\right)$ is the gravity force of the grasshopper i in the iteration of the t, and $A_i\left(t\right)$ is the wind advection of the grasshopper i in the iteration of the t. To create a random grasshopper behavior, Eq. (5) is reconstructed as Eq. (6), where r₁, r₂ and r₃ are random numbers between 0 and 1: (6)${\displaystyle X_i\left(t\right)=r_1{S}_i\left(t\right)+r_2{G}_i\left(t\right)+r_3{A}_i\left(t\right)}$

In this section, we examine the mathematical equations of social interaction $S_i\left(t\right)$, gravity force $G_i\left(t\right)$, and wind advection $A_i\left(t\right)$, respectively. We first start with social interaction, which is defined as Eq. (7): (7)${\displaystyle S_i=\sum _{\begin{array}{c}\hfill j=1\hfill \\ \hfill \left(j\ne i\right)\hfill \end{array}}^{nPop}s\left(d_{ij}\right)\widehat{d_{ij}}}$ Where nPop indicates the number of grasshoppers, d_ij is distance between grasshopper i and grasshopper j which is calculated by this formula: $d_{ij}=\left|x\left(j\right)-x\left(i\right)\right|$, $\widehat{d_{ij}}$ is unit vector from grasshopper i to the grasshopper j which is calculated by this formula: $\widehat{d_{ij}}=\frac{x\left(j\right)-x\left(i\right)}{d_{ij}}$, and then s is a function that indicates the power of social forces which is calculated using the Eq. (8) formula: (8)${\displaystyle s\left(d\right)=f{e}^{-\frac{d}{l}}-e^{-d}}$ where f indicates the intensity of attraction and l is the attractive length scale. Social interaction between grasshoppers is known as attraction and repulsion, which is considered between 0 and 15. It is observed that attraction increases between 2.079 and 4 and then gradually decreases. Repulsion also occurs between 0 and 2.079. The 2.079 area is also called the no-power zone or the comfort zone, where there is no repulsion or attraction.

The gravitational force G_i and the wind advection A_i are also obtained from Eqs. (9) and (10): (9)${\displaystyle G_i=-g\widehat{e_g}}$ (10)${\displaystyle A_i=u\widehat{e_w}}$ where g is the gravitational constant and $\widehat{e_g}$ shows a unity vector towards the center of earth; also where u is a constant drift and $\widehat{e_w}$ is a unity vector in the direction of wind. After replacing the values of S_i, G_i and A_i in Eq. (5), the general Eq. (11) is obtained: (11)${\displaystyle X_i\left(t\right)=\sum _{\begin{array}{c}\hfill j=1\hfill \\ \hfill \left(j\ne i\right)\hfill \end{array}}^{nPop}s\left(\left|x\left(j\right)-x\left(i\right)\right|\right)\frac{x\left(j\right)-x\left(i\right)}{d_{ij}}-g\widehat{e_g}+u\widehat{e_w}}$

After implementing Eq. (11), it is observed that the grasshoppers reach the comfort zone quickly and cannot converge to one point. As a result, the grasshoppers remain motionless and stationary after a while. For these reasons, Eq. (11) cannot be used to solve optimization problems. Therefore, an improved version of this equation can be used, which is presented in Eq. (12): (12)${\displaystyle X_i^d\left(t+1\right)=c\left(\sum _{\begin{array}{c}\hfill j=1\hfill \\ \hfill \left(j\ne i\right)\hfill \end{array}}^{nPop}c\frac{u{b}_d-l{b}_d}{2}s\left(\left|x_j^d-x_i^d\right|\right)\frac{x\left(j\right)-x\left(i\right)}{d_{ij}}\right)+{\hat{T}}_d}$ Where ub_d is the upper bound in the d dimension and lb_d is the lower bound in the d dimension. ${\hat{T}}_d$ isthe value of the d dimension in the target (the best solution found so far). c is a decreasing coefficient to shrink the comfort zone, repulsion zone, and attraction zone. The first c is very similar to the inertia weight (ω) in the particle swarm optimization algorithm. It reduces the movements of grasshoppers around the target. In other words, this parameter balances exploration and exploitation of the entire swarm around the target. The second c is used to reduce the repulsion zone, the gravity zone and the comfort zone between the grasshoppers in proportion to the number of iterations. Eq. (13) is used to calculate c, which is considered as a single parameter:

(13)${\displaystyle c=c_{max}-t\frac{c_{max}-c_{min}}{t_{max}}}$ ${\displaystyle \left\{\begin{array}{c}{c}_{max}=MaximumValueC\left(\approx 1\right)\hfill \\ c_{min}=MinimumValueC\left(\approx 0\right)\hfill \end{array}\right.,\left\{\begin{array}{c}t=\text{Current Iteration}\hfill \\ t_{max}=\text{Maximum Number of Iteration}\hfill \end{array}\right.}$

According to the mentioned equations, the position of a grasshopper is updated based on three features: the current position of the grasshopper, the best global position of the grasshopper and the position of the grasshopper in relation to other grasshoppers in the swarm. These help the grasshopper optimization algorithm not get stuck in the local optimization and reach the global optimization (Meraihi et al., 2021). The flowchart of the grasshopper optimization algorithm is shown in Fig. 2 (Saremi, Mirjalili & Lewis, 2017).

Below you can see the steps of the grasshopper optimization algorithm:

1. Initialize population of grasshoppers

2. Evaluate fitness of each grasshopper

3. While termination criteria not met do:

   1. Calculate distance between each pair of grasshoppers

   2. Calculate attractive and repulsive forces between each pair of grasshoppers

   3. Update velocity and position of each grasshopper based on attractive and repulsive forces

   4. Apply random perturbation to some grasshoppers

   5. Evaluate fitness of each grasshopper

4. Return best solution found during optimization.

Why GOA?

In this article, we will utilize the grasshopper optimization algorithm (GOA) (Saremi, Mirjalili & Lewis, 2017), a recently developed optimization method that shows promise in solving a wide range of problems. GOA draws inspiration from the collective behavior of grasshoppers and employs a population-based approach to search for the best solutions. It offers several advantages, including efficient exploration and exploitation of the search space, resulting in faster convergence and improved solutions compared to other metaheuristic algorithms. GOA is adaptable to various types of optimization problems and excels at maintaining a balance between exploration and exploitation to find good solutions while avoiding getting stuck in local optima. With its population-based strategy, it conducts a comprehensive global search, increasing the chances of discovering the global optimum. GOA is versatile and can handle optimization problems with continuous, discrete, and combinatorial variables. Additionally, it is scalable, making it suitable for problems with numerous constraints and variables. Notably, GOA demonstrates robustness in scenarios where the objective function is noisy or difficult to evaluate accurately (Meraihi et al., 2021; Saremi, Mirjalili & Lewis, 2017).