The prediction of collective Economic development based on the PSO-LSTM model in smart agriculture

LSTM-based economy scale prediction

The development prediction of economic production can be seen as a multi-objective regression problem based on the current year’s conditions. It can also predict future data through the relevant historical data of the economy. Such issues are prediction problems based on historical data. Therefore, a strong, robust model that can achieve multiple objectives is needed. According to the time series of historical data of multidimensional data in multi-objective regression, this article uses the RNN, which is good at processing time series, to build the model.

RNN is a classical feedback neural network, often used in regression and multivariate time series prediction analysis. Different from the back propagation (BP) neural network, RNN adds the association between information among hidden nodes to consider time information. In a typical RNN, the output of the last layer in the hidden layer is one of the inputs of the next hidden layer (Hou, 2022; Nie, 2021). Each output value of RNN is closely related to its previous state. However, RNN has only one memory unit, which leads to gradient disappearance and explosion. Insufficient memory for long sequences often occurs during complex operations. To remedy this defect, LSTM is proposed. The LSTM network model can use its unique gate structure to remember the states much better than the traditional RNN, which has greatly improved its performance in dealing with timing-related problems.

It can be found that compared with the traditional RNN structure, the LSTM cell adds a layer of cell state at the top of the cell, through which the state at all times can be transferred in the LSTM chain. The specific calculation process can be expressed by Formulas (1)–(6) (Shi et al., 2022):

(1)${\displaystyle f_t=\sigma \left(W_f\cdot \left[h_{t-1},x_t\right]+b_f\right)}$ (2)${\displaystyle \widetilde{C_t}=tanh\left(W_C\cdot \left[h_{t-1},x_t\right]+b_C\right)}$ (3)${\displaystyle i_t=\sigma \left(W_i\cdot \left[h_{t-1},x_t\right]+b_i\right)}$ (4)${\displaystyle C_t=f_t\times C_{t-1}+i_t\times \widetilde{C_t}}$ (5)${\displaystyle o_t=\sigma \left(W_o\left[h_{t-1},x_t\right]+b_o\right)}$ (6)${\displaystyle h_t=o_t\times tanh\left(C_t\right)}$

LSTM shown in Fig. 2 can be expressed by the above formula. Compared with BP neural network and RNN, the gate structure in LSTM is the most distinctive and different from the traditional neural network. The specific implementation process is completed according to Formulas (1)–(3). First, the forgetting gate is the most crucial feature, indicating what features in Ct-1 can be used in Ct. Formula (2) $\widetilde{C_t}$ updates the cell state through the tanh function, which the input data and ht-1 can obtain. While it in Formula (3) represents the input gate, similar to ft in Formula (1), which is also calculated by the activation function σ andht-1. After the calculation of it in Formula (3), we can determine which features can be used to update Ct. as shown in Formula (4). Finally, the output h_t can be calculated through Formulas (5) and (6) to complete the next unit’s input and prediction value calculation.

In Formulas (1)–(6), W_f, W_i, W_c, W_o is the weight matrix corresponding to each part, b_f, ${\mathrm{b}}_{\mathrm{i}},{\mathrm{b}}_{\mathrm{c}}^{-},{\mathrm{b}}_0$ is the bias. Σ and tanh both represent activation functions (Chen et al., 2022) which can be expressed as follows (7)${\displaystyle \begin{array}{c}\hfill \sigma \left(\mathrm{x}\right)=\frac{1}{\left(1+{\mathrm{e}}^{-\mathrm{x}}\right)}\hfill \end{array}}$ (8)${\displaystyle \begin{array}{c}\hfill tanh\left(\mathrm{x}\right)=\frac{\left({\mathrm{e}}^{\mathrm{x}}-{\mathrm{e}}^{-\mathrm{x}}\right)}{\left({\mathrm{e}}^{\mathrm{x}}+{\mathrm{e}}^{-\mathrm{x}}\right)}.\hfill \end{array}}$

According to Formula (9), the final predicted value y is obtained through a complete connection layer. (9)${\displaystyle {\mathrm{y}}_{\mathrm{t}}=\sigma \left({\mathrm{W}}_{\mathrm{y}}\cdot {\mathrm{h}}_{\mathrm{t}}+{\mathrm{b}}_{\mathrm{y}}\right)}$

where, W_yandb_y are the weight matrix and bias, respectively.

PSO optimization model

The neural network-related methods usually employ the gradient descent method to learn and optimize the network weight when optimizing the model, so there is a specific sensitivity to its initial value. The algorithm will converge to the local extreme if initialization is improper, significantly reducing the model accuracy. Therefore, this article plans to use the PSO method to globally optimize the initial value of LSTM to improve the network performance (Rokbani, Abraham & Alimi, 2013; Kefi et al., 2016).

The PSO initialization is to generate a swarm of particles randomly. They were then iterated to change the position of particles to obtain an optimal value. In each iteration, the particle changes its speed and position by comparing the relationship between the current one and the optimal local solution pbes_ti and global optimal solution gbes_ti, to achieve the goal of updating the particle. When these two optimal values are obtained, we could update their states as follows: (10)${\displaystyle \begin{array}{c}\hfill {\mathrm{v}}_{\mathrm{i}}={\mathrm{v}}_{\mathrm{i}}+{\mathrm{c}}_1\times \mathrm{r}\times \left({\text{pbest}}_{\mathrm{i}}-{\mathrm{x}}_{\mathrm{i}}\right)+\hfill \\ \hfill {\mathrm{c}}_2\times \mathrm{r}\times \left({\text{gbest}}_{\mathrm{i}}-{\mathrm{x}}_{\mathrm{i}}\right)\hfill \\ \hfill {\mathrm{x}}_{\mathrm{i}}={\mathrm{x}}_{\mathrm{i}}+{\mathrm{v}}_{\mathrm{i}}\hfill \end{array}}$

where i is the particle number in the particle swarm. r represents a random number between (0,1); v_i represents the current velocity; x_i is the current position; c1 and c2 is learning factor; local optimal position and global optimal position are pbesti and gbesti, respectively. Parameter optimization of the LSTM model using the PSO method is shown in Fig. 3. First, the particle swarm is randomly initialized. Then, the fitness function is calculated for the model fitting effect. Then, the fitness is computed, and the best particle that meets the conditions is obtained according to the current particle state.

As shown in Fig. 3, the PSO-LSTM method used in this article first completes the application of the PSO method through parameter initialization. Then it uses its principle to update the parameters and calculate the prediction results of the LSTM. The final model can be determined after meeting the requirements of fitness, prediction accuracy, or iteration number.

The result of the smart agriculture-related economy regression

The results obtained from the LSTM and PSO optimization training models introduced in Section 2.1 and 2.2 are illustrated in Fig. 4. We can find that the methods proposed in both trend and prediction estimation have good results. The relative errors of each year are shown in Fig. 5:

As shown in Fig. 5, the relative error of the model proposed keeps around 1%. The year with the most significant prediction error occurred in 2017, and the year with the minor error was 0.6% in 2015 and 2020. Meanwhile, to illustrate the performance of the model proposed in this article, the PSO-LSTM method used in this article is compared with the LSTM and SVR methods commonly used in machine learning. The comparison curve of the CAE production is displayed in Fig. 6.

As the production curve trend shows in Fig. 6, three models can correctly predict the economic trend, and only SVR had some deviations in 2018. To better illustrate the result of the three models, we employ Root Mean Squared Error (RMSE) and Mean Absolute Deviation (MAE) to assess the model performance. The calculation methods are as follows in Formulas (11) and (12):

(11)${\displaystyle RMSE=\sqrt{\frac{1}{n}{\sum }_{i=1}^n{\left(X_i-\hat{X}\right)}^2}}$ (12)${\displaystyle MAE=\frac{1}{n}{\sum }_{i=1}^n\left|X_i-\hat{X}\right|}$

where X is the prediction and $\hat{\mathrm{X}}$ is the actual value. The model accuracy will increase for these two evaluation indicators if they become smaller. The results of RMSE and MAE among the three models are given in Table 2:

As shown in Table 2, the LSTM model itself has certain advantages over the SVR method, and the advantages are further revealed after optimization through PSO.

Prediction accuracy with historical data

To study the model in more detail, in this part, we will refine each quarter of the four years from 2018 to 2021 and forecast the data for the first quarter of this year. The accuracy rate is shown in Fig. 7:

In Fig. 7, the abscissa represents the use of different amounts of quarterly data for forecasting, and the ordinate represents the accuracy of the forecast results. Four means that only the first quarter data of each year is used for prediction. While eight means that the data from nearly eight quarters are used for prediction, and the results are significantly better than the first quarter of each year. The meanings of 12 and 16 are the same as those of eight, using the latest 12 or 16 quarters to forecast the economic output value in the first quarter. It can be found that the accuracy of the model prediction increases when the data volume expands. However, when using the LSTM model alone to forecast, it is found that the data for 12 quarters is not as good as the data for eight quarters. Through comparison, the model falls into the optimal local solution of the initial value, leading to a decrease in prediction accuracy.