Evaluation of cultivated land quality using attention mechanism-back propagation neural network

Overview of the research area

Located in the transition zone between the western edge of the Sichuan Basin and the Qinghai–Tibet Plateau, Yucheng District covers an area of 1067.3 km², and it belongs to one of the mountain counties around the basin. With the impact of urbanisation, the amount of arable land resources in China is decreasing (Deng et al., 2015), with the per capita arable land area only 2/3 that of the world’s average. Soil quality is seriously degraded whilst indiscriminate and excessive cultivation and occupation for construction occur ubiquitously. Therefore, the use of artificial intelligence methods for evaluating arable land is indispensable to understanding the current status of the quantity and quality of arable land resources in local areas. On the basis of this assessment, targeted arable land protection and management projects can be implemented to achieve smart and sustainable agricultural development.

Data source

The research data are derived from soil testing and fertilisation in Yucheng District, and they are provided by the Yucheng District Agricultural Bureau of Ya’an City. The average area of each sampling unit is 100–200 mu, setting up representative sampling points. The sampling depth is generally 0–20 cm, and the orchard can be collected in layers of 0 to 40 cm or 0–60 cm.

The accuracy rate is controlled by the use of standard samples. To identify the error, we add one standard sample for each batch. When the difference between the testing result and the standard value is out of the range of the standard deviation, the experimental procedure should be checked to find out the reason.

Data preprocessing

Data preprocessing is a significant task. Different dimensional units affect the results of our numerical analysis. Therefore, data must be standardised to ensure their comparability and eliminate such effects. The min-max standardisation for the data is as follows: (1)${\displaystyle Z=\frac{x-x_{\min }}{x_{\max }-x_{\min }},}$ where x_max is the maximum value of the sample data, and x_min is the minimum value of the sample data.

Construction of an index system

In accordance with the regional characteristics of cultivated land in the Yucheng District of Ya’an City, combined with data access, data statistics, expert consultation, and other methods, indexes, such as surface soil texture, soil organic matter, soil nutrient elements, and pH value, are selected from natural elements to build a quality evaluation system for Ya’an cultivated land, as presented in Table 1.

Model building

BP neural network

A BP neural network is composed of an input layer, a hidden layer, and an output layer. In the current study, the number of input layer nodes is seven due to the cultivated land characteristics of the data source and the obtained score. Meanwhile, the output is 1, which represents the quality score of cultivated land. When the number of neurons in the hidden layer is suitable, better results can be obtained (Karsoliya, 2012). The selection of the number of hidden layer nodes is presented in Eq. (2): (2)${\displaystyle n=\sqrt{a+b}+c,}$ where n, a, and b represent the numbers of hidden layer nodes, input layer nodes, and output layer nodes, respectively; and c represents an integer between 1 and 10.

Therefore, the mean square error (MSE) of the model results at different numbers of hidden layer nodes is adopted, as shown in Table 2.

As indicated by the result, the effect of the model is the best when the number of hidden layers is five. In the BP model, the optimiser used is Adam, the learning rate is 0.01, the maximum training time is 500, and batch size is 100. Thus, the establishment of the BP neural network model is completed, as illustrated in Fig. 1.

AM

Attention is one of the most important components of neural networks that are adept at understanding sequences, including video sequences, action sequences in real life, or a sequence of inputs, such as voice, text, or any other data. Obtaining a reasonable vector representation is typically difficult because the input sequence of long short-term memory or a recurrent neural network may be too long. Thus, AM is created. The attention mechanism not only greatly improves the efficiency and accuracy of perceptual information processing, but also provides the interpretability for the model generation process (Niu, Zhong & Yu, 2021). At present, the application of AM to the field of agriculture is gradually becoming popular (Zhao et al., 2021; Zhao, Huang & Nie, 2021). However, it has no relevant application to cultivated land quality.

AM was proposed by Bahdanau, Cho & Bengio (2014). Its original purpose was to automatically search relevant parts from the input sequence of the current predictor by constructing a mechanism. These authors used AM to build a better translation model, proving that AM is reasonable.

To understand AM, we can divide it into three steps: score function, alignment function, and generate context vector function.

Score Function: The similarity between the environment vector and the current input vector is measured. The important input information that should be focused on in the current environment is identified. These authors used AM to build a better translation model, proving that AM is reasonable. The structure of AM is shown in Fig. 2. The calculation method is presented in Eq. (3): (3)${\displaystyle e_{ij}=a\left(c,h_i\right)=v_a^{T}tanh\left(W_a\ast c+U_a\ast h_i\right),}$ where e_ij is the alignment model that is used to measure the alignment of the j-th word at the input position to the i-th word at the output position. a and c represent the alignment model function and the context vector. h_i is the input. v_a, W_a and U_a are the weight matrices.

Alignment Function: Attention weight is calculated. Softmax is typically used for normalisation. The calculation method is presented in Eq. (4): (4)${\displaystyle {\alpha }_{ij}=\frac{exp\left(e_{ij}\right)}{\sum _{k=1}^{T_x}exp\left(e_{ik}\right)},}$ where T_x represents the last symbol of the input vector.

Generate Context Vector Function: The output vector is obtained with the attention weight. The calculation method is presented in Eq. (5): (5)${\displaystyle z_{ij}=\sum _i{\alpha }_{ij}\ast h_i.}$

Ensemble model

The ensemble model imports the data trains multiple models and integrates one model by a certain method. It is easy to understand, simple and efficient. This method is one of the common approaches for improving the model effect. The methods for the ensemble model are generally divided into voting, averaging, boosting, and stacking. In the current study, we use the averaging method to obtain the result, as shown in Eq. (6): (6)${\displaystyle \mathrm{result}=\sum _{t=1}^n{\mathrm{Weight}}_i\ast P_i,}$ where n, Weight_i and P_i represent the number of models, model weight, and the predicted value of the model, respectively.

Evaluation index

To evaluate the consistency between predicted and true values, MSE and the determination coefficient (R²) were used to assess the performance of the models in this study, as shown in Eqs. (7) and (8): (7)${\displaystyle \mathrm{MSE}=\frac{1}{n}\sum _{i=1}^n{\left({\hat{y}}_i-y_i\right)}^2,}$ (8)${\displaystyle R^2=\frac{\sum _{i=1}^n{\left(y_i-{\overline{y}}_i\right)}^2}{\sum _{i=1}^n{\left({\hat{y}}_i-{\overline{y}}_i\right)}^2},}$ where n is the total number of samples, and ${\hat{y}}_i$, y_i and ${\overline{y}}_i$ represent the predicted values of i-th sample, the true values of the i-th sample and the average values of the total samples.

Laboratory environment

This research is completed using a Windows 10 operating system and a TensorFlow 2.0 framework. The experimental hardware environment is an HP8574 motherboard. The CPU model is Intel Core i7 9750H. The GPU model is NVIDIA GeForce GTX 1660 Ti 6G. The software environment is CUDA 10.0, CUDNN 7.6.5, and Python 3.7.

Results of different machine learning methods

Machine learning has five characteristics: accuracy, automation, speed, customisation and scale. It is applied in our daily lives. To test the effect of the machine learning model on farmland datasets and the subsequent ensemble model, 2150 samples from the datasets are used to train 13 machine learning algorithm models, whilst 920 samples are used for testing. The 13 algorithms are as follows: a gradient boosting, extreme gradient boosting (XGBoost), decision tree, linear regression, support vector regression (SVR), k-nearest neighbour (k-NN), random forest regression, adaptive boosting (AdaBoost), bootstrap aggregating, extreme tree regression, lasso regression, ridge regression and light gradient boosting machine (LightGBM). They respectively correspond to GB, Xgboost, DT, LR, SVR, k-NN, RFR, AdaB, Bag, ET, Lasso, Ridge, and Lgb in Fig. 3. The MSE of each model on the test set is obtained. The result is presented in Fig. 3.

As shown in Fig. 3, the k-NN model exhibits the best effect and the minimum MSE. It is more favourable for the subsequent model fusion to reduce MSE. It is selected as one of the subsequent model fusion algorithms.

Comparison of model effects

To integrate AM into the model, we combine the model with the characteristics of AM, as shown in Fig. 4. In the current study, the attention layer is designed to have two layers: a softmax layer and a multiply operation. The softmax layer, which is added after the input layer, can obtain attention scores and then multiply the scores by the input.

As a common practice in machine learning, data are divided into 2150 training samples and 920 test samples. The other parameters are the same except for the different network structures. These parameters are applied to the BP and AM-BP models. The results are presented in Fig. 5. The AM-BP model is better than the BP model. The MSE is reduced by approximately 0.002 (approximately 12%), the R² is increased by approximately 0.005 at the same time.

The MSEs of the BP and AM-BP models on the test set is provided in Table 3. They show that the model with AM is more sensitive to data and exhibits a better learning effect. The AM-BP model also has shortcomings. It allocates attention through weights, which increases the complexity of the network. In addition, the sequential relationship in input cannot be learned by the model.

The AM-BP model can return the value of the attention score, as shown in Fig. 6.

In this figure, pH, SST, TN, AN, AP, AK, and SOM correspond to pH value, surface soil texture, total nitrogen, available nitrogen, available phosphorus, available potassium, and soil organic matter, respectively.

When the score is higher, the correlation between the model and related items is higher. As shown in Fig. 6, soil texture and alkali-hydrolysable nitrogen exert more important influences on the score. Soil texture directly affects the physical and chemical properties and fertility status of the soil. For example, loam soil exhibits better cultivability and fertility, and its capability to supply and maintain fertilizer is moderate. It is most suitable for the growth of fruit trees and vegetables. The cultivability and fertility of clay are worse than those of loam. Large pores are lacking between soil particles, and air and water permeability are poor. Clay is more suitable for planting rice.

Effects of ensemble model

In general, when the learner’s appointment is more accurate and diversified, its integration is better. To improve the accuracy of the model further, the k-NN, BP, and AM-BP models are fused, and the result is presented in Fig. 7. mix1, mix2, mix3, and mix4 are the results of the fusion of the k-NN, BP, and AM-BP models with certain weights, as shown in Table 4. In addition to the mix1 and mix2 model, the accuracy of the two other models is significantly improved.