Improving the accuracy of soil texture determination using pH and electro conductivity values with ultrasound penetration-based digital soil texture analyzer

Material

Some of the data used in this study were taken from the study of Orhan et al. (2022). This system, known as USTA, is presented as a soil texture analysis device that can be an alternative to hydrometer analysis. Hydrometer analyzes of all soils used in the experiments were made by Çukurova University Faculty of Agriculture, Department of Soil Science and Plant Nutrition. The measurement and estimation steps of the system are roughly as follows: Some soil-water mixture is placed into the measuring container and the container’s lid is closed. The intensity of the ultrasound signals passed through the mixture with the help of receiver and transmitter ultrasound sensors operating in the range of 1 MHz and 0V–5V positioned opposite each other on the container, was collected on the computer for 2 h at a frequency of 2 Hz (14,400 columns in total). Then, t = [10 s, 20 s, 40 s, 80 s, 3 min, 7 min, 15 min, 30 min, 60 min, 120 min] data points, which were thought to be the best representative points of the entire time series, were taken and a data set was created, consisting of 80 rows (80 soils) and 10 columns (10 features). These data were given as input to various machine learning methods and sand, silt and clay predictions were made. Compared to traditional sedimentation methods, the ultrasound technique offers several advantages. Orhan et al. (2022) demonstrated that this method, utilizing low-cost electronic components, requiring minimal preparation, and operating independently of human expertise, could achieve deviation values comparable to traditional methods. These advantages underscore the suitability of the ultrasound penetration technique for modern soil texture analysis, addressing the limitations of traditional methods while enhancing accuracy and efficiency.

Based on this, first, new data points that gave the best results were selected from the old data through combinatorial experimentation. The best segments found for sand, silt and clay were t = [48 284], t = [140 392] and t = [138 428] intervals, respectively. Then, soil pH and EC values, two new parameters whose importance we emphasized in texture analysis, have been added to the data set used in the previous study, and the effects of these new parameters on the prediction ability of the USTA analyzer have been investigated.

In this study, pH and EC values of 69 out of 80 soils, which were the same soils used in the previous study, were measured and recorded. A total of 11 soil data were removed from the data set because pH and EC values could not be measured due to insufficient samples. All pH and EC measurements were carried out in a laboratory environment with professional equipment in accordance with the procedure (Jackson, 1964; Waters et al., 1972). The measurement steps are briefly as follows:

• 10 g of air-dried, 2 mm sieved soil is weighed.

• Sample is placed in a 50 ml laboratory beaker.

• Pure water is added at a ratio of 1:2.5 (25 ml).

• The suspension is mixed at regular intervals for 1 h.

• Left to stand for 30 min.

• Before measurement, it is stirred one last time and measured with a glass electrode pH meter (or with an EC meter to measure the EC value).

• The measurement value is recorded by reading at the first decimal level after the comma.

pH measurements of all soils were carried out with the Thermo Scientific Orion Star A221 Portable pH Meter, and EC measurements were carried out with the Thermo Scientific Orion Star A222 Portable Conductivity Meter. Both devices have temperature compensation functionality. Each pH and EC measurement was repeated at least three times to eliminate possible measurement errors and the average of these three repeated measurements was taken. The pH values of the measured soils ranged between 7.5 and 8.5, and the EC values were found to be between 147.9 $\mathrm{\mu }$S/cm and 1,480 $\mathrm{\mu }$S/cm. These values were recorded on the computer as the pH and EC values of the measured soil. Then, two new columns (in other words, two new features) were added to the newly formed data sets, as pH and EC values for each soil. As a result, a data set consisting of 69 rows and 239 columns for sand, 254 columns for silt, 292 columns for clay, including pH and EC values for 69 soils, was obtained. Thus, by excluding and including pH and EC parameters, it becomes possible to see the effects of these new parameters on the prediction success comparatively.

Methods

Today, machine learning methods have significantly improved the analysis and forecasting of time series-based datasets used in various fields. These methods have become very powerful for capturing complex relationships and patterns invisible to the human eye and making precise predictions. Although the literature on machine learning is very extensive, methods such as support vector regression (SVR), Random Forest (RF), artificial neural network (ANN) and their derivatives are among the most used machine learning methods in time series analysis. While alternative methods, such as gradient boosting algorithms (e.g., XGBoost) or deep learning models like convolutional neural networks (CNNs), have been considered, their complexity and higher computational requirements often make them less suited for resource-limited contexts (Akande, Ajayi & Faloye, 2022; Nguyen Duc et al., 2022). Furthermore, RF and multi-layer perceptron neural network (MLPNN) have consistently shown competitive or superior performance in soil-related applications, as evidenced by their success in predicting soil organic carbon (Were et al., 2015) and optimizing soil density and moisture parameters (Nguyen Duc et al., 2022; Zhang, Liu & Tie, 2023). These methods were not only central to the experiments in this study but were also adopted in Kilinc (2022) which can be considered as base work to this study as optimal models due to their cost-efficiency, ease of implementation, and ability to yield reliable predictions across diverse soil datasets.

SVR is used for nonlinear regression problems and is frequently used in time series forecasting such as traffic flow forecasting (Hong et al., 2011; Li, Hong & Kang, 2013), staple food price forecasting (Astiningrum, Wijayaningrum & Putri, 2021), power load distribution (Chen et al., 2017; Tran et al., 2024), cargo volume estimation (Chan, Xu & Qi, 2018; Nieto, Benitez & Martinez, 2021) etc. RF, which is a member of the ensemble learning approach, is a very efficient and easy to implement classification and regression tool especially for the datasets consisting of high variables-to-observations ratio (Biau & Scornet, 2016). It is a highly successful method used in areas like illness prediction (Kane et al., 2014; Wu et al., 2017; Zhang & Nawata, 2017), streamflow forecasting (Papacharalampous & Tyralis, 2018), construction safety assessment (Tixier et al., 2016) etc. because of its predictive power. ANN is also a widely used machine learning method to overcome time series based problems (Bas, 2016). It has significantly wide use cases in areas like predicting agricultural output (Awe & Dias, 2022), power system forecasting (Shahriar, Hasan & Abrar, 2019) etc.

In this study, new experiments on sand, silt and clay ratio predictions were carried out by taking pH and EC values into account as new parameters, with the help of SVR, RF and ANN machine learning methods for 69 soils.

Estimations using SVR

We have used SVR method first, for estimation. In order to see the effects of pH and EC parameters comparatively, estimations were first made with plain soil data without pH and EC values. The SVR model was trained using 68 out of 69 soil data, the remaining one soil data was given as input to the created SVR model as test data (leave-one-out cross validation), and the prediction results of the model were recorded. These processes were repeated for all 69 soil data and prediction results were obtained for all soils. For sand, silt and clay predictions, kernel function, kernel scale and epsilon parameters were tried in combination and the best values are given in Table 2.

After the optimal parameters are determined as in Table 2, the same estimations were made taking pH and EC values into account. Again, these processes were repeated for all 69 soil data. The prediction ratios and error rates obtained as a result of the estimation of 69 soils are given in Fig. 1.

Figure 1 shows the particle ratios measured with the hydrometer on the x-axes, and the estimation results made with the SVR model on the y-axes. The red circles represent the estimation results made using plain soil data, that is, without pH and EC value. The points shown with blue asterisks represent the prediction results obtained by including pH and EC values. The prediction success criterion of the created model against actual values is presented with the R² statistical method. In Fig. 1A, R² = 0.55 was found for sand without pH and EC values, and R² = 0.54 was found by taking pH and EC into account. It is seen that pH and EC values do not increase the estimation results (or even reduce them to a very small extent). This is thought to be due to the fact that the first particles to settle are sand in sedimentation-based measurement methods. Because sand settles almost completely in the first seconds of hydrometer measurements, obtaining a reliable hydrometer reading at 0^th second (the very first moment when all particles are uniformly dispersed and suspended in suspension) is almost impossible and is not usually performed. Instead, after all the sand particles have settled, the total percentage of silt + clay suspended in suspension is found and this value is subtracted from 100 to determine the sand ratio. Calibration is attempted by subtracting the hydrometer value read by immersing it in NaPO₃ and pure water solution before the measurement, also called blank reading, from all particle ratios found after the measurement. In fact, it is not possible to talk about a complete and proper calibration here that includes the effect of all soluble substances. An attempt is made to neutralize only the effect of NaPO₃ in the suspension. In this case, it is actually an expected result that pH and EC, as indicators of soluble substances, do not significantly affect the sand estimation results. In contrast, clay particles, which remain suspended in the soil-water mixture for a longer duration, are more susceptible to interference from water-soluble substances. These substances can alter the suspension and settling dynamics, making pH and EC critical features for accurately estimating clay content. Therefore, the perception of the soluble matter effect is expected to be more evident in silt and clay predictions.

Figure 1B shows the estimation results for silt. It was found that R² = 0.071 without pH and EC value, and R² = 0.28 by taking pH and EC into account. As expected, there is a significant improvement in the prediction values when pH and EC values are included in the calculation. Especially when we look at the right side of the graph, that is, the soils with high silt rate, it can be clearly seen that the estimation results have significantly improved (the blue asterisks get closer to the 1:1 line) as the effect of dissolved substances spread to the measured values for a longer period of time.

Similarly, Fig. 1C shows the effect of pH and EC values on clay estimation. While R² = 0.42 without pH and EC values, it is seen that R² = 0.59 when these values are included in the calculation. Here, pH and EC can be most associated with clay because, as in the hydrometer method, the last recorded time series values in this system consist only of density information created by clay and soluble substances together. Therefore, it can be said that as sand and silt particles settle, the prediction success increases significantly with pH and EC values. The prediction successes obtained with the SVR method for sand, silt and clay particles are given comparatively with the R², MSE and MAE statistical criteria in Table 3.

In Table 3, whichever result is better for each evaluation criterion in the sand, silt and clay categories is written in bold. When we consider the sand prediction, it is seen that the pH and EC properties of the measured soils do not significantly increase the prediction success of the model produced by the SVR method. On the other hand, looking at silt prediction, it appears that the inclusion of pH and EC significantly improves the performance of the model. The increase in the R² value from 0.07 to 0.28 shows that the presented model is effective in silt prediction. The decrease in both MSE and MAE values confirms the prediction accuracy. When we examined the model for clay prediction, it was seen that the performance of the model increased by including pH and EC in the calculation. In particular, the jump in the R² value from 0.42 to 0.59 and the decrease in both MSE and MAE indicate that the inclusion of pH and EC properties in the calculation has positive effects on the prediction success.

Estimations using RF

The RF method creates a set of decision trees with many different sub-data sets by bootstrap sampling from the original data set and predicts the output according to the majority of the trees in the set. The number of decision trees of the method is its main parameter. In order to objectively determine the prediction effect of pH and EC in the study, in the first step, RF models that produced the best prediction results for sand, silt and clay without pH and EC values were determined. In the second step, predictions were made again using the models found, but this time taking pH and EC values into account. In this way, it will be possible to clearly test whether the new parameters further improve the prediction success with the same models. A total of 69 soil sample data, without pH and EC values, were given as input to 10 different RF models containing different number of decision trees [50–500] with the leave-one-out cross validation method (68 for train 1 for test). R² values obtained according to the number of decision trees are given in Fig. 2.

Figure 2 shows the R² values produced by 10 different RF models for sand, silt and clay predictions, created by increasing the number of decision trees by 50 in the range of 50–500. The highest R² values for sand, silt and clay can be seen marked with circles in the figure. RF models with 100 decision trees, 200 decision trees and 200 decision trees produced the best prediction results with values of R² = 0.48, R² = 0.25 and R² = 0.26, for sand, silt and clay, respectively. Increasing the number of decision trees any further did not yield any significant improvements. Therefore, to obtain the best results, models with different decision trees were used separately for sand, silt and clay predictions. As a second step, after determining the models that produced the best results without pH and EC models, the same models were re-trained by taking pH and EC values into account, and the results obtained by taking these values into account are comparatively shown in Fig. 3.

In Fig. 3, the x-axes show the actual sand, silt and clay values, while the y-axes show the predictions made with RF models with 100 decision trees, 200 decision trees and 200 decision trees, respectively. Red circles show the predictions made without pH and EC values, and blue asterisks show the prediction results made by taking pH and EC values into account. The success of the predictions in terms of R², MSE and MAE are presented comparatively in Table 4.

Table 4 shows the RF models with the same features and parameters for sand, silt and clay, and the predictions made with and without taking pH and EC values into account. Whichever result is more successful is indicated in bold. The R² value for sand increased from 0.48 to 0.52, for silt from 0.25 to 0.33, and for clay from 0.26 to 0.31. When we look at the table, we see that there is a significant increase in success, especially when pH and EC values are taken into account in silt and clay estimation, as well as an increase in success in sand estimation, unlike the SVR method. Moreover, the improvement of MSE and MAE metrics shows that the general prediction accuracy has improved and that these parameters must be taken into account in sedimentation-based soil texture analysis methods.

Multi-layer perceptron neural network

MLPNN is simply the improved version of ANN method by adding multiple layers, which are also called hidden layers, between input and output layers. Complex neural network structures can be created with neurons placed in these hidden layers. In the first step of the MLPNN experiments, 69 soil data without pH and EC values were given as input to MLPNN models with eight different structures containing 5, 10, 15, 20 neurons in one hidden layer and 5, 10, 15, 20 neurons in each of two hidden layers; which are herein after shortened as 1L5N, 1L10N, 1L15N, 1L20N, 2L5N, 2L10N, 2L15N, 2L20N, respectively, for ease of representation. Cross-validation method was used for training and testing (68 for train 1 for test). Thus, MLPNN models that produced the highest prediction values for sand, silt and clay without pH and EC values were determined. The epoch number was chosen as 1,000, the activation function was chosen as hyperbolic tangent sigmoid. The backpropagation algorithm was conjugate gradient backpropagation. The R² values obtained with the MLPNN model with eight different structures are given in Fig. 4.

As can be seen in Fig. 4, eight different MLPNN models were used to predict sand, silt, and clay ratios without pH and EC values. Estimation results were presented in terms of R². Accordingly, 1L15N structured model produced the best results for sand, 2L5N structured model produced the best results for silt, 2L10N structured model produced the best results for clay. As the second step, these models were re-trained by taking pH and EC values into account. It was examined whether the prediction successes are increased or not. The sand, silt and clay prediction successes obtained with the determined MLPNN models, taking into account pH and EC values, are given side by side in Fig. 5.

In Fig. 5, while the x-axes show the actual sand, silt and clay values, the y-axes show the R² performance of the sand, silt and clay predictions made with the MLPNN model containing 15 neurons in one hidden layer, five neurons in each of the two hidden layers and 10 neurons in each of the two hidden layers, respectively. All prediction performances are given in Table 5 in terms of R², MSE and MAE.

In Table 5, as a result of the predictions made with and without using pH and EC, whichever result is better is indicated in bold. The model considered for sand increased the R² value from 0.48 to 0.54 when pH and EC values were taken into account. The slight decrease in both MSE and MAE values confirms that pH and EC features should be included in training the model. On the other hand, the success rate in silt and clay is better only by fractions and no significant improvement is achieved as in the case of SVR and RF methods. All success rates for the trained and tested models are given comparatively in Table 6.

In Table 6, better results are indicated in bold. When we look at the results in Table 6, the best results for sand prediction were obtained with MLPNN and RF methods, which increased the R² value from 0.48 to 0.52. The SVR method was insufficient to produce better results with pH and EC values in sand estimation, but in clay estimation, a significant improvement was achieved by increasing the R² value from 0.42 to 0.59. In fact, when we look at overall silt and clay estimates, we see that higher estimates can be made by using pH and EC values in all three methods. Specifically, for silt, it is seen that the SVR method increases the R² value from 0.07 to 0.26. However, the RF method, which already has a silt prediction success of R² = 0.25 without pH and EC parameters, produced the most meaningful results by making even better predictions with these parameters, and managed to increase the R² value to 0.33. No matter which parameters we use with the RF method, R², MSE and MAE values were always better in all predictions where pH and EC values were included.

In short, to assess the significance of pH and EC as predictors, models were trained using two datasets: one with only ultrasound time-series data and another with ultrasound data, pH, and EC included. Performance was evaluated using R², MSE, and MAE metrics. Results indicated that the inclusion of pH and EC improved R² values for sand, silt, and clay predictions by approximately 25%, with corresponding reductions in MSE and MAE. These findings underscore the importance of integrating chemical properties like pH and EC to capture soil texture variability more effectively.

Classification experiments

In this study, the impact of including pH and EC values as features on classification accuracy was investigated using eight widely used machine learning classifiers. For training to be done properly, there should be three or more representative soil sample of same class in the dataset. A dataset comprising 66 samples was derived from the original dataset by selecting soils that had at least three additional samples belonging to the same classification group. The distribution of soil samples across the classification groups is detailed in Table 7.

The objective was to evaluate how the inclusion of pH and EC affects classification performance and to explore the role of feature selection in optimizing model outcomes. The classifiers employed, along with their parameters, are detailed in Table 8.

To address computational efficiency, the size of the time-series dataset was reduced to 100 features. This reduction was achieved by retaining higher frequency data points at the initial rapid rise of the time series and progressively selecting more sparse data points as the values stabilized. Using these 100 features, classification was performed with the models listed in Table 1, first without pH and EC values, and subsequently with their inclusion. Leave-One-Out Cross Validation was employed for model evaluation, ensuring each sample served as a test instance exactly once while the remaining samples were used for training. Classification accuracy, defined as the ratio of correctly classified samples to the total number of samples, was used as the evaluation metric, as expressed in Eq. (10):

(10) $$Accuracy=\frac{TotalNumberOfCorrectPredictions}{TotalNumberOfSamples}$$

The accuracy results for the models, both with and without pH and EC values included, are presented in Table 9.

The results indicate that including pH and EC as features generally improved classification accuracy in most models. Logistic regression demonstrated the most notable improvement, with accuracy increasing from 57.58% to 66.67%. Gradient boosting and XGBoost also exhibited enhanced performance, achieving 57.58% accuracy with the inclusion of pH and EC. These findings highlight the relevance of pH and EC as predictive features for soil texture classification. However, k-nearest neighbors and SVM did not show notable increase in accuracy, potentially due to overfitting caused by the additional features.

An important consideration is that while pH and EC values enhance the prediction of sand, silt, and clay percentages, soil texture classification presents distinct challenges. Texture classes are defined using the soil texture triangle, and small improvements in sand, silt, and clay predictions (e.g., 3–5%) may not alter classification outcomes if the sample is centrally located within a texture class. However, for samples near the boundaries of texture classes, even minor changes in sand, silt, or clay percentages can shift the classification, as illustrated in Fig. 6.

As depicted in Fig. 6, soil sample 1, located in the center of the “clay” class, remains classified as “clay” even if there is a 10% change in sand, silt, or clay percentages. In contrast, soil sample 2, positioned at the boundary between “clay” and “silty clay,” shifts classification with only a 3–5% change. This demonstrates that including pH and EC improves prediction performance, particularly for samples near class boundaries in the soil texture triangle. However, since such boundary cases are limited, the overall impact on classification accuracy is also expected to be limited.