Predictive maintenance in Industry 4.0: a survey of planning models and machine learning techniques

Built-in design

PdM enhances product quality while lowering downtime by forecasting system failures. Two techniques employed to gather condition data during continuous monitoring and inspection are the direct (offline) monitoring and the indirect (online) monitoring technique, where either one of the techniques can be used. The offline monitoring technique requires the maintenance team to be present physically to supervise the machinery regularly interrupting the processes. On the contrary, online monitoring, without interrupting the processes, supervises the machines constantly using sensors (Abidi, Mohammed & Alkhalefah, 2022). For PdM decision-making, it is further necessary to combine several types of data, including maintaining data-logs of maintenance activities, identifying causes of failures, collecting data for monitoring, analysing the impacts of failures, and setting maintenance requests for any necessary repairs or maintenance. Figure 1 depicts the PdM planning model’s design.

The PdM planning model contains five key stages: data cleansing, data normalisation, optimal feature extraction, decision model, and prediction model. First, the datasets are cleaned by locating misfits and adding any missing data. After being cleaned, the values are normalized, putting them inside a defined range (0–1). This is followed by the removal of redundant data by optimal feature extraction (Sisode & Devare, 2023). With the widespread use of sensing devices for condition tracking brought on by I4.0, making decisions under the constraints of time is now made easier. As a result, the period between the occurrence of an expected breakdown and the point at which it breaks down into an operational failure is the time during which decision-making algorithms can be used to suggest measures required to prevent or minimize the effects of the expected malfunction (Zine, Belkhiri & Mohamed, 2022). Though this article discusses the data normalisation and FS steps for the PdM planning model, they need not be mandatory steps for a standalone application of ML techniques for maintenance problems. They may or may not be necessary depending on the research question, the ML algorithm selected, data availability, etc.

Data cleaning

When data is cleaned, problems are identified in information and fixed by referring to all different kinds of actions and operations in the dataset. Here, abnormality recognition and value replacement are used to achieve data cleansing (Abidi, Mohammed & Alkhalefah, 2022).

Outlier data: In analytics, outliers are values that differ notably from the other values in a dataset—either by being considerably larger or smaller. For various applications, some outlier identification models have already been presented where these techniques vary in how generic they are (Akhiat, Chahhou & Zinedine, 2018). Data patterns known as outliers do not correspond to the usual dynamics in the data. For example, misfits can be found and replaced by utilising the fill approach through the filloutliers() MATLAB function. To obtain the remaining components and decompose them early, anomaly detection is performed.

Missing data management

This problem happens whenever there are certain values absent from the data. Both managing incomplete data and analysing the issues brought about by missing data need more computing and analytical time. The fillmissing() MATLAB function can replace missing records in an array with fixed values to fill in missing information (Akhiat, Chahhou & Zinedine, 2019). The function carries out the synthesis of assumptions and residuals. The variables may also be divided as residual and approximative components using harmonic analysis. After that, data recovery is done. The residual component is a distortion element, while the primary value is used as the approximation component. An arbitrary value is determined after calculating the standard deviation and mean of the residual component. After that, missing values are filled in using the distortions or the total of the assumption components. The algorithm might have eventually rebuilt the data. Once missing data is filled and outliers are found, the cleansed data will move to normalisation.

Data normalisation

Data normalisation is a preliminary processing technique that transforms characteristics into a standard range, preventing larger values from dominating lesser ones. The primary goal is to reduce the bias of characteristics that have a bigger numerical impact in distinguishing pattern classes. If the relative significance of the characteristics is not known, every attribute in the data is given identical weight when determining the final class of an unexpected sample. Statistical learning methods benefit from the equal contribution of all data characteristics. Though data normalisation assures that all characteristics provide an equivalent numerical impact, this does not indicate that they are identically relevant in the classification result. Certain data aspects may be important to varied degrees, whereas others may be completely useless or superfluous (Singh & Singh, 2020). There are several normalisation approaches such as mean and standard deviation based normalization methods, minimum–maximum value based normalization methods, decimal scaling normalization (DSN), median and median absolute deviation normalization (MMADN), tanh based normalization (TN), and sigmoidal normalization (Singh & Singh, 2020).

Optimal feature selection

Feature selection (FS) is a component of feature weighting, yet both methods vary in terms of feature significance. The existence of undesirable features hampers learning and expands the feature space. These characteristics conflict with valuable features, confusing the learning process and reducing the accuracy of classification. It also raises the processing efficiency of ML algorithms, which is determined by the number of features and occurrences in the training data. Whenever the importance of characteristics varies, FS operates superior to data weighting (Singh & Singh, 2020). FS looks for selecting useful features while rejecting inappropriate and superfluous ones. Whereas, feature weighting relies on the idea of feature significance, assigning weight to every feature based on its significance. Therefore, the weight for undesirable traits is expected to be zero, although it may range between greater to lower for different features. As FS relies on the concept of choosing the most efficient sample within a set of features, redundant and insignificant ones are deleted from the entire collection since they impair or fail to impact the learning process. This may alternatively be considered as feature weighting, with weights assigned to binary values (Singh & Singh, 2020). It is a stochastic optimisation problem that requires an optimum collection of features to get accurate classification outcomes. Many approaches are currently developed for determining the optimum subset, which is divided into the following three groups: filter, wrapper, and hybrid. These approaches are discussed in detail in the following sections. A few approaches that have been developed for determining subsets of features are mutual information, value-difference metric, iterative RELIEF, cross-category feature importance, particle swarm optimisation, differential evolution, teacher learner-based optimisation, and ant colony optimisation (Singh & Singh, 2020). A generic framework may be derived from most techniques for selecting features. Feature generation, feature assessment, terminating condition, and result validation are its four fundamental processes (see Fig. 2) (Dhal & Azad, 2021). Until an ending requirement is fulfilled, the approach for choosing the variables, generate a subclass and assess it (Xue, Yao & Wu, 2018). The classification method then uses actual data to verify the subset that was discovered.

Decision modelling

Decision—making process in PdM pertains to the stage that is initiated by sensors-enabled, dynamic prediction to produce early recommendations about planned repair and methods that minimise or lessen the impact of the predicted malfunction. As an outcome of I4.0, for PdM, which enables short-time decision-making. It is therefore feasible to consider the P-F interval, or the duration between the event occurring of a likely breakdown and the point at which it transforms into a mechanical failure, as a door of possibility in which decision-making techniques can suggest actions meant to avoid the predicted operational malfunction or decrease its impacts. A variety of approaches and strategies that attempt to improve decision-making have been developed as a result of the manufacturing environment’s sophistication and unpredictable nature (Ruschel, Santos & Loures, 2017). Intelligent decision-making is one of I4.0’s core aspects, and it is made possible by analysing information from sensors (Sharma et al., 2021).

Nevertheless, because of the ambiguity in predictive analytics, the deterioration operation, and the time constraints that must be met, the application of the decision-making techniques is complicated. As time has progressed, maintenance management has adopted the use of PdM as a leading method, there has been an increase in research in methods created to better assist maintenance decisions that prolong machinery operability.

PdM and ML models

RUL and anomaly identification are two management tools for asset health for which the PHM system is currently a secure solution. It is accomplished through the methodical application of recent advancements in IT and AI technology (Nelson & Culp, 2022). Based on the outcomes of the prediction, the emerging problems that could result in catastrophic breakdowns can be rightly predicted, and suitable procedures can be made in order to prevent these flaws (Nelson & Culp, 2022). Industrial machinery can be changed or fixed before a flaw appears, returning the machine to its initial state until every servicing task is finished. A machine, component, system, or other piece of equipment’s status can also always be evaluated. It is possible to achieve efficiency with minimal downtime by anticipating and planning for periodic outages (Paul, Biswas & Mukherjee, 2022). Leveraging predictive data to precisely arrange upcoming maintenance operations is the major emphasis of PdM (Zonta et al., 2022). The PdM modelling process essentially entails four interconnected stages: (i) continuous deterioration analysis; (ii) service effects modelling; (iii) maintenance policies formulation; and (iv) performance assessment.

Imperative AI algorithms for PdM

AI algorithms play a key role in the prognostics of the equipment in a manufacturing I4.0. The vast amount of data created by various IoT devices in the industry can be utilised to analyse and gain meaningful insights on the RUL of equipment, anomalies identified, strategies to avoid unplanned downtime, reduce planned downtime, how maintenance resources can be optimised, and many more. Figure 3 depicts a few of the ML models and deep learning models. The models are segregated based on i) supervised learning ii) unsupervised learning iii) semi-supervised learning. The models are further classified as Classification and Regression models of supervised learning, Clustering and Dimensionality Reduction of unsupervised learning, and semi-supervised techniques. In recent years, ML methods have been extensively used in many different academic domains. Choosing an appropriate and simple algorithm that is effective is vital. Massive volumes of data from breakdown circumstances and health conditions instances must often be collected for training the ML systems. In recent years, ML and deep learning techniques have been vital tools for researchers to work extensively on PdM.

Clustering

Data analysis constitutes an unsupervised learning process that extracts observations and concealed associations from data. Data analysis is increasingly important in data science and ML since it reveals all concealed information and facilitates the determination of various characteristics of the data collection. Clustering is a data analysis approach that groups data according to similarity measurements. A cluster consists of comparable items or data points. Likewise, objects or data points in different clusters will be identical. However, when these clusters are examined together, they show significant differences. Clustering is regarded as the most essential unsupervised learning approach since it involves identifying patterns in a set of unlabelled data (Ali et al., 2020).

Density-based spatial clustering (DBSCAN): An approach for grouping spatial datasets with different densities and shapes is called DBSCAN. Depending on the density of their neighbourhoods, DBSCAN divides data points into three categories: noise, border points, and core points. Clusters are formed by repetitively growing from core points, which have enough density of neighbours along a certain radius cluster (Yang et al., 2022). Although noise points do not constitute any cluster, border points though located close to core points are also not considered as core points. As part of the algorithm’s operation, the seed point is initialized, core points are found, and clusters are expanded recursively. The algorithm’s sensitivity may be adjusted by users using the parameters Eps and MinPts (a user-defined threshold). DBSCAN is good at finding clusters of any shape, resilient to outliers, and does not need to provide the number of clusters in advance. Nevertheless, it may have trouble processing high-dimensional data or clusters with different densities, and its ability to perform might be dependent on parameter selections. Many academics have worked to enhance DBSCAN (Wang et al., 2019) throughout time to use it more successfully. They have done this by using meta-heuristic algorithms (Jian & Zhu, 2021; Agarwal, Mehta & Abraham, 2021; Zhang et al., 2021; Singh, Singh & Kaur, 2021) to ensure that the EPS and MinPts parameters in DBSCAN are searched and chosen automatically. An approach for multi-segment optimisation was presented in Lai et al. (2019). It can swiftly pick the right EPS parameter and has a strong optimisation potential as an individual variable updating technique. It can also acquire good DBSCAN accuracy. In order to address the clustering problem, the authors in Jian, Li & Yu (2021) suggested an adaptive DBSCAN. This is done by using the target solution and its motion range as noise locations, where the neighbourhood is impacted by physical elements. By using the harmony search optimisation technique on DBSCAN, in Zhu, Tang & Elahi (2021) researcher was able to improve both the clustering parameters and the clustering outcomes. KR-DBSCAN, a density-based clustering technique employing space and reverse closest neighbour, was introduced in Hu et al. (2021).

Hierarchical agglomerative clustering (HAC): One of the popular techniques for unsupervised learning is HAC. The performance of HAC is significantly impacted by the linking criterion that this approach uses to determine how similar both clusters are. It is a popular bottom-up clustering technique that begins by creating an initial segmentation and unites the most identical clusters until a single cluster comprising every item is created. The appropriate use of HAC is seen when the number of clusters is unknown. It generates a dendrogram that illustrates the relationships between the clusters (Randriamihamison, Vialaneix & Neuvial, 2021). The primary flaw in HAC is the linkage criteria, which specifies how the relationship between clusters is determined based on the separations between pairs of components. Single, average, and complete linkage are the three primary linkage criterions in HAC. Each criterion has unique properties and often yields segments with varying attributes. Analysing the distance between two clusters’ closest members, single linkage calculates the degree of similarity the two clusters are to one another. It is straightforward and computationally effective; however, a chaining issue may cause extended clusters. Although average linkage provides a more intricate definition of similarity, it calculates the average proximity between every pair of points between each cluster to estimate their resemblance (Emmendorfer & de Paula Canuto, 2021). Finally, complete linkage is more sensitive to noise and outliers in the data as it determines how similar two clusters are based on how far apart their most distant members are from one another. Numerous studies have been conducted on hierarchical agglomerative clustering. The work in Jianfu, Jianshuang & Huaiqing (2011) offers the HACNJ method, a hierarchical clustering algorithm based on the Q-criterion. When working with big datasets, HACNJ’s time complexity of O(n3) and space complexity of O(n2) is prohibitive, like that of simple hierarchical clustering. A thorough analysis of a few enhanced hierarchical clustering techniques is provided in Rani & Rohil (2013). The goal of these methods is to get over the drawbacks of simple hierarchical clustering techniques. Clustering Using REpresentatives (CURE), Balanced Iterative Reducing and Clustering using Hierarchies (BIRCH), ROCK (RObust Clustering using linKs), CHEMELEON Algorithm, Linkage techniques, Leaders—Subleaders Leaders are some of the techniques that are described.

K-means: This unsupervised clustering technique divides a dataset into clusters according to similarities. The total squared distance between every data point and its designated centroid is what it attempts to minimize. K-means functions exceptionally well on massive datasets and has a fast clustering speed, but it has low clustering consistency and is prone to solitary data and noise. The k-means clustering algorithm has been extensively researched in the literature, with several adaptations, and utilised in numerous academic domains (Alhawarat & Hegazi, 2018; Meng et al., 2018; Lv et al., 2019; Zhu et al., 2019). Nevertheless, initializations often have an impact on these k-means clustering methods, thus a predetermined number of clusters must be provided. Validity indicators can be applied in this situation to determine a cluster number which is expected to be unhindered by clustering techniques (Halkidi, Batistakis & Vazirgiannis, 2001). The literature identified a number of cluster validity indicators for the k-means clustering technique, including the Bayesian information criterion (BIC) (Kass & Raftery, 1995), the Akaike information criterion (AIC) (Bozdogan, 1987), Dunn’s index (Dunn, 1973), the Davies-Bouldin index (DB) (Davies & Bouldin, 1979), the Silhouette Width (SW) (Rousseeuw, 1987), and the Calinski and Harabasz index (CH) (Calinski & Harabasz, 1974), Gap Statistic (Tibshirani, Walther & Hastie, 2001), generalized Dunn’s index (DNg) (Pal & Biswas, 1997), and modified Dunn’s index (DNs) (Nejc, 2012). The two main kinds of clustering validity indicators are internal indicators and external indicators (Rendón et al., 2011). Through analysing the cluster memberships generated by the clustering procedure with already available information, such as an explicitly provided class label, external indicators are applied to analyse clustering outcomes (Lei et al., 2017; Wu et al., 2009). Internal indicators, on the other hand, assess the quality of cluster topology by emphasizing the inherent information included in the data (Jegatha Deborah, Baskaran & Kannan, 2010). The most prevalent and often utilised approach for determining the number of clusters is X-means (Pelleg & Moore, 2000). This approach has been employed in previous research by authors like Witten et al. (2000) and Guo et al. (2017). Pelleg & Moore (2000) expanded k-means in X-means by dividing itself and locally deciding cluster centres in every iteration of k-means to achieve improved clustering. After which, a range of cluster numbers within which the genuine cluster number falls is indicated. The splitting procedure is then carried out using a cluster validity indicator, such as BIC.

Fuzzy C-means (FCM): A robust unsupervised technique for data analysis and developing models is fuzzy clustering. FCM algorithm (Ruspini, Bezdek & Keller, 2019), which is a fuzzy clustering technique, is a very effective algorithm for processing data and generating rules from a dataset that has a high frequency of fuzzy features. Fuzzy clustering can seem more intuitive than hard clustering in a variety of scenarios. Instead of being required to completely conform to one of the classes, objects that are on the borders of many classes are given membership degrees between 0 and 1, which indicate their selective membership. The most commonly adopted approach is FCM. In 1974, (Ruspini, Bezdek & Keller, 2019) published the first study on FCM clustering in the literature for a particular example (m = 2). Fuzzy partitioning is used by the FCM to allow a data unit to be assigned to any cluster with a membership grade that ranges from 0 to 1. This method uses the distance from the cluster centre to the data point as the factor to determine membership for each data point that corresponds to that cluster centre. The closer the data is to the cluster centre, the more affiliated it is with that specific cluster centre. The total membership of all the data points must correspond to one. Certain parts of data in real-world issues could be incomplete or contain ambiguous information. Fuzzy set theory, in conjunction with the membership function notion defined in the interval [0, 1], offers a suitable means of expressing and handling these kinds of data components. Every component in this concept has the potential to be associated to several clusters. In an effort to get around some of the algorithm’s drawbacks and enhance its clustering capabilities in various scenarios, a number of novel techniques built on the FCM algorithm were released. A summary of these new approaches’ performance may be found in Jie et al. (2017).

Dimensionality reduction

The process of converting high-dimensional data into lower dimensionality requires dimensionality reduction (Ayesha, Hanif & Talib, 2020). Many dimensionality reduction approaches have been applied in the recent couple of years to sort the dataset under considered data samples. In order to reduce dimensionality, high dimensionality input must be transferred to lower dimensionality inputs, resulting in the visualisation of comparable points in the input field to nearby points on the main field. The benefits of using this approach on a given dataset are reduced data storage capacity because of the reduced number of dimensions, increased computation speed, elimination of redundant, noisy, and insignificant data, enhanced data quality, increase in the accuracy and efficiency of an algorithm, and provide data visualization. In general, there are two primary categories of dimensionality reduction techniques: feature extraction (FE) and FS. FS is regarded as a crucial technique because data is generated at a rapid pace. By using FS, certain major dimensionality issues can be reduced, including redundancy and insufficient information removal. The process of feature engineering is a crucial pre-processing stage that aids in the extraction of modified features from the raw data, which serves to enhance the quality of a ML algorithm’s output and simplify the ML model. FE also tackles the problem of identifying the most distinct, useful, and concise collection of features to enhance the effectiveness of data handling and storage.

Principal component analysis (PCA): While there are other dimensionality reduction solutions, PCA is the most implemented. Similar to wavelet decomposition or Fourier analysis, this provides the advantage of evaluating results between datasets and determining the relative significance of the features. It is one of the unsupervised learning methods that lower the dimensionality of data. Feature reduction from enormous data sets into smaller features that hold significant information is a common application of this technology. It streamlines complicated datasets, increases data interpretation, and boosts computing speed. By dividing the data into low-rank and sparse components, robust PCA allows it to manage noise and outliers in the data. Additionally, it can adjust to flowing data using live PCA, enabling adaptive dimensionality reduction and real-time analysis. Exploratory data analysis is aided by PCA’s ability to visualise high-dimensional data in a lower-dimensional environment. Furthermore, PCA can recognise and resolve multicollinearity problems in datasets, yielding a more autonomous component set. Nevertheless, PCA needs interpretation to determine the optimum number of components to maintain, presupposes linearity, which may result in loss of data. PCA and linear discriminant analysis (LDA), were examined in Reddy et al. (2020), where the authors implemented it on the dataset available in the UCI ML repository. Features were scaled down from 36 dependent components to 26 by preserving 95% of the dataset. It was noted that PCA classifiers outperform LDA classifiers. In Saraswathi & Gupta (2019), using a random forest (RF) classifier, the authors developed a multi-class tumour diagnosis strategy. They also gave a comparative study of the RF-PCA, RF, and RF-PCA approaches with random selection strategies. The experimental findings demonstrate that RF-PCA’s random selection strategy yields higher accuracy compared to alternative approaches. Furthermore, in Jamal et al. (2018) the authors have concentrated on how feature extraction might lower the quantity of characteristics needed for the categorization of breast cancer from the initial White Blood Cell (WBC) data set. The dimensionality reduction achieved with the K-means cluster is nearly as effective as PCA, according to the metric assessment. However, in order to eliminate unnecessary features and keep the best attribute subset, Salo, Nassif & Essex (2019) suggests a unique hybrid approach that combines PCA with Information Gain (IG). PCA efficiency was compared with Isomap, a deep autoencoder, and a variational autoencoder. The robustness of the suggested technique produced encouraging results in both NSL-KDD and Kyoto 2006+ datasets. MNIST, Fashion-MNIST, and CIFAR-10 are three popular image datasets that were used in the experiments. Compared to its neural network counterparts, PCA’s computation time was a couple of orders of magnitude quicker. A suitably wide dimension was possessed by the two auto-encoders (Fournier & Aloise, 2019).

T-distributed stochastic neighbour embedding (t-SNE): t-SNE, created by Hinton & Roweis (2003), is one of the nonlinear dimension reduction techniques that has obtained a lot of research interest lately. It has been used for a variety of application areas, including remote sensing images, computational fluid dynamics, single-cell RNA-sequencing data, microbiome data, and computational fluid dynamics. The primary goal of this technique is to create probability distributions from paired distances in which greater distances are associated with lower probabilities and vice versa. This is the most popular learning technique for single-cell analysis. Multidimensional data sets might be effectively projected onto a 2D or 3D plane using the t-SNE technique, maintaining the majority of the local architecture of the data in the original high-dimensional environment. Nevertheless, the t-SNE approach is rarely used for classification or regression tasks as it lacks an integrated mechanism to translate incoming data points to the appropriate low-dimensional form Maaten (2009). By implementing neural networks for feature extraction and subsequently performing classification over the projected low-dimensional space from t-SNE, several researchers have taken steps to address the out-of-sample extension challenge (Maaten, 2009; Oliveira, Machad & Andrade, 2018). For t-SNE to operate well, an array of parameters must be adjusted. One such parameter is the perplexity per, which is a smooth measure of the effective number of neighbours. According to certain research, this parameter often has a value ranging from 5 and 50 (Van der Maaten & Hinton, 2008). In actuality, accurate tuning of per necessitates practical knowledge and an understanding of the internal workings of the t-SNE approach.

Autoencoders: Autoencoder is an artificial neural network, through unsupervised learning, can acquire new information encodings (Charte et al., 2018). Through a number of hidden layers, AEs are taught to acquire an internal representation that enables replication of the input into the output. In order to use the original information in the hidden layers for other activities, it is intended to be produced in a more compact representation. Due to their qualities and effectiveness, AEs are commonly employed for dimensionality reduction tasks (Rosenberg, Hebert & Schneiderman, 2005; Dópido et al., 2013; Leistner et al., 2009; Lee, 2013). An AE’s fundamental structure is identical to a multilayer perceptron’s. In AE the data constantly flows in one direction as it functions as a feedforward neural network with no cycles. A sequence of layers, comprising an input layer, many hidden layers, and an output layer—each layer’s units coupled to those in the subsequent layer—usually make up an AE. Since the primary objective of AEs is to replicate the input into the output at every step of the learning process, the output and input both have an identical array of nodes (Charte et al., 2018). The encoder and the decoder are the two parts that make up the AE. The input layer and the first segment of the hidden layer constitute the first portion, while the output layer and the other half of the hidden layer form the second. It is possible to distinguish between two types of AEs based on the quantity of units present in the concealed layers. Those with fewer units in their hidden layer than in their input and output layers are referred to as undercomplete AE. Its primary goal is to have the network learn a concise form of the input to derive new, more advanced features. Those with more units in their hidden layer than in their input and output layers are referred as overcomplete AE. When obtaining an expanded form of the input, it is required to employ additional methods to avoid a major issue where the network duplicates the input to the output without learning anything beneficial. Furthermore, AEs are a great instrument for creating a new input space that is lower dimensional and composed of features at a higher level.

Semi supervised ML algorithms in PdM

Previously, there were two main areas of ML: supervised and unsupervised learning. In supervised learning, a series of data points having an input value and an output value are shown to the learner from which a classifier or regressor model is built that is capable of estimating the output value for a set of inputs that haven’t been seen before. On the contrary, no precise output value is given in unsupervised learning. Rather, an attempt is made to deduce a basic pattern from the data. The aim of unsupervised clustering is to determine an association between the supplied inputs to categories so that comparable inputs correspond to the identical category. A subfield of ML called semi-supervised learning seeks to integrate these two objectives. Semi-supervised learning algorithms often use data that usually corresponds with one of these two areas in an effort to increase efficiency in that area. For example, more data points for which the label is uncertain may be utilised to help with the classification process while solving a classification issue. However, for clustering algorithms, the fact that some data points are part of the same class may help the learning process. The main objective of semi-supervised learning is to leverage unlabelled data to create more effective learning processes. It turns out that this isn’t always feasible nor is it easy. Unlabelled data is only valuable if it contains information valuable for label prediction which is either absent from the labelled data or difficult to derive from it. The algorithm then has to be capable to retrieve this data in order to for any semi-supervised learning approaches to be implemented.

Self-training: Self-training models are a subset of semi-supervised learning techniques that train repeatedly with labelled and pseudo-labelled data using several supervised classifiers. Unlabelled data points that have labels applied to them by the classifiers using the predictability of those labels are known as pseudo-labelled data. Training and pseudo-labelling are the two distinct stages that make up the self-training process. The labelled data as well as potentially pseudo-labelled data from earlier iterations are used to train the classifiers during the training phase. The most reliable recommendations will be applied to the labelled data set for the following iteration after the classifiers are employed to determine labels for the unlabelled data points in the pseudo-labelling stage. Self-training can be applied and modified in many ways, including object identification (Chen et al., 2022) and hyperspectral picture classification (Li et al., 2023). Important design choices in self-training include choosing which data to pseudo-label, reusing pseudo-labelled data, and establishing ending conditions. Assuming accurate probabilistic predictions are provided, the procedure may resemble the expectation-maximization (EM) method. Non-probabilistic algorithms may require modifications, such as enhancing prediction probability estimations or through local distance-based metrics. In de Vries & Thierens (2021) the use of self-training in random forests is investigated and a termination criterion centred around out-of-bag error is provided. Furthermore, in Chen et al. (2022) a progressive method known as the pseudo-label technique is discussed in which data points are pseudo-labelled during the training phase. This method is specifically applicable to neural networks.

Semi-supervised support vector machine (S3VM): The volume of data collected globally across multiple technologies has increased dramatically in the past few years. Over this period, a plethora of technologies and methods for effectively gathering this information have been established. However, there are a lot of unlabelled data chunks in this acquisition, so labelling them all will take a lot of effort and time. In order to improve learning behaviour, the S3VM approach was explored in conjunction with support vector machines (SVMs), as semi-supervised learning techniques often focus on mixing labelled and unlabelled input (Calma, Reitmaier & Sick, 2018). ML techniques built around statistical learning theory are known as SVMs. They prove useful in several areas, including generalisation, simplicity of the solution, global optimisation, and an established theoretical basis. They are proven to be useful in real-world engineering domains. One constraint of SVM is that it is best used with supervised learning and, thus, only on labelled data. A learner model that can function well with little labelled data is quite uncommon. Because of this, semi-supervised learning may constitute a useful technique to use with SVM in situations when labelled data is scarce but adequate unlabelled data exists (Devgan, Malik & Sharma, 2020). Finding the best classification hyperplane that meets classification objectives is the primary goal of SVM. An S3VM’s concept is to construct a single, fundamental SVM for each labelling before selecting the SVM with the largest margin. These are created using the same methodology as SVM but with a few minor parameter adjustments and the addition of a penalty term to the objective function. Although SVM’s aim is convex, the semi-supervised SVM’s objective is non-convex and predominantly hat-shaped. Investigation of various S3VM optimisation methods may prove quite rewarding. Several well-known applications are Branch and Bound, SDP convex relations, deterministic annealing, SVM^light, ▿S3VM, continuation S3VM, and CCCP. S3VMs have several applications where they can be useful. These hold true anywhere that SVMs are utilised, thereby continuously enhancing SVM performance. Because it is based on a well-defined mathematical structure, it operates more quickly and requires less computing time. Since S3VMs utilise unlabelled data, they function more effectively than SVMs, which has made them increasingly prevalent in recent times. Even while they have been useful, there is a chance that they will lead to even worse performance than if you simply used labelled data. To prevent these types of issues, just those cases that are anticipated to be beneficial must be chosen, avoiding the cases that might harm the system’s efficiency in general, rather than taking advantage of all the unlabelled data. These issues can be simplified using one of the two methods: S3VM-c or S3VM-p. Unlabelled data may prove useful when component density sets are discernible or observable, which is the only situation in which S3VM-c operates. This involves computing the label and confidence for every cluster in SVM and S3VM. Conversely, S3VM-p primarily relies on confidence estimates in graph approaches, such as those where confidence may be expressed as the threat involved in unlabelled cases. In each of these scenarios, the S3VM prediction findings are selected if they exhibit a high degree of confidence in the label propagation bias; if not, the SVM prediction is employed. Adopting S3VM-c and S3VM-p might lessen the likelihood of performance loss nevertheless are certain drawbacks to its use.

Generative adversarial networks: Generative adversarial networks (GANs) are a new kind of learning framework that was most recently presented. They operate on the principle of concurrently building generative and discriminative learners (Goodfellow et al., 2020). This method, which is typically applied with neural networks, trains a discriminative classifier to determine if a given data point is “real” or “fake” (i.e., artificially generated) at the same time as a generative model, which is tasked with producing data points that are challenging to differentiate from actual data. Two neural networks, a discriminator, and a generator, participate in a competitive training procedure to form GANs. The generator creates synthetic data using random noise as input with the goal of producing samples that are identical to actual data. The discriminator continuously adapts to better separate real from synthetic data while it is being trained to do so concurrently. The discriminator gains proficiency in identifying authenticity, while the generator improves its capacity to produce realistic data through recurrent adversarial training. Stability is reached as an outcome of the above dynamic process, where the discriminator has a 50% probability of accurately categorising the outstanding synthetic data that the generator provides. Using a game-theoretic methodology, GANs enhance efficiency across the discriminator and generator through adversaries (Sreevallabh Chivukula et al., 2022). Their learning is guided by their desired function; the discriminator’s goal is to correctly categorise synthetic and real data, while the generator aims to maximise the possibility of deceiving the discriminator. After being trained, GANs may produce new, genuine samples that closely resemble the training set. This makes them useful for a variety of tasks, including data augmentation, picture production, and style transfer.

Ensemble models

Support vector machine: Regression and classification challenges are handled using this ML model. Support vector regression is a variant of SVM’s fundamentals to predict continuous values. SVR aims to minimize the margin of error and determine the optimum hyperplane to match the data. The SVR algorithm is designed to minimise both the level of sophistication of the model and the empirical risk, which is defined as the difference between the values that are predicted and those that occur. To accomplish this, a margin of tolerance (ε) is introduced, which penalizes data points that exceed a particular error margin while permitting certain data points to remain within it. To enable SVR to process non-linear connections, the kernel technique is frequently utilised to translate data into a higher-dimensional feature space (Berk & Berk, 2020). Polynomial kernels and the Radial Basis Function (RBF) are examples of popular kernel functions. The efficiency of SVR is influenced by the kernel function selection and its hyperparameters. SVR’s interpretability, scalability, and ability to handle big datasets have all been the subject of recent studies. An optimization issue is addressed, usually using gradient-based approaches or sequential minimal optimization (SMO) techniques, to train an SVR model. SVR is used by Lee et al. (2020) to anticipate dependability in engine components, where he suggests an innovative method that uses legitimate genetic algorithms to find SVR’s best parameters, and then uses those values to build SVR models. Non-linear regression and time series issues have been successfully resolved using SVMs (Sexton et al., 2017; Chang & Lin, 2011). In contrast to conventional neural network models, which employ the empirical risk minimization principle, SVMs employ the structural risk minimization principle, which aims to reduce an upper bound on the generalisation error rather than the training error. According to this theory, SVMs are more generalizable than conventional neural networks. In various disciplines, SVMs have been used to solve predicting issues. SVMs have, however, seldom been used to predict reliability. Malhotra et al. (2021)’s study represents the preliminary approach to using an SVM model to forecast engine reliability, with the goal of assessing the viability of SVMs in reliability predicting. SVMs are used by Chou & Truong (2019) to evaluate software dependability with simulated annealing algorithms (SVMSA), which yields more accurate forecasts compared to the different techniques. In comparison to ANN-based and current SVM-based models, Chou & Truong (2019)’s suggested SVM-based software reliability prediction model may attain a stronger prediction precision.

Decision tree: A non-linear supervised learning approach called decision tree regression is utilised for both regression and classification problems. Decision trees are used in regression to predict the target variable by iteratively dividing the dataset into subsets and making decisions at all nodes. This is done depending on the input attributes. Averaging all the target values at the leaf nodes is the eventual estimation (in the case of regression). Decision trees render insight on the significance of features, which aids in locating important elements that lead to breakdowns in equipment. Its interpretability is essential for comprehending the process of making decisions. Based on the collected historical data of the equipment’s operation and performance from sensor readings, maintenance logs, and any other appropriate data, the model provides inferences on the machine’s condition and degradation. The data may show anomalous patterns that point to malfunctions or upcoming breakdowns. The model can innately identify the non-linear relationships and categories them as complex degradation patterns, thus indicating possible problems with equipment or machinery. Mean absolute error (MAE) or root square error (RMSE) metrics can be used to estimate how accurate the prediction of the model is.

Random forest: It is a ML algorithm designed by Breiman (2001) designed RF, which can be used for regression and classification tasks. It is an ensemble learning method that constructs a multitude of decision trees at training time and outputs the class that is the mode of the classes (classification) or mean prediction (regression) of the individual trees. Reducing overfitting and increasing prediction accuracy are two advantages of this ensemble technique. Random subsets of the data and features are used to train each decision tree in the ensemble. Because of this unpredictability, the model is more resilient to noise and data outliers, and the correlation between the trees is decreased. The average of all the trees in the ensemble’s forecasts yields the final forecast. Every RF decision tree is trained using a bootstrapped subset of the initial training data, which implies that each tree sees marginally distinct data points, in order to minimize overfitting. Furthermore, RF only takes into account a random subset of characteristics at every split when dividing nodes in decision trees. This decreases the possibility of overfitting and further extends the trees. The generalisation inaccuracy for forests converges as the number of trees in the forest rises. The RF has significant additional advantages. Predicting the state of a robust automated facility uses ML techniques like linear regression, RF, and symbolic regression (SR) (Zenisek, Holzinger & Affenzeller, 2019). They put forth a strategy for identifying and forecasting idea drifts—drifting behaviour—in uninterrupted data feeds. A scenario-based analysis on manufacturing radial fans was also given. Approaches, drift detection, and prediction showed accurate outcomes, according to the outcomes produced using the simulated data. Additionally, researchers located at the strained radial fan revealed that the congestion detection technique was successfully used based on the actual research that was carried out. Although the analysis results were encouraging, the forecasting of notion variations presently depends on the assumptions and cannot be assessed due to the absence of ongoing degradation data.

For continuous conditioning and problem detection in rotating machinery, Janssens, Loccufier & Van Hoecke (2019) developed a multi-sensor system that combines vibration measures in addition to infrared thermal imaging data. Prototype characteristics are created using vibration measurements, while statistical characteristics are generated using infrared thermal imaging data, using a technique called feature synthesis. For practical defect identification, the collected characteristics are then pooled and given to RF classifiers. Researchers have shown in the investigation that combining both types of sensor information paves the way for more precise measurement of a wide range of cases and defects than when using a single sensor feed. A learning method that can accurately identify and analyse heterogeneous datasets of turbofan engines was created by Lacaille & Rabenoro (2018). The proposed model employs a vast proportion of pre-trained and statistical tests on the dataset, and it is capable of choosing effective sets of tests having pre-identification rates of more than 85%. A new technique to use RF to identify damaged rotor bar breakdown in a Line Start-Permanent Magnet Synchronous Motor (LS-PMSM) was put forth by Quiroz et al. (2018). A working motor and a damaged motor were used to collect the instantaneous current signal while setting up the motor. Thirteen distinct analytical time-domain features were employed to extract features for the model’s training, and later all these features were utilised to identify the status of the motor when it is running properly or improperly. To choose only a few features from the RF, they took into account the relevance of each function. According to the findings, RF accurately classifies motor disorders as healthy or inadequate with a 98.8% confidence when using all characteristics and a 98.4% accuracy when employing only the mean index and impulsion features. The generated framework was compared to various well-known ML techniques, such as decision tree, naive Bayes classifier (NBC), linear ridge, and SVM. The RF regularly surpasses these methods, outperforming them with greater precision. The recommended approach can be employed in the industry for continuous monitoring and fault diagnosis of LS-PMSM machines, and the results may be useful for creating preventative maintenance schedules in manufacturing facilities.

By examining the historical information of aviation safety systems, Yan & Zhou (2017) postulated a prediction model using Term Frequency-Inverse Document Frequency (TF-IDF). He also stated that RF can estimate failures of great specificity well ahead of time, and corrective actions could be conducted based on the model’s predictive accuracy. The previous aircraft in a row have used TF-IDF to generate the attributes from the unprocessed data. The developed RF model’s classification of the problems took into account various priorities. As a result of the dataset’s extreme disparity, the receiver operating characteristic (ROC) curve was selected as a benchmarking tool. The proposed methodology achieves the highest true positive value of 100% and the least false positive rate of 0.13% when matched to some of the other approaches. The suggested technique obtains a true positive rate of 66.67% and a false positive rate of 0.13% for the test sample.

Gradient boosting machine: This model is one of the ensemble ML techniques for regression and classification problems. It builds a series of weak learners sequentially, which are basically decision trees that have a small number of nodes and are usually shallow. At every stage, the model aims to rectify the error or the residuals of the previous one. Every data I assigned with a weight in this model. To maximize the overall efficiency of the approach, gradient boosting regression fits a new model to the errors of the earlier model. The learning rate is one of the hyperparameters that manage the impact of each weak learner until the final prediction. The model is more resilient when the learning rate is lower, nevertheless, it needs more trees for a convergent result. The weighted aggregate of all the estimates made by the weak learners is the ultimate prediction (Aziz et al., 2020). The metrics used for the Decision Tree can be considered for evaluating the efficiency of the Gradient Boosting model.

Light gradient boosting machine: An ensemble ML technique called Light GBM serves to address challenges with regression as well as classifications. Despite this, a boosting family including GBM and Adaboost is nevertheless employed since it is important to generate effective classifiers from the weak ones. Several researchers have begun to build and advance toward Light GBM and its variants due to the obvious benefits of focusing on data weighting (Aziz et al., 2020). Researchers have used Light GBM extensively to attain state-of-the-art outcomes on a variety of machine-learning studies, including predicting the types of breakdowns of reinforced concrete panels (Hoon et al., 2018), identifying malfunctions in the robot sectors (Liang et al., 2019; Tang et al., 2020), and accurately classifying airline data (Ye et al., 2019), home credit datasets, and financial sectors (Aziz et al., 2020). The majority of the researchers use learning rate, number of leaves, depth, and tree count to create the majority of the hyperparameters for Light GBM. The accurate tailoring of hyperparameters to reduce overfitting is crucial for Light GBM’s accuracy in predictions. In contrast to the GBM family of boosting algorithms, which includes Adaboost, Light GBM possesses unique advantages. It offers additional benefits, such as distributed learning, improved training efficiency, and support for parallel learning to handle massive data sets (Aziz et al., 2020). Both GBM and Light GBM are ML techniques that are frequently employed in the field of prediction. For example, they are used to identify the seismic failure mode of concrete shear walls (Song et al., 2019) and to forecast bubble points (Dev & Eden, 2019) in order to get valuable insights specific to the field. Li et al. (2018) presents an innovative method that uses the Light Gradient Boosting Machine (LightGBM) algorithm to estimate the RUL of aviation engines. Using actual aircraft engine data, the study reviews the suggested method and finds that it performs better than current RUL estimate techniques in terms of accuracy and computing efficiency.

Extreme gradient boost: This model is categorized among boosting techniques, which aim to enhance the efficiency of the model by creating strong classifiers from weak classifier combinations (Aziz et al., 2020; de Oliveira, 2019). Benefiting from an effective and refined version of the three boosting techniques, Adaboost, GBM, and Light GBM, XGBoost produced excellent outcomes across multiple benchmark dataset classifications, including the credit scoring domain (Daoud, 2019), biochemical index prediction (de Oliveira, 2019), and genome segment-based breast cancer prediction (Xia et al., 2017). The basic principle of this approach is to expand the size of decision trees by optimizing them by applying gradient descent when new decision trees are included for training. For regression problems, predictions are combined with the weighted sum, and for classification, with the majority voting. All the algorithms covered in this study under the ML algorithms have been represented in the form of a chart in Fig. 4.

Condition monitoring techniques in PdM

The primary uses of AI in maintenance schedules are fault diagnostics and anomaly identification. The cause of a failure can be determined through fault diagnostics. Only the frequency of a failure can be determined by anomaly detection, but it provides benefits for dataset production. The above viewpoints are used in this segment to examine the literature.