Energy optimization for wireless sensor network using minimum redundancy maximum relevance feature selection and classification techniques

The proposed system

In this study, we propose a smart model to optimize energy consumption in WSNs. In the first stage; we apply the MRMR algorithm as a classification algorithm that is used to select the sensors from the most to the least significant. Here, we assume that each feature represents a sensor in WSN; reducing the number of sensors is equivalent to reducing the number of features. Besides, reducing the number of features/sensor minimizes the energy consumed by sensors. The purpose of this process is to quickly distinguish and dismiss weak features. So, the number of inputs to the selection process will be minimized and reducing the required time for the final selection procedure. We assume that a lifetime extension factor is expressed as shown in Eq. (1) (Alwadi & Chetty, 2012): (1)${\displaystyle LT=\frac{overallnumberofsensor}{numberofutilizedsensor}.}$

Equation (1) demonstrates the relationship between the number of used features and the lifetime extension factor. If we reduce the number of used features, the lifetime extension factor will be increased. Therefore, using a suitable algorithm enables the selection of the most important features/sensors and eliminates redundant features while maintaining the same accuracy.

Feature selection is an effective technique for processing high-dimensional data and improving learning efficiency (Cai et al., 2018). FS detects the relevant features and dismisses the redundant and irrelevant elements to obtain a subset of features that accurately depict the main problem without affecting performance (Ramírez-Gallego et al., 2017). It is categorized across different standards: one of these criteria is based on the training dataset used (unlabeled, labeled, or partially labeled). Accordingly, the feature selection approaches are categorized into unsupervised, supervised, and semi-supervised feature selection, respectively. A good FS approach is evaluated according to its high accuracy and should have less computational overhead (time and space complexity) (Cai et al., 2018). In our research, we focus on the supervised feature selection algorithm MRMR.

MRMR for feature selection

The data size such as video, text, and image, has grown exponentially because of the quick development of artificial intelligence and neural networks. Moreover, due to the complexity of the data volume and its multi-dimensionality, a problem of irrelevant data and redundancy appeared. As a result, extracting useful features from the data is a hot spot. Feature selection is an essential approach that is used to reduce the dimensionality of the data. Briefly, feature selection is used to minimize the dimension of features while increasing the training speed of the model (Ren, Ren & Wu, 2022).

As we know, in any original set, the features are divided into four groups, which are: (a) weak relation and redundant features; (b) completely noisy and irrelevant features, (c) very strong relevant features, and (d) features that have weak relation and are non-redundant. Supervised FS consists of two groups: (c) and (d). Where the redundancy in features and relevance can be converted to two optimization problems: minimum redundancy and maximum relevance, a classic standard for feature selection depending on redundancy and relevance analysis MRMR (Cai et al., 2018). MRMR is a feature selection technique used to select features that have a low correlation between them and a high correlation with the output/class (Radovic et al., 2017). The MRMR approach is considered a grateful filter across the field of ML as evidenced by large citations (Ren, Ren & Wu, 2022). Initially, this approach was used to classify the DNA microarray data, which is considered a challenge in the ML felid because of the very large number of features and the low number of samples. MRMR approaches are widely used in several fields, such as the analysis of the movement of eyes and the analysis of satellite images that have multispectral and gender classification. Furthermore, MRMR is used in the preprocessing step of various high-dimensional problems, such as image and text analysis (Ramírez-Gallego et al., 2017).

The MRMR algorithm depends on the principle that it chooses a group of features (low-redundant) that have the highest correlation with the output result (max-relevance) and the least correlation between features in the main feature set.

The maximum correlation feature is represented by Eq. (2) (Ren, Ren & Wu, 2022): (2)${\displaystyle maxD\left(S,c\right),D=\frac{1}{\left|S\right|}\sum _{x_i\in S}I\left(x_i;c\right).}$

S represents the feature subset that consists of features {x_i}$.D\left(S,c\right)$represents the average value of the mutual information between the classification target c, and $I\left(x_i;c\right)$ symbolizes the ith characteristics in S.

A large amount of redundancy is created between the chosen features by max-relevance, which results in feature dependence. When a redundant has occurred between two features, the redundancy is eliminated by using Min-Redundancy, as shown in Eq. (3): (3)${\displaystyle minR\left(S\right),R=\frac{1}{{\left|S\right|}^2}\sum _{x_i,x_{j\in S}}I\left(x_i,x_j\right).}$

The MRMR algorithm in Eq. (4) is produced by combining Eqs. (2) and (3): (4)${\displaystyle max\phi \left(D,R\right),\phi =D-R.}$

Practically, if we obtain the feature subset S_m−1, we use the remaining X − S_m−1 subset to choose the m feature. If you want to maximize φ(⋅), you should improve the corresponding incremental algorithm as shown in Eq. (5) (Ren, Ren & Wu, 2022): (5)${\displaystyle \underset{x_j\in X-S_{m-1}}{max}\left[I\left(x_j;c\right)-\frac{1}{m-1}\sum _{x_{i\in S_{m-1}}}I\left(x_i;x_j\right)\right].}$

Here $I\left(x_j;c\right)$symbolizes the j ^th characteristics in X − S_m−1.

In the second stage, three classifiers are applied which are: the SVM with linear kernel classifier, the naïve Bayes classifier, and the k-nearest neighbors classifier (KNN), which are described in the next section. Classification is the most popular technique that is used in machine learning to predict the value of a categorical feature depending on the value of another attribute. In other words, classification is used to predict the new data class based on the training data. Where the classifier learns from the specific dataset and then classifies the newly observed data into several groups or classes (Priyam et al., 2013). The redundant or irrelevant features increase classification algorithm complexity, along with training and prediction time. As a result, it is very important to choose the best feature that affects the object and remove redundant and irrelevant features to increase the classifier’s efficiency.

The proposed system is shown in Fig. 1. We have two datasets inside the dataset block, which are the ionosphere and gas sensor array drift datasets. MRMR is applied in the first stage to select features/sensors from most to least significant. The second stage runs the selected classifier among three ones: SVM with a linear kernel, naïve Bayes, and KNN classifier.

Classification algorithm

• K-nearest neighbors algorithm

The primary idea of the KNN approach is to predict test data labels based on the majority rules, initially choosing the closest training samples k to the test samples. After that, predict the test sample according to the major class of the k closest training. In other words, KNN requires calculating the distance of the whole training samples for every test sample in the operation of choosing the k closest neighbors (Deng et al., 2016).

• Naïve Bayes algorithm:

The naïve Bayes algorithm is considered strongly for several reasons, such as its very easy and simple understanding, high accuracy, and speed of classification. As a classifier, it depends on Bayes’ theorem with a hypothesis of characteristic conditional independence (Ning, Junwei & Feng, 2019). It supposes that the occurrence of each feature is independent of the occurrence of other features. That is why it is called Naïve, and it is widely used in classification due to its reasonable performance (Agarwal et al., 2018).

• Support vector machine (SVM)

Support Vector Machine is a vastly utilized classification algorithm in machine learning. The main purpose of SVM is to separate the various classes in the training set by using a surface that increases the margin between them. SVM can be used to obtain powerful and precise classification results, even in the case of input data that cannot be linearly separated. It uses kernels to map the data into higher dimensional space. Firstly, SVM was suggested for binary classification, and then it is used to solve a multi-category problem because it independently learns from the dimension of the attribute space and gets a unique result. Generally, there are two types of SVM, soft margin and hard margin. Soft margin is used to classify nonlinear datasets and hard margin is used for linear data (Yalsavar et al., 2021).