Exploration of Chinese cultural communication mode based on the Internet of Things and mobile multimedia technology

ResNet50

The early algorithm based on feature extraction has a limited ability to express features, and its retrieval effect needs to be improved. The computational cost has also increased significantly with the increasing complexity of neural networks. Using ResNet can reduce the training burden, increase the depth of the neural network, speed up its convergence, and improve the speed and accuracy of searching shadow images. The residual mapping is defined as: (1)${\displaystyle F\left(x\right)=H\left(x\right)-x}$

where, x represents the input value, that is, the feature mapping of the ResNet output of the previous layer; $H\left(x\right)$ represents the observed value of ResNet; $F\left(x\right)$ represents the residual mapping. ResNet comprises stacked residual blocks, and a short circuit mechanism is used to avoid gradient disappearance and explosion. Identity mapping solves the problem of over-fitting caused by increasing network depth, and the network performance will not be reduced. According to the network depth, ResNet can be divided into ResNet 18, 34, 50, 101 and 152. Among them, ResNet 50 has an appropriate network depth, which can achieve faster speed while maintaining high accuracy. Therefore, ResNet 50 is selected as the primary network of the shadow image retrieval model.

CBAM

The single ResNet50 network lacks pertinence to the feature structure of shadow puppets, and its ability to distinguish similar shadow puppets is lacking. CBAM is a lightweight general attention module (Nie et al., 2019) for feed-forward CNN. This module obtains attention graphs from different dimensions and performs adaptive feature optimization. Channel attention assigns features according to the input feature map by knowing the importance of each part of the image, focusing on the target in space and adjusting or getting the weight. CBAM is composed of the channel and spatial attention modules arranged in series, and its calculation formula is as follows:

(2)${\displaystyle Mc\left(F\right)=\sigma \left(MLP\left(Avgpool\left(F\right)\right)\right)+MLP\left(Maxpool\left(F\right)\right)}$ (3)${\displaystyle Ms\left(F\right)=\sigma \left(f^{7\times 7}\left(Concat\left(Avgpool\left(F\right),Maxpool\left(F\right)\right)\right)\right)}$ (4)${\displaystyle F^{\prime }=MC\left(F\right)\otimes F}$ (5)${\displaystyle F^{″}=MS\left(F^{\prime }\right)\otimes F^{\prime }}$

where F is the input image; MC is the channel attention weight; F′ is the adjusted feature map for channel attention; MS is the spatial attention weight; F″ is the feature map adjusted for spatial awareness; σ is the sigmoid activation function; f is the convolution kernel size; MLP is the shared full connection layer transformation; Avgpool is the pooling of average value; Maxpool is the pooling of maximum value; Concat represents a splicing operation and ⊗ is the multiplication of the corresponding array elements.

Loss function

The loss function determines the direction of gradient decline in the backpropagation process. Continuous training and optimization minimize the loss function value, and finally, the optimal network model (Tolpadi et al., 2022) is obtained. Shadow play image retrieval is a nonlinear regression problem. The mean square error (MSE) is used as the loss function, and the formula is as follows: (6)${\displaystyle MSE=\frac{1}{n}\sum _{i=1}^n{\left(di-Di\right)}^2}$

where n represents the characteristics of shadow play images, and respectively represent the characteristics of the original shadow play image d and the value of the features D of the shadow image processed by CBAM-ResNet50 in i the moment.

However, the network converges prematurely and falls into the local optimum, which leads to low retrieval accuracy due to the little difference between the detail features of the shadow image and the inconspicuous gradient change. We propose an improved loss function; the compensation error is calculated by weighting the mean square error and Pearson distance and establishing a loss function under double constraints.

The Pearson correlation coefficient measures the correlation between different features (Panrawee, Shen & Sakdirat, 2022). The correlation degree is calculated by covariance function, then divided by standard deviation to unify the variables to the same order of magnitude to avoid the oscillation of loss function caused by the difference between features, which will affect the learning ability of the network. It can be used to measure the correlation degree between the original and CBAM-ResNet50 processed shadow image features, and its calculation formula is: (7)${\displaystyle \rho d,D=\frac{\sum _{i=1}^n\left(di-\overline{d}\right)\left(Di-\overline{D}\right)}{\sqrt{\sum _{i=1}^n{\left(di-\overline{d}\right)}^2}\sqrt{\sum _{i=1}^n{\left(Di-\overline{D}\right)}^2}}}$

where $\overline{d}$ and $\overline{D}$ respectively represents the mean value of original shadow play image feature d andfeature Dafter CBAM-ResNet50 processing, respectively; ρd, D represents the Pearson correlation coefficient between d and D, the range of which is (−1,1). The larger the coefficient, the higher the correlation degree. While the smaller the loss function is, the better the model is, so Pearson distance 1 − ρd, D is used as a part of the loss function, and the weighted sum with MSE is used as the loss function. The calculation is shown in Formula (8). (8)${\displaystyle L=\lambda 1MSE+\lambda 2\left(1-\rho d,D\right)}$

where λ1 and λ2 are the weight coefficient.

On the premise that the magnitude of MSE and Pearson distance is consistent, different weights are allocated for weighted summation so that the influence of the two on the gradient is constant, and the gradient change tends to a single loss function so that the network can learn various characteristics. The weight is decided based on the influence of each assessment indicator by the entropy method’s guiding concept. The changes in mean square error and Pearson distance loss in the training process are observed, respectively, and the weights are distributed according to the changes. The smaller the loss changes, the lower the weight, and the larger the loss changes. Since the degree of difference between them equals the weight coefficient. λ1 and λ2 are both set 1. When the loss function is minimized by gradient descent, the gradient change can be more apparent that the CBAM-ResNet50 network can jump out of the local optimum in the gradient descent process, thus improving the retrieval accuracy.