The influence of music teaching appreciation on the mental health of college students based on multimedia data analysis

Extraction of music features

We must extract short-term energy features, time-domain variance features, and other music appreciation features to construct the music appreciation classification model for the diversity of music therapy features (Modran et al., 2023). The specific extraction process is as follows:

1) Short-term energy features of appreciation music. There is a big difference between the energy of ordinary sound and appreciation music. Compared with ordinary sound, appreciation music has a higher energy value. Therefore, we can extract the energy features of short-term from each frame in appreciated music. Assume that E_m represents the energy of the music appreciation signal $\left\{\mathrm{y}\left(\mathrm{m}\right)\right\}$. The short-term energy of the music appreciation signal $\left\{\mathrm{y}\left(\mathrm{m}\right)\right\}$ is calculated as follows:

(1)${\displaystyle E_m={\sum }_{-\infty }^{+\infty }{\left[y\left(n\right)\cdot c\left(n-m\right)\right]}^2}$

where $\mathrm{c}\left(\mathrm{m}\right)$ refers to the window function. We regard N as the duration of the appreciated music. Therefore, E_m can be presented as follows:

(2)${\displaystyle E_m={\sum }_{n=0}^{N-1}{\left[y\left(n\right)\cdot c\left(n-m\right)\right]}^2}$

2) The variance features Time-Domain in appreciated music. The following formula can express any frame of appreciated music:

(3)${\displaystyle Y_t\left(n\right)=\left\{y_t\left(n,1\right),y_t\left(n,2\right),\dots ,y_t\left(n,N\right)\right\}}$

where ${\mathrm{Y}}_{\mathrm{t}}\left(\mathrm{n}\right)$ represents a frame of appreciation music. We apply the wavelet transform smoothing operation for ${\mathrm{Y}}_{\mathrm{t}}\left(\mathrm{n}\right)$ and achieve the processed frame ${\hat{\mathrm{Y}}}_{\mathrm{t}}\left(\mathrm{n}\right)$ of appreciation music. At this time, a frame of appreciation music after processing can be described by the following formula; the specific formula is as follows:

(4)${\displaystyle {\hat{Y}}_t\left(n\right)=\left\{{\hat{y}}_t\left(n,1\right),{\hat{y}}_t\left(n,2\right),\dots ,{\hat{y}}_t\left(n,N\right)\right\}}$

Combined with formula (4), we calculate the mean ${\mathrm{E}}_{\mathrm{t}}\left(\mathrm{n}\right)$ and variance ${\mathrm{D}}_{\mathrm{t}}\left(\mathrm{n}\right)$ of the time domain of music appreciation. The formula for calculating the mean and variance is as follows:

(5)${\displaystyle E_t\left(n\right)=\frac{1}{N}{\sum }_{i=1}^N{\hat{y}}_t\left(n,i\right)}$

(6)${\displaystyle D_t\left(n\right)=\frac{1}{N}{\sum }_{i=1}^N{\left[{\hat{y}}_t\left(n,i\right)-E_t\left(n\right)\right]}^2}$

3) The variance features of Frequency-Domain in appreciated music. Any frame ${\mathrm{Y}}_{\mathrm{f}}\left(\mathrm{n}\right)$ of appreciated music is obtained from the frame ${\mathrm{Y}}_{\mathrm{i}}\left(\mathrm{n}\right)$ by Fourier transform. Then the description of the specific frame of music appreciation after transformation is represented in the following formula:

(7)${\displaystyle Y_f\left(n\right)=\left\{y_f\left(n,1\right),y_f\left(n,2\right),\dots ,y_f\left(n,N\right)\right\}}$

We adopt the wavelet transform smoothing operation ${\mathrm{Y}}_{\mathrm{f}}\left(\mathrm{n}\right)=\frac{1}{\mathrm{N}}{\sum }_{\mathrm{i}=1}^{\mathrm{n}}{\left[{\hat{\mathrm{y}}}_{\mathrm{f}}\left(\mathrm{n},\mathrm{i}\right)-{\mathrm{E}}_{\mathrm{f}}\left(\mathrm{n}\right)\right]}^2$ to obtain ${\hat{\mathrm{Y}}}_{\mathrm{f}}\left(\mathrm{n}\right)$, whose expression is as follows:

(8)${\displaystyle {\hat{Y}}_f\left(n\right)=\left\{{\hat{y}}_f\left(n,1\right),{\hat{y}}_f\left(n,2\right),\dots ,{\hat{y}}_f\left(n,N\right)\right\}}$

We associate Eq. (8) to calculate the mean ${\mathrm{E}}_{\mathrm{f}}\left(\mathrm{n}\right)$ and variance ${\mathrm{D}}_{\mathrm{f}}\left(\mathrm{n}\right)$ of music appreciation in the frequency domain. The specific calculation formula is as follows:

(9)${\displaystyle E_f\left(n\right)=\frac{1}{N}{\sum }_{i=1}^N{\hat{y}}_f\left(n,i\right)}$

(10)${\displaystyle D_f\left(n\right)=\frac{1}{N}{\sum }_{i=1}^N{\left[{\hat{y}}_f\left(n,i\right)-E_f\left(n\right)\right]}^2}$

Feature fusion

1) We assume ${\mathrm{Y}}_{\mathrm{i}}=\left({\mathrm{y}}_{\mathrm{i}}\left(1\right),{\mathrm{y}}_{\mathrm{i}}\left(2\right),\dots ,{\mathrm{y}}_{\mathrm{i}}\left(\mathrm{m}\right)\right),\mathrm{i}=1,2,\dots ,\mathrm{m}$ to represent the feature vector composed of the short-time energy feature, the variance features of Time-Domain, and the variance features of Frequency-Domain in the appreciated music.

2) We apply the preprocessing ${\mathrm{Y}}_{\mathrm{i}}=\left({\mathrm{y}}_{\mathrm{i}}\left(1\right),{\mathrm{y}}_{\mathrm{i}}\left(2\right),\dots ,{\mathrm{y}}_{\mathrm{i}}\left(\mathrm{m}\right)\right)$ To set the reference object, the randomly selected feature vectors. We regard the selected feature vectors as ${\mathrm{Y}}_0^{\prime }=\left({\mathrm{y}}_0^{\prime }\left(1\right),{\mathrm{y}}_0^{\prime }\left(2\right),\dots ,{\mathrm{y}}_0^{\prime }\left(\mathrm{m}\right),\right)$ And combine the selected feature vectors and other groups of feature vectors as a pair of feature vectors.

3) We apply ${\Delta }_{\mathrm{i}}\left(\mathrm{k}\right)=\left|{\mathrm{y}}_0^{\prime }\left(\mathrm{k}\right)-{\mathrm{y}}_{\mathrm{i}}^{\prime }\left(\mathrm{k}\right)\right|$ To compute the maximum and minimum deviation of a feature vector group. The detailed calculation process of the maximum deviation and minimum deviation is as follows:

(11)${\displaystyle \left\{\begin{array}{c}{P}_1=maxmax{\Delta }_i\left(k\right)\hfill \\ P_2=minmin{\Delta }_i\left(k\right)\hfill \end{array}\right.}$

4) We calculate the grey correlation coefficient of music appreciation features. The formula is:

(12)${\displaystyle g_i\left(k\right)=\frac{n+\alpha M}{{\Delta }_i\left(k\right)+\alpha M}}$

where α represents the resolution coefficient,

We calculate the grey correlation degree of music appreciation features. The formula is as follows: (13)${\displaystyle g_i=\sum _{k=1}^n{w}_i{g}_i\left(k\right)}$

where w_i represents the weight. According to Equation (13), the contribution value of music appreciation features can be obtained. The contribution value can describe the influence degree of each feature on music appreciation.

Through the above methods, we can directly obtain the short-time energy feature, time domain variance feature and frequency domain variance, and realize the fusion of their features. Using the fusion feature, we adopt ResNet as the backbone of our method, with the help of ResNet’s residual structure, to achieve the classification of musical emotions. However, considering the characteristics of feature fusion, we need to optimize the neural network structure.

PSO

The process of the PSO algorithm is as follows: we assume that the search space is L dimensions, in which the groups occupy the K particles. The positions of the j-th particle are expressed as ${\mathrm{X}}_{\mathrm{j}}=\left({\mathrm{x}}_{\mathrm{j}1},{\mathrm{x}}_{\mathrm{j}2},,\dots ,{\mathrm{x}}_{\mathrm{jl}}\right)$. The corresponding optimal solution for the positions of the j-th particle is ${\mathrm{P}}_{\mathrm{j}}=\left({\mathrm{p}}_{\mathrm{j}1},{\mathrm{p}}_{\mathrm{j}2},,\dots ,{\mathrm{p}}_{\mathrm{jl}}\right).{\mathrm{p}}_{\mathrm{y}}$ represents the optimal global individual. The speed for the positions of the j-th particle is expressed as vectors ${\mathrm{V}}_{\mathrm{j}}=\left({\mathrm{v}}_{\mathrm{j}1},{\mathrm{v}}_{\mathrm{j}2},,\dots ,{\mathrm{v}}_{\mathrm{jl}}\right)$. The position and velocity of each particle with the change of iteration can be represented as follows: (14)${\displaystyle v_{jl}\left(t+1\right)=g\cdot v_{jl}\left(t\right)+z_1\cdot rd\cdot \left(p_{jl}\left(t\right)-x_{jl}\left(t\right)\right)+z_2\cdot rd\cdot \left(p_y\left(t\right)-x_{jl}\left(t\right)\right)}$ (15)${\displaystyle x_{jl}\left(t+1\right)=x_{jl}\left(t\right)+v_{jl}\left(t+1\right)}$

where g represents the inertia factor, z₁ and z₂ denotes the acceleration factor and the acceleration factor is an average number. In addition, rd indicates a random value between $\left[0,1\right]$ and t refers to the current iteration number. Since the velocity and initial position are randomly set, the position and velocity of the particle can be iterated through Eqs. (14) and (15) until the termination can be realized or the iteration reaches the ending. The formula of inertia weight in Eq. (14) is as follows:

(16)${\displaystyle g=\left\{\begin{array}{cc}{g}_{max}-\frac{\left(g_{max}-g_{min}\right)\left(s-s_{ave}\right)}{s_{max}-s_{ave}},\hfill & s\ge s_{ave}\hfill \\ g_{max}\hfill & s<s_{ave}\hfill \end{array}\right.}$

where s represents particle adaptation value, s_ave and s_max represent the average adaptation value of particles in each generation and the maximum adaptation value in the particle swarm, respectively.

Steps of optimization

According to the principle of the particle swarm optimization algorithm analyzed above, we can achieve the optimal thresholds and weightings by training the weightings and thresholds of the deep network with the PSO. Therefore, we can construct the network by these optimal weights and thresholds. The optimization process can be as follows:

1) First, we normalize the training and testing samples and set up a three-layer neural network topology structure. Then, we regard dimension M of the weighted three vectors obtained in section 1.2 as the input nodes number S_I. Besides, we take many appreciative music classes L_B to be the output nodes S_I of network. The geometric average of input and output layer nodes is the number of the hidden node. The individual particles in the population are represented as real number vectors obtained by encoding all the thresholds and connection weights between neurons.

2) Design the fitness function. The fitness function is the metric of evaluating the neural network to solve problems. We adopt the mean square error of the output value of our model to obtain the objective function. Furthermore, we treat the inverse of the objective function as the fitness function.

3) Initialize and set parameters. The parameters include the number of particles, initial position, and velocity. We control the initial position and velocity between [0,1] to train our model faster. We also set the acceleration factor and the maximum number of iterations simultaneously.

4) According to the input and output samples, we calculate the fitness function to obtain the value of individual particles using Eq. (17). At the same time, we regard the historical best position as the best position of each particle to start the iteration for the particle.

5) We follow Eqs. (14)–(16) in the particle swarm algorithm to renew the velocity of particles and the position of particles. Then, we check whether the location of the particles and the momentum of the particle are trapped in the boundary. If particles have crossed the border, it is necessary to exclude the crossing particles and update the velocity of particles and the position of the particles again.

6) We calculate the fitness function to obtain a value of the updated particles and change the inertia weight according to the fitness value. Then, we search for the best position of the particle and check whether it satisfies the end judgment.

However, the performance of PSO is affected by the selection of the initial particle swarm, and local optimization is still a potential influence for complex non-convex problems. It is difficult to adjust parameters, such as inertia weight, individual learning factor and group learning factor, and the setting of these parameters may affect the performance of the algorithm. Meanwhile, compared with our proposed model, user survey and psychological test can directly obtain students’ subjective feedback and psychological feelings, which involves higher labor and time costs. Our model can quickly process large amounts of data and automate sentiment classification, saving labor and time costs.

Classes for the emotion of music therapy

Before conducting experiments to compare with other methods, we should divide all music samples into different types to prepare for the further analysis of emotion, as shown in Table 2. Furthermore, we artificially set the music of emotion to 10 other emotions: anxiety, fear, depression, euphoria, melancholy, happiness, sadness, peace, and irritability.

In addition, we analyze the distribution and statistics of the entire dataset; the results are presented in Fig. 2. According to this, we can demonstrate that the dataset we collect is excellent for meeting the demand for emotion classification. Then, we count the distribution of the ten emotions in our dataset. The distribution and statistics are presented in Figs. 3 and 4. It can be seen that the feelings of happiness, euphoria, and peace occupy the high ratio, which are the primary emotions.

Comparison with other methods

To evaluate the classification results of our model in this paper, we adopt the interactive music appreciation pitch and length to represent the frequency features of music appreciation. At the same time, we apply the reverberation time and tone to describe the time characteristics of music appreciation. Besides, the mean square of short-term energy in music represents the energy features of music appreciation. We use the music appreciation emotion classification model to output the 10 emotion classes of music appreciation. The performance of our model is presented in Table 3.

As shown in Table 3, we can demonstrate that our model achieves excellent classification performance on the emotion of interactive music appreciation teaching. Our model can classify the emotion in the music of Ensemble, Orchestral Silk & bamboo, Piano, Chamber, Vocal, Military, Symphony, Instrumental, and Percussion with accuracy of 74.1%, 85.6%, 77.0%, 82.5%, 84.3%, 86.8%, 88.8%, 80.3%, 82.6%, and 84.2%, respectively, whose average score of accuracy can reach 82.6%. By comparing the classification results with the actual emotion categories of music, we can find that the results of the emotion classification model for appreciated music are the same as the ground truth of appreciated music, which indicates that the emotion classification model of interactive music appreciation teaching is highly accurate. We can conclude that our model can effectively classify the emotion in the interactive music appreciation teaching to adjust the psychology of college students. Since we use three completely different features to predict emotion in music, our model is able to guarantee the prediction effect in different data sets and subjects, and our prediction accuracy can verify this.

Music appreciation teaching can provide students with rich and colorful music experience and help them better understand the language and expression of music. By learning about different types of music and emotional categories, students can develop an appreciation for music and an aesthetic awareness. Music appreciation teaching can also enhance students’ emotional expression ability, improve emotional cognition and emotional management ability, and play a positive role in relieving stress and emotional regulation.

The significance of accurate predicting emotional categories for teaching strategies and mental health. Accurately predicting students’ emotional categories can help teachers better understand students’ emotional states and emotional needs. According to the predicted results of different emotional categories, teachers can formulate corresponding teaching strategies to promote students’ positive mental health. For students to show positive emotions, teachers can use more active and pleasant music to teach, stimulate students’ learning interest and participation. For students who show negative emotions, teachers can adopt a more caring, supportive and understanding attitude, provide positive music to relieve them, encourage students to talk about their emotions and seek professional help.

In addition, the accurate prediction of emotional categories can also be used to evaluate teaching effectiveness and mental health status, providing schools with important student emotional data and mental health feedback. By obtaining the information of students’ emotional state in time, schools can take more effective mental health intervention measures, provide necessary psychological counseling and support services, and promote students’ all-round development and mental health.