A 2D image 3D reconstruction function adaptive denoising algorithm

Background area processing

Let the image signal processed for noise reduction be I = f + n, where f is the image signal. n is the noise signal. In this article, the noise reduction function of the image is obtained by improving the soft threshold denoising algorithm. The expression of the function is defined as follows: (8)${\displaystyle {\hat{C}}_{\mathrm{ij}}=\left\{\begin{array}{c}\mu \cdot \mathrm{sgn}\left(C_{i,j}\right)\left(\left|C_{\mathrm{i},\mathrm{j}}\right|-\lambda \right)\hfill \\ 0\hfill \end{array}\right.\begin{array}{c}\hfill ,\left|C_{\mathrm{ij}}\right|\ge \lambda \hfill \\ \hfill \left|C_{\mathrm{ij}}\right|<\lambda \hfill \end{array}}$

Where $\mathrm{sgn}\left(n\right)=\left\{\begin{array}{cc}1\hfill & ,n>0\hfill \\ -1\hfill & ,\mathrm{n}\le 0\hfill \end{array}\right..\mu$ is the weight constant. λ Is the threshold value. Although the selection of the threshold value directly affects the noise reduction process of the image. Considering the region’s more practical information, it is easy to cause the “overkill” phenomenon. In this article, the weight μ is defined. The value of μ is 0.6 and is substituted into the image noise reduction formula. Then the formula is as follows: (9)${\displaystyle {\hat{C}}_{\mathrm{ij}}=\left\{\begin{array}{c}0.6\mathrm{sgn}\left(C_{i,j}\right)\left(\left|C_{\mathrm{i},\mathrm{j}}\right|-\lambda \right)\hfill \\ 0\hfill \end{array}\right.\begin{array}{c}\hfill ,\left|C_{\mathrm{ij}}\right|\ge \lambda \hfill \\ \hfill \left|C_{\mathrm{ij}}\right|<\lambda \hfill \end{array}}$

Target area processing

Confidence processing.

This study proposes a method to introduce a confidence factor in GAN. The confidence data is obtained by combining the data generated by the generator with the data in the discriminator.

As shown in Figs. 4 and 5, the ground truth of S is calculated from the 2D points x_j,k and x_i,k annotated in the image. where x_j1,k  and x_j2,k denote the two key points j₁ and j₂ corresponding to the real pixel points of a person k in the figure.

If a pixel point C_real is located on this target node. ${\mathrm{L}}_{\mathrm{c},\mathrm{k}}^{\mathrm{\ast }}\left({\mathrm{C}}_{\text{real}}\right)$ the valued table is a unit vector from a key point j₁ to key point j₂. The corresponding vector is a zero vector for pixel points not on the torso. Then the values of ${\mathrm{L}}_{\mathrm{c},\mathrm{k}}^{\mathrm{\ast }}\left({\mathrm{C}}_{\text{real}}\right)$ Are as follows: (10)${\displaystyle L_{c,k}^{\ast }\left({\mathrm{C}}_{real}\right)=\left\{\begin{array}{c}vif{\mathrm{C}}_{real}onc,k\hfill \\ 0otherwise\hfill \end{array}\right.}$

where $\mathrm{v}=\frac{\left({\mathrm{x}}_{{\mathrm{j}}_2,\mathrm{k}}-{\mathrm{x}}_{{\mathrm{j}}_1,\mathrm{k}}\right)}{|{\mathrm{x}}_{{\mathrm{j}}_2,\mathrm{k}}-{\mathrm{x}}_{{\mathrm{j}}_1,\mathrm{k}}{|}_2}$ Denotes the unit direction vector corresponding to this torso.

The target pixel point satisfies the following function: (11)${\displaystyle 0\le v\left({\mathrm{C}}_{real}-x_{j_1,k}\right)\le l_{c,k}and\left|v_{\perp }\left({\mathrm{C}}_{real}-x_{j_1,k}\right)\right|\le {\sigma }_l}$

where the inner table σ_l shows the distance between pixel points. The torso length is l_c,k = |x_j2,k − x_j1,k|₂ and ${\mathrm{v}}_{\perp }$ Denotes the vector perpendicular to v.

Generator and discriminator

We optimize the loss L_MSE of the generator and the adversarial loss L_adv of the discriminator. (12)${\displaystyle L_{MSE}=\sum _{i=1}^N\sum _{j=1}^M{\left(C_{ij}-{\hat{C}}_{ij}\right)}^2}$ (13)${\displaystyle L_{adv}=\sum _{i=1}^N{\left({\hat{C}}_j-D\left({\hat{C}}_j,X\right)\right)}^2}$ (14)${\displaystyle L_G=L_{MSE}+\lambda L_{adv}}$

The primary objective of the discriminator is to discern whether a given heat map is real or fake, generated by the generator (Vo et al., 2021; Dong et al., 2021; Li et al., 2022b; Lu & Su, 2021; Luo et al., 2021). To accomplish this, we optimize the loss function of the discriminator to improve its ability to differentiate between real and fake heat maps. The optimization process aims to enhance the discriminator’s discriminative capabilities and make it more effective in accurately identifying the authenticity of the input heat maps. By fine-tuning the loss function, we aim to facilitate the discriminator in becoming increasingly proficient at recognizing real and generated heat maps. (15)${\displaystyle L_{real}=\sum _{j=1}^N{\left(C_j-D\left(C_j,X\right)\right)}^2}$ (16)${\displaystyle L_{\text{noiseless}}=\sum _{j=1}^N{\left({\hat{C}}_j-D\left({\hat{C}}_j,X\right)\right)}^2}$ (17)${\displaystyle L_G=L_{real}+k_t{L}_{\text{noiseless}}}$ (18)${\displaystyle k_{t+1}=k_t+{\lambda }_k\left(L_{real}-L_{\text{noiseless}}\right).}$

The kt in the above equation is used to constrain the capability of the resolver.

As shown in Fig. 6, it can be seen that all components in this network, including the confidence map, are learned from the image only. In processing this deep neural network, the feature factors are derived from the original image. These feature factors are mapped to the image with depth information in a recombination manner to construct a new image.

If the expected target is irrelevant for noisy samples, the pose in this image is not credible for the body construction. Therefore, if the information is closer to the target information, the corresponding vector is 1, and vice versa, the corresponding vector is 0. The formula is as follows: (19)${\displaystyle {\mathrm{C}}_{real}=\left\{\begin{array}{c}1if\left|\left|K_{ij}\right|\right|<\tau \hfill \\ 0if\left|\left|K_{ij}\right|\right|\ge \tau \hfill \end{array}\right.}$

K_ij represents the value of the affiliation function for the background factor.

The system framework is shown in Fig. 7.