Parallax information fusion-based for dance moving image posture extraction

The existing motion image posture contour extraction results have low definition and serious detail loss. To solve this problem, we propose a novel dance moving image posture extraction method based on parallax information fusion. Firstly, the image with motion information is statistically analyzed by using the information fusion process to determine the position of the motion region. After the noise is reduced by morphological processing, the initial motion posture profile is obtained. The parallax between different control points and the center is used as the active contour model to shape the contraction force and expansion force, which can effectively assist the initial edge contour curve to gradually approach the real edge contour. Finally, the contour of the current moving image is extracted from the sequence image contour to obtain the attitude contour of the moving image. The experimental results show that the proposed method can extract the contour of the moving image clearly with less detail loss, which proves that the proposed method has strong practical performance and can effectively find the contour of the moving object.


Introduction
Moving image target contour extraction refers to the process of finding the required moving target contour from all frames of images in a certain video sequence [1.2]. With the increasingly high requirements for image information accuracy in related fields, the maturity of target contour extraction technology may be further enhanced. It is very important to realize the effective extraction of posture contour of moving image for postprocessing such as moving target recognition and tracking [3]. For this reason, Yu [4] et al. proposed a pose contour extraction method for non-rigid human motion action images. Li et al. [5] proposed a method of contour feature extraction of moving images based on multi-threshold optimization. However, the traditional method of motion image pose contour extraction results are low definition, and the loss of details is serious, resulting in poor overall application effect.
At present, there are two widely used image contour feature extraction methods [6,7]. 1) At the bottom of the moving target image, the edge contour is extracted by integrating the gray level and color information of the image, and the extracted results are effectively grouped to make the extracted edges more complete and closed curves.
2) From top to bottom. Target edge fitting is carried out continuously under the action of external forces through a constant model [8]. The above model is the active contour model, and the realization process of the model can be regarded as an evolution curve that can be automatically transformed in the image. In the process of transformation, the curve will be affected by external forces related to its own structure and other influencing factors, and the internal forces and external forces will be combined to form an energy functional. The transformation process of the above evolution curve is to achieve the fitting of the target contour through continuous iteration until the minimum energy functional is found [9,10].
Information fusion is a process of obtaining accurate identity and location assessment information and comprehensively processing information based on the synthesis and association of multiple information resources. In this process, information can realize continuous self-correction and greatly ensure the effectiveness of information processing results. However, in the process of conventional moving image detection, target contour feature extraction is a very complex process, which needs to study a computationally convenient method. A feature extraction method of complete edge contour based on parallax information fusion is proposed. Simulation results show that this method has high precision and good stability.

Contour analysis of initial moving image
If it realizes the extraction of moving image contour, the initial image contour should be analyzed first. Under normal circumstances, the location of the region segmented by the motion detection process contains the location information of the target in the previous frame, as well as the shape of the current frame, and the result of motion segmentation indicates the current image position of the target. Therefore, the image difference method based on statistical model is used to complete motion segmentation [11,12].
Assuming that the target is moving, the camera equipment is still, the noise of the image before and after the two frames is not correlated, and the mixed noise of the lens and the environment follows the Gaussian distribution 2 / 2  . D(k) is assumed to be the two frames of difference images before and after, where k represents a random pixel in the image. If D(k) is accumulated in the field ) (k  ,  is used for normalization processing, and the specific composition is as follows: Through the above statistical process, the position of the moving region of the image is obtained, the morphological processing is carried out, the noise is removed, and the edge is extracted, which is used as the contour data of the initial moving image.

Image color information analysis
In the detection process of moving target, there is usually a big difference between target color and background color. Therefore, color information can be used to segment the image and derive the color force of the contour at the current position of the image.

Color information image segmentation
There are many segmentation methods of color image, such as Markov random field method, region segmentation and merging method, clustering method and so on. This study uses HIS color model method to achieve simple and fast image segmentation.
In HIS color model, I represents brightness, H represents hue, and S represents saturation [13,14]. When S is large, the tone of the image is very pure, and all regions can be easily separated by H. On the contrary, when S is small, the color of the color image is low and almost becomes grayscale image. At this time, only brightness I can be used to segment the region.
Based on the above principle, S is used to presegment the image, and on this basis, H and S are used to deepen the segmentation. The specific process is shown as follows.
1) Analyze the histogram of S, find the nearest average gray threshold of the image, and then decompose S into

Image color force establishment
According to the above analysis, all colors in the moving target should be in the template obtained after segmentation. Due to the scene statistical error in the process of motion, only part of the target region is in the dynamic frame image. At the same time, a small part of the background will also exist in the background area of the template where there is no occlusion. Because the overall position of the color region corresponding to these error parts is small, the following judgments can be made for the colors of the overlapping parts in the segmentation process: If there is movement in a large area of the color region, the color region belongs to the target movement region; Conversely, it is the background area [15,16].
In order to retrieve all the color regions inside the target and exclude the background of the region, color force should be introduced on the basis of the image contour. In other words, relative to the arbitrary color area A that intersects the image contour, if it is part of the area i A located inside the image contour, then i A is called the effective area, that is, the effective area proportion in A is as follows: If the normal direction N of A control point on the image contour is set to point outside the image contour, then the color direction and size of all control points in area A are: Therefore, in the same area, the color and size of control points are the same, and their directions are the same, which are the analysis of their respective normals. When the contour is in a color region inside the target, because a R >0.5, its color force causes the attitude contour to expand outward, forming the target edge. When the posture contour of moving image is in the background area, due to a R <0.5, resulting in color force to shrink the outline, and the same is to form the target edge. Therefore, its color force will point to the edge of the target to ensure the convergence of image contour [17].

Motion image pose contour extraction
In order to extract a clear posture contour of the moving target, it is necessary to predict the edge of the image contour image obtained above to obtain the maximum values in vertical and horizontal directions, namely the height w and width h of the target. The ratio k of human body height to width can be obtained by comparing these two values. The specific comparison formula is as follows: Where n indicates the number of dynamic frames.
Assume that the moving target changes from lying posture to sitting posture and then to lying posture, and repeat the above twice. According to equation (4), the comparison values of height and width parameters of human body can be obtained, as shown in figure 1. Figure 1. Diagram of human movement transformation curve (X-axis: time/frame n. Y-axis: aspect ratio k). Figure 1 shows a waveform image approximating a sinusoidal curve. The peak value of the waveform refers to the value of sitting standing compared with the maximum point of the aspect ratio k, and the minimum value of the waveform refers to the value of lying down [18].

Posture contour extraction
When obtaining the basic human motion posture, the sequence image contour is used to extract the current motion image contour, and the sequence image contour determines the motion posture accuracy. Then, The Canny operator is used as the detection operator, and Freeman code is used to represent the obtained contour nail, etc. The specific high-precision contour extraction process is shown in figure 2.

Figure 2. Sequence image contour extraction algorithm process
In the acquisition of continuous human motion images, the correlation degree of the two frames in the target sequence image is very high, that is, the contour fluctuation based on the initial human motion. According to the highest gradient contour obtained in the search process, that is, the target contour of the first frame of human motion, other contour lines are obtained by using heuristic search method. The specific process is as follows: 1) Calculate the normals of contour points. Set the contour line of a certain movement of human body. Assuming that the target contour point is P(x,y), the specific relation formula of the normal of the contour point is shown as follows:   3) Determine the current frame contour point. First, Fourier transform was applied to the first image, and Gaussian low-pass filtering was applied to the previous image to reduce noise [19]. In the first image, a point in the outer normal line is searched and the gradient threshold is calculated. The points within the normal direction are searched, and the first point higher than the gradient value is stored, which is regarded as the new contour point of instant frame, so as to obtain the posture contour line of human movement.

Improvement of energy function
The internal force of the image is set as the main influencing factor, and the improved object is set as the energy function. The modification is mainly aimed at the following two aspects: 1) The internal force of the moving image is reconstructed to increase its adaptability and shrinkage ability. 2) Reshape the external force of the image, and add the external energy that can make the contour curve of the moving target image adaptively generate convergence within the range of the relatively weak external force. Based on the traditional Snake model, the proposed method extracted the parallax information from the current evolution curve of the edge contour and subtracted it from the parallax information of the given moving image in the current stage. According to the parallax value, the construction scheme of the external energy of the edge contour was determined. For natural images, the corresponding parallax value can be measured directly by laser ranging equipment or obtained by image matching method. For each control point of the moving image, the Snake model with parallax energy is introduced, then: In the proposed method, the introduction of parallax information can effectively separate the multi-moving image target region from the background region. In view of this differentiability, the relationship between the parallax of the target control point and the fixed central parallax can be determined accurately so as to determine the position of the target control point. When the parallax value is within the set threshold range, it indicates that the control point belongs to the target region, and the external force of the moving image is the expansion force. When the parallax value is higher than the set threshold, it can be considered that the position of the control point is located in the background area or other areas of the image. At this time, the external force acts to assist the edge contour curve to approach the central area gradually. In general, the parallax information of the control point of the moving target image is affected by its neighborhood control point, namely Where, D represents the field centered on the control point. N is the total number of targets. Generally, for image energy detection, the first step is to determine whether there are significant texture features in the neighborhood of the target control point. t is assumed that there are significant texture features in the neighborhood of target control points, so it is unreasonable to determine whether to reduce smoothing constraints only based on image energy [20]. Without the guidance of the staff, the initial position of the edge evolution curve of many moving target images is far from the real boundary. At this time, the parallax information of the image is needed to assist the curve to approach the real boundary gradually. Only when the evolution curve is very close to the real boundary, the Snake model can be used to enhance the convergence ability of the image. The distance between the control point and the actual edge contour is used to determine whether to set its coefficient to 0. The determination method is: before calculating the energy function, it is necessary to determine the direction of the external force at the control point. The energy of each control node is calculated during the first iteration. Assuming that the position of the control point is within the edge contour of the target object, the parallax external force is regarded as the expansion force, and the direction of the control point is recorded as the direction of the external force. In for a second iteration process, the calculation to the above control point, assuming the position of the control points in the target object edge profile, the parallax in the direction of the external force and the force of contrast, if the two are in opposite directions, are the control point has infinite approximation to the real boundary, calculates the strategy adjustment, the energy function are calculated by use of a Snake model.
The direction of parallax energy is affected by two conditions, one of which is the parallax information corresponding to the control point, and the other is the parallax size of the center of the moving target image. The parallax energy can be determined by the Euclidean distance between the control point and the image center of the moving target. Assuming that the parallax value is within the set threshold interval, it means that the control point is located inside the edge contour region. At this point, the external force is regarded as the expansion force, and the larger the Euclidean distance between the control point and the center point, the larger the diop E . Assume that the parallax value is larger than the preset threshold and the external force at this time is regarded as the contraction force. The larger the Euclidean distance between the control point and the center point, the smaller Assuming that the external force is an expansion force, then, (13) In order to effectively enhance the distance constraint between control points, the elastic energy cont E needs to be reconstructed as shown in equation (14): (14) In the traditional Snake model, when the distance between different control points reaches the mean of avg d , the value of cont E is the minimum. Meanwhile, in order to avoid overlapping between control points of evolution curve. It is assumed that the interval between the set control point and the vertex of the left and right evolution curve in its neighborhood is the same, and the average distance between the interval and the control point is the same, the internal force of the moving target image reaches the minimum. The above process can enhance the distribution of control points and provide the prerequisite for parallax information processing of control points.  Formula (18) is used to calculate the gray mean of moving target images.

Contour feature extraction from multimotion images
The variance between foreground region and background region of moving target image is: Assuming that the value of g is the maximum, the optimal threshold is: In practical application, the initial boundary contour of moving target image needs to be set in advance. However, there are some differences in this method, and the results obtained by different methods to depict the edge contour of moving target image are also different. In order to obtain more accurate results, the edge of moving image needs to be extracted twice.
The edge contour of a single moving target image is extracted from continuous edges, and the active contour model adopted needs to extract the outer contour. In order to obtain the contour accurately, the result of single edge contour extraction is filled globally, and the area where the two images intersect is the target area [21].
The shape of the edge contour extracted from the image of the secondary moving target is very similar to the real boundary contour, which can effectively remove the sunken area. The extraction results of the secondary contour are taken as the initial edge of the active contour model to gradually reduce the energy of the boundary contour. When the energy value reaches the minimum, the obtained edge image is the final target contour of the moving image [22].
It is assumed that x and y represent the coordinates of the initial edge points of the moving image, and s represents the normalized arc length on the edge of the moving image. Formula (22) is used to calculate the initial edge of the moving image.
Where  and  represent the weighting coefficients of moving images.  mainly controls the elasticity of the moving image curve, and  mainly controls the rigidity of the curve.
Where, the external energy is set to be: The moving object image can limit the trend of evolution curve. In the iterative stage, minimizing the internal energy of the moving object image can make the evolution curve continuously shrink or expand, and ensure that the evolution curve tends to be smooth. The external force of the moving image can bring the target object close to the initial contour phase. For multi-motion images, the gray mean difference between the image edge contour area and the background area is large, and this gray difference is more obvious at the position of the contour edge. In order to gradually indent the initial edge contour into image features, the external energy is set as a gradient function. The use of active contour model is to rapidly approach the target boundary under the influence of the energy produced by the target. When the energy value reaches the minimum, the contour features of multiple moving images are obtained.

Experiments and analysis
In order to verify the comprehensive performance of the proposed method, the proposed method, reference [23] and reference [24] are compared and analyzed to identify errors of different human movements. The experimental environment is: AMD A10-5750m2.50GHz CPU, 4GB memory, Win10 operating system, Visual Studio2013 + 0pencv3.0, Matlab2013 software. The experiment collected various human body sequence movements of 300 freshmen in 2021 from a physical education college in a city, and different subsets contained 10 different sampling postures. There are about 20 directions for the projection of various poses. Experiments are carried out using 20 kinds of basic motion to distinguish the recognition errors of different motion images (%). Specific experimental results are shown in Table 1. According to table 1, compared with the other two methods, the recognition error of the proposed method is the lowest. The following experiments mainly focus on the recognition error (%) of human body image in walking, and the specific comparison results are shown in Table 2.  Table 2 shows that compared with the other two methods, the proposed method has the lowest human limb image recognition error.
For the performance of contour feature of single motion image, the proposed method generates the initial pose and final pose of a gymnast through a 3D model. Assuming that the spatial position of the gymnast remains unchanged, it is set as the original posture of human body movement. After calculation, the posture of the front frame is set as the original value of the time frame. The specific comparison results are shown in Table 3. In Table   3  In order to compare the practical performance of the proposed method and the traditional method, the number of sub-blocks and the effective area ratio of the target contour are taken as the discriminant criteria.
The expected goal of correct attitude contour extraction from moving images is to obtain a single connected complete contour block. Therefore, the less the number of disconnected sub-blocks is, the more complete the extracted contour is. However, due to the interference of occlusion and noise, the moving image will lose information to a certain extent. It is embodied in the binarization image processed, that is, the area with the initial gray level of 1 appears the information loss area with the gray level of 0. Therefore, the ability of different methods to obtain effective information is evaluated by calculating the area ratio where gray level is 1.
The proposed method and the two traditional methods are used to extract the correct attitude contour of the same sequence of moving images respectively, and the specific comparison results of the number of disconnected subblocks and effective area ratio are shown in Table 4.  Table 4, we can see that a sequence of moving images is divided into 32, 46 and 60 frames. After processing with the method in this paper, when the image is in the sequence of 32 frames, it is less than that of the method in reference [24] and the method in reference [23]. When the image is in the sequence of 46 frames, the number of disconnected sub-blocks is 3, which is slightly less than that of the method in reference [24] and significantly less than that of the method in reference [23]. When the image is in the sequence of 60 frames, it is slightly less than that of the method in reference [24] and significantly less than that of the method in reference [23]. It can be shown that the number of disconnected subblocks in the method in this paper and the method in reference [24] is relatively low, which means that the extracted contour is more complete. At the same time, the effective area ratio of the extracted information is significantly higher than that of the other two comparison methods after the proposed method is used to extract the attitude contour of the moving image, indicating that the proposed method can ensure less detail loss of image information.

Conclusion
With the increasing demand on the accuracy of motion image pose contour extraction in related fields, the application effect of traditional methods is greatly reduced due to the poor extraction effect of contour and boundary and the easy loss of details. Therefore, this study designs a dance posture contour extraction method based on parallax information fusion. Firstly, the original image is analyzed based on statistical model and image difference method, and the contour of the initial image is obtained. Then, the image is segmented by HIS color model method, and the color force is constructed based on the image contour. Based on the result of feature extraction, attitude contour of moving image is extracted by using sequence image contour. Simulation results show that compared with the traditional method, the motion image posture contour image extracted by the method designed in this study is clear and the details remain intact, indicating that the method has high application performance.