A novel Gauss-Laplace operator based on multi-scale convolution for dance motion image enhancement

Traditional image enhancement methods have the problems of low contrast and fuzzy details. Therefore, we propose a novel Gauss-Laplace operator based on multi-scale convolution for dance motion image enhancement. Firstly, multi-scale convolution is used to preprocess the image. Then, we improve the traditional Laplace edge detection operator and combine it with Gauss filter. The Gaussian filter is used to smooth the image and suppress the noise, and the edge detection is processed based on the Laplace gradient edge detector. The detail image extracted by Gauss-Laplace operator and the image with brightness enhancement are linearly weighted fused to reconstruct the image with clear detail edge and strong contrast. Experiments are carried out with detailed images in different scenes. It is compared with traditional methods to verify the effectiveness of the proposed method.


Introduction
Image enhancement technology is widely used in aerospace, maritime rescue and military target detection [1][2][3][4]. However, due to the influence of the sensor's own attributes, the contrast between the background and the target in the image is low and the edge is fuzzy. At the same time, due to the far distance between the target and the sensor in the scene, there is obvious noise in the image due to the influence of atmospheric thermal radiation, and the target texture information is not prominent enough, and the visual effect of the image is poor. Image enhancement is an effective method to effectively improve image contrast, highlight image details and improve image visual effect [5,6] The image consists of pixels with different gray levels. Although point and line detection is very important in gray level discontinuity detection, edge detection is by far the most common method. Image edge detection is realized by using the extreme value of the first derivative (gradient operator) or the zero-crossing information of the second derivative (Laplace operator) [7]. Edge detection is the process of determining and locating sharp discontinuities in images. Edge detection plays an important role in image analysis. It is one of the traditional segmentation techniques. Gaussian filter plays a crucial role in edge detector [8]. Gaussian  Recently, histogram equalization [9] and transformed domain equalization [10,11] have greatly improved the image quality and received much attention from scholars. Histogram equalization technique is a contrast stretching image enhancement method, which can effectively improve the brightness and contrast of low-quality images. Tsai et al. [12] used different thresholds to divide the original histogram into two or more sub-intervals to improve the image quality. However, these methods do not consider the distribution of the threshold selection, especially when the image histogram has local peaks. Palanikumar et al. [13] dynamically divided the original image histogram according on the image information entropy. Although the method had no parameters and specified the number of interval divisions, the information entropy of the image needed to increase the complexity of the algorithm. Singh et al. [14] adopted the histogram shear technology. Sengee et al. [15] used the frequency weighted technology. Huang et al. [16] used gamma function to correct histogram cumulative function, which could be used for image enhancement. These methods are applicable to single image target and histogram curve fluctuation, they can artificially control the degree of image enhancement. Ghosh et al. [17] enhanced the images combining with optimization theory and functional analysis. These methods do not rely on the original histogram distribution and just solve the target value according to the set conditions, but their mathematical operation are complex and require a clear understanding of the target histogram distribution [18]. Prasath et al. [19] segmented the original image in space and then equalized each sub-block independently to achieve the effect of improving the local details of the image. However, due to the equalization operation on sub-blocks by these methods, block fault effect will be caused and the image enhancement effect will be affected. In addition, many times of equalization are needed to increase the complexity of the algorithm. The above methods are mainly based on the image histogram attributes, and have achieved good enhancement effects. Therefore, histogram equalization technology is widely used in image defogging, medical image processing and target detection.
Image enhancement methods based on transform domain [20][21][22] mainly transform to gradient domain or wavelet domain by means of gradient domain transform and wavelet transform for image equalization and image enhancement. The gradient domain mainly reflects the detail information of the image and can display the gradient value of the image at the gray level. Gopinathan et al. [23] introduced the gradient domain into the histogram prescriptive technology, and reconstructed the gradient field of the target image by means of the weighted fusion of the bi-modal Gaussian function and the original image. Li et al. [24] realized the effect of outdoor image contrast enhancement and fuzzy image sharpening by reflecting the detail characteristics of image gradient domain. In the wavelet domain, the edge and detail features of the image correspond to the high frequency coefficient, and the background and contour information of the image correspond to the low frequency coefficient, so the image enhancement in the wavelet domain is mainly achieved by using the equalization wavelet coefficient. Kaur et al. [25] improved the image quality by using stereo endoscope image enhancement method in wavelet domain. Sdiri et al. [26] achieved the de-noising and enhancement of medical images respectively by using dual-tree complex threshold wavelet transform combined with other traditional methods. Muhammad et al. [27] realized the detail enhancement of infrared image by using the idea of image layering. Such method is mainly used to enhance the detail and texture information of the image, but cannot improve the brightness and overall quality of the image.
Reference [28] proposed an image segmentation model, which included edge detection based on Canny and normalized cut feature vectors. Before edge detection, median, Gaussian or Frost filters based on noise type were used for preprocessing. Reference [29] developed an edge detection technology for medical images to track the boundaries of anatomical organs based on intensity gradient and texture gradient features. Compared with the traditional active contour model, the proposed model could produce effective results. Reference [30] used various gradient-based edge detectors on images. The Canny operator produced good results, but the parameters needed to be adjusted in some cases. Reference [31] had proved that compared with Laplace and Sobel edge detectors, morphological edge detectors produced effective results on images in the presence of noise. Reference [32] proposed edge detection based on image morphology operation, which determined edge by taking the difference between expanded and eroded images. Reference [33] had analyzed various edge detection operators in [34] of Berkeley data set. Gaussian filter pretreatment was carried out before image and edge detection, and excellent results were produced.
To sum up, in order to better improve image quality and enhance image texture detail information, this paper combines multi-scale convolution, threshold segmentation, and deep fusion of image contour and Gauss-Laplian operator for dance image enhancement, which achieves the effect of enhancing image contrast, brightness and texture details.

Related works
Most edge detection algorithms are based on image derivatives or gradients. In general, the image is easily damaged by Gaussian noise or affected by speckle noise. Therefore, edge detection in noise image plays an important role. The purpose of the edge detector is to trace the desired boundary or contour of the region of interest in the image. The gradient of the 2D function g(x,y) is shown in equation (1).
The edge intensity is given by the magnitude of the above vectors, as shown in equations (2) and (3). The direction of gradient is determined by equation (4) ) / ( tan 1 x The edge detection algorithm should carefully track the edges and must eliminate the false edges caused by noise pixels. The following assumptions are made in the image edge tracking: a) The gradient of edge pixels is larger than that of noise pixels; b) The edge size and direction change slowly along the edge.

Sobel operator
The Sobel operator consists of a pair of 3×3 convolution kernels. The kernel in the Sobel operator maximizes both vertical and horizontal running edges. A separate kernel is applied to the input image to produce different measurements of the gradient components in each direction ( x G and y G ). The kernel reactions are then combined to find the absolute magnitude of the gradient at each point and the direction of that gradient [2]. The gradient is given by equation (5).

Roberts cross gradient operator
Roberts cross gradient operator is mainly used to calculate two-dimensional spatial gradient of images, which is simple and fast. Edge detection is the process of determining and locating sharp discontinuities in images. Roberts cross gradient operator belongs to first order differential operator. For the existence of noise and uneven illumination, the edges of objects can not be completely delineated, so the noise can not be suppressed. The kernel in the Roberts operator maximizes running on an edge of 45°.

Prewitt operator
The Prewitt operator, similar to the Sobel operator, is used to detect vertical and horizontal edges in images. Unlike the Sobel operator, it does not value pixels near the center of the mask. Similar to Roberts cross gradient operator, Prewitt operator is also a first-order differential operator. It is also sensitive to noise and easy to be destroyed by Gaussian noise or affected by speckle noise.

Laplace operator
Different from the gradient operators introduced earlier, the Laplace operator is based on the second derivative of the image to find the edge and search for zero crossing. The Laplace transform I(x,y) of the image with pixel intensity value L(x,y) is shown in equation (6). Traditional Laplace operators often produce edges that are two pixels wide. Figure 1 is a common template for Laplace operators. It is easy to see that when the edge detection of bright spots in the dark region is carried out, the Laplace operation will make them brighter. Therefore, like gradient operator, Laplace operator can not suppress image noise. In this paper, an improvement is made to determine whether the edge pixel is dark or bright after the edge pixel is known. Figure 2 shows the improved Laplace operator extension template.

Gauss Laplace operator
From section 2.4, we can see that the Laplace operator does not suppress noise. If there is an operator that can combine the Gaussian smoothing filter with the Laplace operator to smooth out the noise before edge detection, good results will be achieved. Based on this, Gauss-Laplace operator is proposed in this paper. Figure 3 is a commonly used 5×5 Gauss-Laplace template. Laplace edge detection consists of two stages: edge enhancement and tracking. Gaussian filter is used to smooth the image, and its large core size reduces the sensitivity to noise. After the direction of the edge is determined, non-maximum suppression is applied to track the path of the edge and pixels that are not part of the edge are ignored. Finally, the lag threshold is applied to eliminate the fringe. Define 2 1 t t  , two thresholds 1 t and 2 t are applied to the image gradient size. A pixel with a threshold greater than 1 t is considered an edge pixel. Gaussian gradients use derivatives of the Gaussian method to determine gradients/derivatives of scalar 2D images and 3D volumes.
The two-dimensional representation of the Gaussian kernel is: The  term in Gaussian filters is called smooth scale.
This ratio has great influence on the response of Gaussian filter. The larger  denotes that the image will become blurred and the sensitivity to noise will be reduced. In Instead of using 2D cores, separable filtering methods are used to calculate the gradients of 1D kernels along the x and y directions. The Gaussian function is separable and can be decomposed into the product of two 1D Gaussian functions. )) The important property of Gaussian filter is that it can satisfy the uncertain relation.
 are the variances of space and frequency domain respectively. Unique attributes make the best compromise between the goals of localized conflict between the spatial and frequency domains respectively. For filters, the Gaussian kernel's response is non-zero in infinite domains and in most domains. It is very small because it is exponential.

Proposed image enhancement method
This new algorithm can be divided into four stages: multiscale convolution (MSC), Gaussian Laplace sharpening and Liner Fusion (LF). First, six scale feature sub-graphs are extracted by convolution from six Gaussian convolution check images with different sizes and parameters. The the weighted fusion histogram equalization (WHE) is used for reference. The idea of WHE is to linearly fuse the feature sub-graph with the original image to obtain the convolution enhanced image C. Then, with the maximization of the variance within the class and the minimization of the variance between the classes as fitness functions, genetic algorithm (GA) is used to solve the threshold, and the image C is divided into two sub-graphs of the light layer and the dark layer. The two layers are equalized in the two regions by using the double-interval histogram with the introduction of details, and the two layers are linearly merged. At the same time, the image brightness is improved by means of mean and mean square deviation homogenization, and the image H with brightness enhancement and detail preservation is obtained. Secondly, Gaussian Laplace transform is used to extract the target contour and detail features in Image C. Finally, the target contour and texture detail image C are weighted and fused with the image H with brightness enhancement and detail preservation by using WHE idea as reference, so as to obtain the enhanced Image with clear texture, significant contrast and better visual effect. Deep learning mainly relies on multi-scale convolution operation to extract deep detail features of samples [35][36][37]. In this paper, using the idea of multi-scale convolution in deep learning for reference, six Gaussian convolutions with different scales and convolution kernel size of 3×3 are used to verify the convolution operation of low-quality images. The detail features of the image are extracted from 6 scales, and then the extracted feature sub-graphs is weighted and fused with the original image, namely, Where C I represents the fused image.

Threshold solution
GA is an intelligent optimization algorithm that simulates biological genetics and takes population as the research object. It can process data in parallel to achieve the purpose of quickly solving the optimal solution of objective function. It is very important to set fitness function in threshold solution. In this paper, based on the histogram curve of image gray scale, the fitness function is maximized with the intra-class variance and the interclass variance is minimized, and the fitness function is evaluated with the histogram of image as a variable. The fitness function  can be expressed as: Where, Where T represents the threshold value.  The improved Gauss-Laplace gradient edge detection steps in this paper are as follows: 1) The input image can be grayscale or color. The variable  is used to determine the Gaussian kernel in both directions. The appropriate  value is selected in the Gaussian gradient edge detection method.
2) Gaussian kernels are formed in x and y directions. The generated Gaussian kernel involves convolution of Gaussian function and the first derivative of Gaussian function.
3) Gaussian smoothing is performed on the image using the generated kernel. The Gaussian kernel in x and y directions is shown in equation (18).

Linear fusion
Image fusion is mainly to fuse two or more images with low image quality, details and textures, so as to get a better visual effect and high quality image with prominent details. Drawing on the idea of image fusion, this paper

Experiments and analysis
The experimental platform of this experiment: CPU Intel(R) Core (TM)-I7 9700K 8-core 3.6GHz, memory 16G. It runs on Windows10. The running software is Matlab R2017a. In this paper, infrared moving images with relatively dark brightness and gray images with rich detail information are selected for experiments in different scenes. Firstly, MSC and BABHE functions in the algorithm are analyzed. Then the validity of the proposed method is verified by comparison with the traditional method from quantitative and qualitative analysis. Firstly, the function analysis of each operation of the algorithm is carried out.

Multi-scale convolution
MSC is mainly to enhance the details of the image. The number of convolutional kernels, filtering radius and fusion weight will affect the enhancement effect of image details. Therefore, infrared images of moving scenes are used for experiments to set the number of convolutional kernels, filtering radius and fusion weight. However, when the number of convolution kernels is 9 and 6, the image quality is not significantly improved but the running time is longer. Therefore, the number of convolution kernels is set as 6, and the convolution radius is set as 0.1, 0.3, 0.5, 0.7, 0.9 and 1. The setting of fusion weight  will affect the image quality, and the value is usually 0.5-0.9. In this paper, moving infrared images are used for fusion experiment to determine weight  , and the experimental results are shown in figure 6. As can be seen from the figure, when  is 0.5 and 0.6, the brightness of the image is dark and the details are not prominent enough. When  is 0.8 and 0.9, the image brightness is significantly improved, but there is local overexposure. When  is 0.7, the image brightness exposure is moderate and the details are EAI Endorsed Transactions on Scalable Information Systems Online First

R E T R A C T E D
A novel Gauss-Laplace operator based on multi-scale convolution for dance motion image enhancement 7 prominent. Therefore, the weight  is set as 0.7 in this paper. Figure 6. Image fusion result with different β.
In order to further verify the effectiveness of MSC operation, moving infrared images are used for the experiment, where the filter radius of Gaussian kernel function of MSC is set as 0.1, 0.3, 0.5, 0.7, 0.9 and 1, respectively. The weight of linear Fusion is set as 0.7. Figure 7 shows the enhancement results of Histogram Equalization with Edge Fusion (HEEF) and MSC and their corresponding histograms of the original infrared image. The first column is the original image, the second and third columns are the enhanced results of HEEF and MSC respectively. As can be seen from figure 7(a)~(c), infrared images are clearer and image quality is improved after HEEF and MSC processing. Compared with the original image, both methods have better enhancement effect. Compared with HEEF algorithm, the brightness of MSC image is improved.
The same conclusion can be drawn from the image histogram in Figure 7(d)-(f). The histogram of the infrared image almost has no distribution after the gray value is about 200, and the gray value of image A still has distribution after the MSC processing, indicating the distribution of the gray value of the MSC extended image. In terms of detail enhancement, the red box in the infrared image is taken as an example to illustrate that the demarcation line of buildings in the original image is blurred, but the lines are clear and prominent after MSC processing. The experimental results show that MSC can enhance the image details while improving the image brightness.

Linear fusion
LF is mainly for linear superposition of two or more images with different characteristics, so as to obtain highquality images. However, the weight setting in LF will have a great influence on the final fusion image. Relevant literature shows that the best effect is when the weight of LF is set at 0.5~0.9. In order to determine the weight input, infrared images of moving scenes were used for experiments under different  , and the experimental results are shown in figure 8. As can be seen from the figure, the setting of weight  can affect the brightness and detail enhancement of the image. When  =0.5 and  =0.6, the image brightness is dark. When  =0.8 and  =0.9, details are lost and overexposure is obvious. When  =0.7, the brightness of the image is moderate and the details are rich. Therefore, the weight  is set to 0.7 in this paper.

Overall analysis
In this paper, moving infrared image with obvious edge and gray image with rich detail information are used for enhancement experiment. The performance of the proposed algorithm is analyzed quantitatively and qualitatively, and compared with BBHE, DOTHE, RLBHE, FCCE, Wan and HEEF to verify the validity of the proposed algorithm. A. Qualitative analysis Figure 9 shows the enhancement results of infrared images with obvious edge contouring by different The enhancement effect of RIBHE,BBHE and HEEF on the image is not obvious and the brightness is still dark, indicating that the image quality processed by these two algorithms is improved, but the contrast is still low. In terms of local details, the yellow boxes in the two images are taken as an example. As can be seen from the enlarged details in Figure 9, the details of DOTHE,BHE, FCCE and the image processed by this method are clear, but the brightness of detail images processed by BBHE and RLBHE is relatively dark. The detail image after HEF treatment is still dark and the detail image after Wan treatment is fuzzy. After processing by the method in this paper, the exposure of the window in the yellow frame of image A and the road in the yellow frame of image B are moderate and clearly visible. Therefore, compared with the contrast aspect, the algorithm in this paper has better enhancement effect, improves the brightness and global contrast of the image, enhances the image details and improves the image quality.  Figure 10 shows the overall effect and local detail effect diagrams enhanced by different algorithms for two gray scale images of people and animals with rich details and texture features. As can be seen from the figure, the brightness of images processed by DOTHE,FCCE and Wan is improved, but the yellow frames of images E and F are still blurred after DOTHE and Wan processing. After FCCE processing, the contrast of yellow area of image E and F is low, especially the goose's feet in the yellow frame of image F are still blurred. RLBHE,HEEF and BBHE images are dark and have low contrast. For image F, the feather texture was relatively prominent after HEEF treatment, but the feet were still unclear, while the goose's feet were still blurred after BBHE treatment and the feather texture was almost absent, and the image details were seriously lost after RIBHE treatment. The brightness and contrast of images E and F are improved after the enhancement processing, and the overall visual effect of the image is improved. In terms of detail enhancement, the yellow area of image E is obvious after processing the head of later generations. In image F, the goose's feet in the yellow frame are prominent and the feather texture is clear. En represents the average amount of information of an image, and the maximum value of image information entropy between [0, 255] is 8. In general, the higher information entropy denotes the better image quality. En is calculated as: (21) Where En represents the image information entropy. i represents the gray value of the image. ) (i p is the probability of image gray value i appearing in the whole image. EME represents the difference between the brightest and darkest gray values in the image, reflecting the contrast of the image. Generally, the larger EME denotes the greater contrast of the image, indicating the more significant the image enhancement effect is. Assuming that the image is divided into  (22) In addition, the running time is mainly used to measure the efficiency of the algorithm in image processing. This paper mainly evaluates the performance of the algorithm from the aspects of AG, En, EME and running time. Table  2 shows the AG, En, EME and running time of each algorithm for images A, B, E, F. As can be seen from Table 2, BBHE, DOTHE, RLBHE, FCCE, Wan and the method presented in this paper can all improve the AG of the image, and the AG of the image enhanced by the method presented in this paper is the largest, followed by Wan indicating that in terms of detail enhancement, the above methods can all enhance the detail information of the image. The En of images processed by HEEF and Wan is lower than that of the original image, while the En of images processed by other methods is higher than that of the original image, and the En of images processed by this method is the maximum, indicating that HEEF and Wan can reduce the image contrast, while other methods can improve the image contrast and the image processed by this method has a higher contrast. The EME of the image processed by Wan is lower than that of the original image, while the EME of the image processed by other methods is improved, indicating that BBHE, DOTHE, RIBHE and FCCE can improve the image quality while Wan reduces the image quality. In terms of running time, the method in this paper and Wan consume more time, while other methods consume less time and Wan method consumes more time, mainly because both methods in this paper and Wan need to use intelligent algorithms to process images, while other methods operate directly on the gray value of images. In summary, it can be concluded that the proposed method can solve the problems of low contrast, blurred details and poor image quality of infrared images.

Conclusion
In this paper, a dance motion image enhancement method based on multi-scale convolution and Gauss-Laplace operator is proposed. In this method, multi-scale convolution is used to enhance the detail information of the image, and the brightness and contrast of the image are enhanced by adaptive dual-interval histogram enhancement. Gauss-Laplace enhances the edge and contour of the image, and finally obtains high quality image through linear fusion. Through experiments of enhancing infrared images and gray images with rich detail information in different scenes, and comparing with other algorithms in subjective visual perception and objective evaluation, the results show that the proposed method can enhance image contrast and highlight texture details while improving image brightness. However, this method needs to use genetic algorithm to get the optimal solution, which leads to a long running time and low efficiency of the algorithm. The next stage will mainly solve this problem.