Non-local clustering via sparse prior for sports image denoising

Image denoising is very important in image preprocessing. In order to introduce the priori information of external clean image into the denoising process, a non-local clustering image denoising algorithm is proposed. A sparse representation dictionary is obtained by combining the image blocks of external clean image and internal noise image. The sparse coefficient estimation of ideal image is obtained by global similar block matching. Based on the class dictionary and the estimated sparse coefficient, a sparse reconstruction method based on compressed sensing technology is used to denoise the image. Experimental results show that compared with traditional image denoising methods, the proposed algorithm can significantly reduce the denoising block effect and preserve more details while transitioning more naturally in the flat area of the image.


Introduction
Vision is one of the important channels for human beings to obtain information. Image is an important medium for transmitting visual information. The ubiquitous noise in life reflects sound people's visual experience, the noise in the image will seriously affect other work in the image processing process, such as image fog, image feather, image coding and target recognition. So image denoising is an important and meaningful thing [1][2][3].
Strong noise images because of strong noise disruption, SNR is very low. The edges and details of the image are much more blurred than the ideal image. The human eye can only see the rough outline of the object, and the details cannot be identified. Because of the presence of strong noise, it is more difficult to denoise the strong noise images [4,5]. Strong noise denoising is still an understudied topic so far. At present, most of the image denoising algorithms do not specifically denoise for strong noise, and many denoising methods have poor results when faced with strong noise.
Many classical denoising algorithms have emerged in the process of image denoising studies. Depending on the scope, the image denoising algorithm can be roughly divided into two categories: spatial domain denoising and transformation domain denoising. Spatial domain filtering is to directly process the pixels of the image, and can be divided into local spatial domain filtering and non-local spatial domain filtering according to the way of candidate domain filtering [6]. The candidate block images selected by local spatial domain filtering are mainly concentrated near the target pixels, which is limited by the spatial distance. Common local spatial filters such as mean filtering, median filtering, etc. Non-local spatial filtering method makes full use of the non-local self-similarity of Ying Zhang 2 image filtering [7], such as the typical non-local mean denoising method, which is the mainstream algorithm of airspace denoising. It first uses the non-local selfsimilarity of the image, finds the similar block group to be denoised image blocks in the same image, and then performs weighted average processing for image denoising according to the similarity degree, which has achieved good results. The non-local self-similarity of images depicts patterns of image blocks spatially similar to each other, which are widely exploited by subsequent denoising algorithms and have been combined with denoising algorithms for the transformed domain. The transform domain method mainly uses the basis function, including a Fourier transform [8], a discrete wavelet transform, and a sparse representation method based on dictionary learning. Wavelets are a common orthogonal basis. The wavelet transform is a time-frequency analysis tool and image processing technology and is widely used in the field of image denoising. The wavelet decomposition transforms the image signal to the wavelet domain for multi-layer decomposition to achieve the effective separation of signal and noise. Such as the wavelet threshold denoising method and some improved algorithms. The wavelet threshold denoising method is a more popular wavelet denoising method, by the threshold processing and correction of the wavelet coefficient to achieve the denoising purpose. However, the wavelet transform-based denoising methods often bring additional artifacts into the denoising images because the fixed wavelet bases cannot accurately represent the structure. To overcome the disadvantages of using fixed base representations, the dictionary learning-based denoising method uses a set of base functions called dictionaries to obtain a more flexible image sparse representation, suppressing the noise by more accurately and sparsely representing the sparse components of image useful information, and achieving the goal of improving the denoising performance. The classical K-SVD (K-Singular Value Decomposition) denoising method, as proposed by Elad et al. a redundant dictionary with fixed atom size was used to represent a set of orthogonal bases learned by singular value decomposition (SVD) [9], then solving the convex optimization equation by orthogonal matching tracking algorithm, improving the learning dictionary was also a popular topic.

EAI Endorsed Transactions on Scalable Information Systems
In the transformed domain approach, the compression sensing based denoising algorithm is near ten years of research hotspot and has achieved very excellent performance. Noise can be suppressed well by squeezing sparse representations such as dictionary learned and sparse coefficient reconstruction by solving the convex optimization problem, thus obtaining high quality output images. The effectiveness of sparse can be explained from multiple angles, first of all, natural images are usually sparse, and common noise such as Gaussian noise, quantization noise is non-sparse, through sparse transformation, the signal and noise separation, facilitate denoising, the existing classical denoising algorithm such as Fourier transform, wavelet transform methods are based on this feature; Secondly, from the perspective of human visual response mechanism, the neurons of the main visual cortex satisfy sparsity when receiving natural images as input stimuli, that is, most neurons have weak response and only few neurons have strong response; Finally, from the perspective of compressed perception, the image block is a K sparse signal and can reconstruct well in the appropriate way. There are already many excellent and classical compression-sensing-based denoising methods. Among them, K-SVD, LSSC (Learned Simultaneous Sparse Coding) [10], EPLL (Expected Patch Log Likelihood) [11], WNNM (Weighted Nuclear Norm Minimization) [12], and NCSR (Nonlocally Centralized Sparse Representation) [13] are the landmark denoising algorithm. The feasibility of the K-SVD based pioneering approach proposed by Elad et al; the LSSC method by Mairal et al, a simultaneous sparse coding (SSC) model was proposed to place similar image blocks together for sparse representation. For the first time, image non-local self-similarity is combined to a compression-sensing-based denoising framework. The denoising of the denoising performance of the algorithm is improved. The WNNM method based on low-rank minimization assigned the different weights to the different singular values, made the soft threshold become more reasonable. The main information of the image was more effectively preserved. The EPLL denoising algorithm combining a Gaussian hybrid model (GMM) with a sparse representation used the GMM model [14] to learn external clean image prior-guided denoising processes, also achieved competitive denoising performance.
The landmark denoising method [15] was proposed by Dong et al. The noise was suppressed by minimizing the residue between the ideal sparse and the sparse coefficient to be sought. The dictionary was constructed using kmeans-based clustering and principal component analysis (PCA) methods. The NCSR method effectively improved the well-based denoising model of compression sensing. The constraint terms of the convex optimization equations in the denoising model were extended from the sparsity of the sparse coefficient to the sparse coding error. Yet, the NCSR algorithms still have limitations. When making a similar block matching, the limitations of the search window prevent it from searching for similar blocks in a global image. Similar blocks outside of the window are missed. The similar blocks causing matches are not textured similar. Meanwhile, the NCSR algorithm only utilizes the prior information of the noise-containing image itself, ignoring the usable external natural clean images. The substantial reduction in noise image prior information at strong noise leads in poor denoising performance. While natural clean images can make up for the insufficient prior information under strong noise.
To make better use of the prior information of natural clean images, by studying the limitations of the NCSR algorithm, we propose a non-local clustering image based on sparse priors. The denoising algorithm, first, introduces the image block of the external reference image, cluster together with the edge texture image block of the noise image, obtain better clustering effect to improve the dictionary, secondly, using the cluster-based global block matching strategy, the block matching method can search for similar blocks in the internal noise image and external reference image global, improve the accuracy of sparse coefficient estimation in the model to improve the denoising performance.

NCSR algorithm
The NCSR denoising model can be characterized with the following objective function: Where Y is a noisy image. H is an image degradation factor. D is constructed dictionary.  is the regularization The upper formula is divided into two terms, the previous one is the data fidelity term used to ensure solving the resulting denoised image  D X = similar to the overall image between the noise image Y, the latter term is the regularization constraint item to make the pending sparse coefficients with the estimated ideal clean image sparse. The smallest error between the coefficients and the minimal sparse coding noise obtains the purpose of noise.
The NCSR denoising process is mainly divided into 5 modules.
Module 1. Image block classification based on the kmeans algorithm. The image with M×N to be denoised divided into n n  multiple square image small blocks. n is the number of small pixels. Image blocks serve as a flat region class for texture edge class plots with large variance. Like blocks, using the k-means algorithm, these small blocks are clustered into Euclidean distance. The image blocks with similar distance will be divided into the same class, namely the structure or grain similar principles will tend to be in the same classification.
Module 2. PCA-based dictionary design. For Module 1, it obtains each cluster, the principal component analysis (PCA) method is used to obtain the correspondence adaptive dictionary D [16].
Module 3. Based on the non-local mean. The image block matching method for each image block to be denoised comes in the local window according to Euclidean distance. The similarity of each small block to the central block is measured, and the highest similarity is selected. The t blocks serve as the final similarity block q i X , with high similarity for each similarity block.
where W is a normalized variable. Module 4. Based on the maximum posterior probability estimate (MAP) [17] method, the estimator of regularization parameter λ is derived, as shown in formula (3).
Module 5. Based on the iterative contraction algorithm, the NCSR model is solved, the sparse coefficient  is obtained, and the denoised image X is estimated together with the dictionary D, as shown in formula (4): All the above steps are iterative. The algorithm flow in a single iteration is shown in Figure 1.

Non-local clustering image denoising algorithm based on sparse priors
By studying the limitations of the NCSR algorithm, this paper proposes that based on non-local clustering of the image denoising algorithm for sparse priors.The algorithm matches from the block both strategies and sparse prior sources are studied in order to achieve better the denoising performance. The prior information of the image can narrow down the image denoising problem. The solution range, so how to obtain more prior letters of the original image X interest comes to do denoising is a critical issue. This paper will be sparse a priori the source extends from the internal noise-containing image itself to the natural clean map outside. For example, sparse priors of internal and external images together guide the image denoising process. Meanwhile, in order to better utilize the internal and external prior information of the image, in expanding fig like the prior source, improve the block matching method from window block matching to base the global block of the cluster matches the noisy image as in a naturally clean image. All image blocks participate in the matching of similar blocks, improving the ideal sparse coefficient of the estimate effect.

1) Limitations of window block matching
The classical NCSR algorithm uses the block matching method based on local window search when searching the similar blocks of denoised image blocks. However, when the natural clean image is added to participate in the matching of similar image blocks, the relationship between external natural clean image and noisy image is not local and non-local. Therefore, the two can not be unified in a local window, so a new block matching strategy must be proposed, so that small blocks from natural clean images can participate in similar block matching. In addition, some image blocks with protruding structures (such as circular edges or corners) do not have repeating patterns in the neighborhood, and some image blocks matched from window regions with low repetition are not similar enough. It needs to expand your scope to search for more similar image blocks [18,19]. However, the large size of the window leads to high computational complexity, and the window cannot cover the global situation where similar blocks may exist at any location.

2) Global block matching policy
In order to solve the above problems, the method of clustering first and then performing block matching in the class is proposed. Specific as follows. For from the variance of the noise image Y with M×N size, or on the basis of adding all small pieces from natural clean image, using the k -means algorithm, and put these small pieces of clustering, Euclidean distance of similar tiles will be divided into the same class, that is similar will tend to be in the same structure or texture classification.
Where, h is the filtering parameter.  (c), the estimated block comparison diagram is reconstructed based on the two similar block groups using the method shown in formula (7). It can be seen from Figure (c) that compared with window block matching, the estimated blocks obtained by clustering based global block matching are more similar to clean blocks. This is because the clustering based global block matching method can have a wider search range of similar blocks than window block matching. Therefore, image blocks that are more similar to the texture of denoising center blocks can be found, and then the estimated blocks obtained by using the weighted average strategy can retain more texture information. However, window block matching is easier to blur in weighted average estimation due to the larger deviation of similar block texture.   Table 1 compares the structural similarity measure (SSIM) values of 16 similar blocks obtained by two block matching methods and denoising center blocks corresponding to figure 2. The SSIM value of the estimation block and denoising center block obtained by global matching is higher than that of window matching, indicating that the structural texture is more similar. It proves that global matching has better performance in preserving structural texture than window matching. Based on clustering, global block matching can limit the number of matches by clustering, and search for similar blocks globally in internal and external images, which improves the performance of window matching. Especially under strong noise, local similarity information is reduced, and global matching includes global (including external image) similarity information, which will improve the processing effect of the algorithm. The texture similarity between the final similar block and the denoising center block is higher.

Introducing external reference image sparse priori
The rationality analysis of external image sparse priori The denoising methods based on sparse clustering, whether sparse representation based on dictionary learning or clustering strategy for image blocks, are expected to better learn the prior information of the original image X. The prior information of images can come from different sources. Generally speaking, the learning dictionary based on natural image library takes up more time in the early training stage, and learning dictionary from the image to be denoised has stronger adaptability to denoising images. However, it is undeniable that no matter in the natural clean image or the noisy image itself, the priori information of ideal image can be obtained for denoising.
NCSR algorithm only obtains the prior information from the noisy image itself. In the case of strong noise, the noise image is seriously damaged by noise, the texture structure becomes blurred, and the available prior information becomes less and less with the increase of noise, resulting in poor final denoising performance. On the contrary, the introduced external reference image does not change with the change of the noise level of the noise image and has better robustness to noise. By providing similar image blocks without noise, the external reference image can promote the estimation of the parameters of the sparse model, such as the ideal sparse coefficient  in the algorithm in this paper. At the same time, the addition of external reference images can strengthen the similarity of image blocks obtained by clustering, improve the clustering effect, and thus strengthen the low-rank characteristics. The low rank feature is proved to be efficient in many image reduction problems, which is beneficial to the image denoising process.
In some cases, before obtaining a noisy image, there may already be some low-noise or even no-noise images in the case. With high similarity of image structure, these low-noise images can provide effective reference information for denoising of noise images in the same scene and improve denoising performance. For example, images of the same scene but taken at different times or from different viewpoints. More specifically, such as in monitoring equipment, monitoring image due to its low light characteristics of shooting at night with strong noise, but the images taken with occasions during the day, relative to the image of strong noise at night can is worth reference for low noise image, can give a night high noise image denoising process guidance.

Denoising method combining internal and external image sparse priori
In order to combine the two prior information sources of natural clean image and noisy image, this paper adds "clean" natural image to participate in the construction of learning dictionary and matching of similar blocks to improve the denoising performance.
The specific steps are as follows. For the given i-th For the image blocks from the flat region class of noisy images, after obtaining dictionary D by using principal component analysis (PCA), the original window-based block matching method is used for block matching. There is no need to use the global block matching strategy for the image blocks of the flat region class, because the small blocks from the flat region contain less edge texture information, enough other flat similar blocks can be searched in the window, and there is no need to expand to the whole image to search. For natural images, flat area generally accounts for a large proportion, and the block matching method based on small window can greatly reduce the running time of the algorithm.
For the image blocks from the texture edge class in the noisy image Y, the k-means algorithm is used together with all the small blocks from the clean image c Y to obtain different texture categories. To make noise from noise image edge texture class center piece more match to the reference images from clean c Y similar piece, the images from the clean reference images small pieces don't classify the edge texture and flat area, all the images of small pieces in clustering division and block matching, prevent similar to block the leakage of retrieval.
When the k-means algorithm is used for classification, this structure is considered effective only when the number of image blocks contained in the class reaches a certain number. Therefore, the clustering algorithm fuses the small clusters in the later stage to eliminate the influence of noise and outliers on clustering, ensure that the final clustering is on the main core features of the image, and also reduce the influence of uncertainties caused by the center sensitivity of k-means on the algorithm effect. At the same time, the number of clustering center k is usually preset to an appropriate value. According to the image data scale and the setting value of the classical algorithm as a reference, the initial value of the preset number of centers is 70, and the final optimized clustering number is roughly between 40 and 60. The clustering results show that the k value can fully present the main core features of the image.
Suppose that the image blocks are divided into k classes in total, and for each class of image blocks, PCA After obtaining all the similar blocks and corresponding weight information, the maximum posterior probability estimation (MAP) method is used to estimate the regularization parameter  , and then the algorithm based on iterative contraction is used to solve the model to obtain the de-noised image X. The calculation is shown in formulas (3) and (4). Figure 3 shows the single-iteration de-noising flow chart of the non-local clustering image de-noising algorithm based on sparse priori.  The non-local clustering image denoising algorithm based on sparse priori uses the global block matching strategy based on clustering rather than window block matching to obtain image blocks with more similar textures and retain more details in the denoising results. At the same time, under the strong noise, the effective information of the image is almost covered by the strong noise, so it is very difficult to recover. The proposed algorithm can search similar blocks in a wider range, and the quality of similar structures is higher than that of local search, so it is more suitable for the situation where the image quality is seriously damaged under strong noise.
As the noise gradually increases, the prior information obtained from the noise image becomes less and less, while the prior information of the expanded external reference image does not decrease because of the increase of noise, which can be used to suppress the interference of strong noise and play a greater role in the process of standard denoising. Therefore, compared with algorithms that only use the noise image itself for denoising, such as K-SVD and NCSR, the algorithm presented in this paper has more significant denoising performance for strong noise images, better suppression ability and stronger robustness for strong noise.

Evaluation index and parameter setting
In order to verify the effectiveness of the proposed algorithm, the denoising algorithm is tested in standard test set images and real noise images respectively. The test results are compared with the classical denoising methods K-SVD, EPLL, BM3D and NCSR. Peak signalto-noise ratio (PSNR) and structural similarity measure (SSIM) are used as objective evaluation indexes for image denoising performance, and PSNR is calculated as follows: ) )( ( Where, X  and Y  are the mean value corresponding to images X and Y.  All experiments are carried out in Windows10 system based on Inter i7-4160CPU, 64GHz processor and MATLAB R2017a software. K-SVD, EPLL and BM3D algorithms run with the default parameters in the reference. The NCSR algorithm and the new algorithm in this paper use the search radius of 15 when designing the window block matching process, and the default experience values in the NCSR algorithm are used for setting other parameters.

Experimental results and analysis on standard test images
The experiment selects five standard test images (cameraman, peppers256, house, boat, monarch-full) with size of 256×256 to make denoising experiments. Different levels of Gaussian white noise with mean value 0, standard deviation  =20, 30, 50, 70 and 100 are added to the test image to be de-noised. Table 2 shows the comparison of PSNR and SSIM results of denoising experiments of various denoising algorithms. The bold values in the table represent the optimal values of the five denoising algorithms. As can be seen from the data in Table 2, K-SVD algorithm has the worst performance at all noise levels. When the noise is low ( =20), the PSNR and SSIM values of the proposed algorithm are slightly lower than those of EPLL,BM3D and NCSR algorithms. However, with the increase of noise intensity, the algorithm is convex. In strong noise (  ≥50), the PSNR and SSIM values of the proposed algorithm are higher than those of EPLL and NCSR algorithms. The PSNR values of the proposed algorithm are close to those of BM3D algorithm under strong noise. However, the SSIM value of the proposed algorithm is higher than that of BM3D, indicating that the proposed method has better performance in detail retention. Table 2 shows that compared with weak noise (  <50) image, the algorithm in this paper in strong noise ( ≥50) The denoising performance advantage under the image is more obvious, this is because with the gradual increase of noise, the image prior information provided by the noise image is less and less, while the external natural clean image as a reference image can be. The more prior information you have to compensate for. Therefore, compared with the denoising algorithms such as NCSR and EPLL, which only rely on the prior information provided by the noise image itself, the proposed algorithm has better robustness to noise. In terms of subjective evaluation, test results of three standard images (Boat, monarch-full, peppers256) under strong noise ( =100) were selected as representatives to display, as shown in figure 4. In order to make the contrast effect more obvious, zoom in on the small green box area in each picture and place it in the corner of the image for observation. As can be seen from Figure 4, due to serious damage of strong noise, some details in the original image cannot be restored in the results of various denoising algorithms listed above, which is inevitable with the increase of noise level.  By comparing the subjective effects of several denoising algorithms, it is found that K-SVD algorithm has the worst denoising performance because of the overall blurring and the loss of more image details. Some visual artifacts are not found in the original images after denoising by EPLL and BM3D algorithms, and the visual artifacts of EPLL algorithm are more obvious than that of BM3D algorithm. NCSR algorithm has a smooth phenomenon on the whole, so part of the detailed texture information is lost. At the same time the algorithm on the edge of the image of a part of the texture region will appear obvious Mosaic plaque, this is because all areas in an image using NCSR algorithm based on block matching window, the window has restricted its on the edge of the part of the texture region matching to the similar block similar to denoising center piece is not enough, caused by the similar estimates the estimated blocks is not close to ideal image block is even larger Deviation, after denoising in the corresponding position will appear Mosaic plaque phenomenon; Compared with other denoising methods, the proposed algorithm can retain more details of the original image, while the flat area transitions naturally, and the visual artifacts are not obvious compared with other algorithms. Because clustering based global block matching is used, more similar blocks can be matched, and the phenomenon of Mosaic patches is greatly reduced compared with NCSR. By comparison, the proposed algorithm has the best subjective denoising effect on standard images.

Test results and analysis in real images
In order to further verify the feasibility of the proposed algorithm, denoising experiments were carried out on four real noise images (Grey, MCC, scene1, 12233) with a size of 608×800. The real noise image is captured under low light conditions, the overall image is dark and the SNR is low. In order to eliminate the influence of bad points in the process of denoising and obtain a better visual experience, it is necessary to preprocess the image before denoising, and carry out simple denoising and image enhancement. The pretreatment process and effect are shown in Figure 5. The reference image used in this algorithm is a clear image obtained under sufficient exposure, as shown in Figure 6. Since the original clean image of the algorithm is unknowable, the experimental results are only compared and evaluated on the subjective effect. The results of denoising are shown in Figure 7.   The denoising effect is similar to the comparison effect obtained on the standard test graph. It can be observed from Figure 7 that the denoising result of K-SVD algorithm is the fuzziest. EPLL and BM3D still have extra visual artifacts, but they are less obvious because the test images are dark. NCSR has a smoothing trend. In irregular texture areas such as characters and single lines, the texture details obtained by the algorithm in this paper are clearer than other algorithms, and more details can be retained.
However, in some areas with a large number of regular stripes, the experiment result shows NCSR with BM3D algorithm has more clearly than algorithm in this paper the relative stripes, this is because in these areas in the local window already contains abundant since the repeat information, does not need in the whole image and external reference images to obtain similar blocks, only in order to make similar piece can guarantee in a window and go The texture similarity of noise center block is high enough. At the same time, in these regions, due to the interference of noise, the global block matching method will find some "similar blocks" in the region outside the global but window of the image, and the imaging features of these "similar blocks" are similar to the noise features in the flat region. But the texture is not as similar as the similar blocks obtained by using window block matching. In other words, the global block matching will make the image block with similar noise but less similar texture replace the image block with extremely similar selfrepeating texture in the window, resulting in the smooth texture part of the algorithm in these regions, and the degree of detail retention is not as good as that of NCSR and BM3D algorithms. Compared with K-SVD, EPLL, BM3D and NCSR algorithms, the proposed algorithm has better strong noise denoising performance in most areas of the image, and has better effect on the actual strong noise image denoising.

Conclusion
This paper studies the limitations of NCSR algorithm. We propose a non-local clustering image denoising algorithm based on sparse priori. By combining the sparse priors of external reference image and internal noise image for denoising, the source of image priors is expanded. At the same time, the clustering-based global block matching strategy is applied to the denoising method based on sparse coefficient reconstruction, which makes the similar block image more similar to the denoising center block image, and makes more reasonable use of the selfsimilarity of image. Experimental results on standard test images and real noise images show that compared with the classical denoising methods k-SVD, EPLL, BM3D and NCSR, the proposed algorithm can retain more details and have less visual artifacts. Especially under strong noise, SSIM and PSNR have leading or competitive denoising performance. Although the algorithm can preserve details on most edge textures well. However, in the fringe window region with high self-repetition, the image features are similar to the noise features of the smooth region due to the fine and strong noise damage of the fringes, and the algorithm treats the fringes as the smooth region filtering, so the reservation degree of texture is insufficient. How to further improve the restoration effect of fine texture under strong noise is also the key research direction in this field in the future.