Extending Color Properties for Texture Descriptor Based on Local Ternary Patterns to Classify Rice Varieties

In this study, a proposed descriptor based on the improved local ternary patterns (ILTP) that also uses the color properties of rice varieties is presented. Not only gray-scale intensity, but R, G, and B color components of the rice grains are considered. Combining a support vector machine (SVM) with the proposed descriptor for classification of 17 rice varieties planted in Vietnam gives an overall accuracy of 95.53%. To evaluate and compare the e ff ectiveness of the proposed descriptor with other analysis techniques for rice varieties classification, texture descriptors based on local binary pattern and local ternary patterns are combined with SVM to classify these 17 rice varieties. Experiment results show that the classification accuracy from the SVM using the proposed descriptor is significantly higher than using ILTP or other texture descriptors from the literature. This technique can be used to build an automatic system of rice varieties identification and classification.


Introduction
Rice varieties are diverse to deal with diseases, climate change, and pests, so knowing their characteristics is necessary. However, external observation of grains from different varieties of rice reveals that they are quite similar, so classification and identification from measuring their shapes and sizes, or through only visual observation, is time consuming and often leads to inaccurate results. Therefore, an automatic rice classification method is necessary for rice gene bank centers, farmers, and researchers. Therefore, this study focuses on texture analysis to enhance the classification accuracy of rice varieties.

Related Work
Texture is important information from an image, and it is applied in many fields, such as pattern recognition, medical image analysis, and image retrieval [10]- [16]. Texture analysis is usually categorized into four main methods: statistical, structural, model-based, and transform-based [17]. In another more detailed study [18], the authors classified texture feature extraction methods into seven categories: statistical approaches, structural approaches, transform-based approaches, model-based approaches, graph-based approaches, learning-based approaches, and entropybased approaches.
Statistical methods for texture, including basic descriptors and the gray level co-occurrence matrix (GLCM), were used in some research on rice variety classification. Statistical descriptors such as mean, standard deviation, uniformity, and third moment of gray-scale intensity, along with color and morphological features, were applied for classification of six different rice varieties [7]. The highest classification accuracy (90.54%) was obtained when these features were combined with a random forest model. Texture features including GLCM and geometric features were combined with a multiclass SVM to classify six varieties of rice from western Odisha [19]. In [8], the authors used a neural network and combined 15 texture features in a GLCM, 13 morphological features, and 6 color features to classify nine rice varieties. In these two studies, the GLCM was calculated from four directions (0 0 , 45 0 , 90 0 , 135 0 ), and both studies achieved overall classification accuracy of 92%.
For more classes, Haralick texture features of a GLCM were extracted from edge images of bulk samples to classify 15 paddy varieties [20]. However, the accuracy rate was not high (87.8%). In [21], seven texture features including brush ratio and six GLCM features (mean, variance, uniformity, entropy, contrast, and correlation) were combined with morphological features, color features, and Fourier descriptors to classify 30 rice varieties, giving accuracy of 89.1%.
To compare the effectiveness of some texture methods for rice classification, the authors in [22] extracted features including a GLCM, a grey-level run length matrix, a grey-level size zone matrix, a neighborhood grey tone difference matrix, and wavelet features. In addition, three sets of texture features (SET-A, SET-B and SET-C) were formed from four different proposed schemes. The performance of feature SET-B achieved average accuracy of 99.63% using a backpropagation neural network. In [23], the authors compared classification accuracy with six rice varieties by using hand-crafted descriptors and convolutional neural networks. The results show that the DenNet21 framework achieved the highest accuracy at 99.04%, while the highest classification accuracy from handcrafted descriptors was 84.10%, obtained from a histogram of oriented gradients.
Besides statistical methods such as histogram properties and co-occurrence matrixes, local binary pattern (LBP) is widely used because of its simple implementation, improved high classification accuracy, and independence from rotation. Some methods were developed from LBP to enhance its robustness and discrimination against image rotation and noise. Besides considering sign features like the traditional LBP, the authors in [24] proposed a completed LBP (CLBP) scheme by combining sign, magnitude, and central value features for rotation-invariant texture classification. The authors in [25] proposed a method named featurebased LBP (FbLBP) for rotation-invariant texture classification. FbLBP features are calculated from the sign and magnitude, where magnitude includes the mean and variance of the magnitude vector. The authors in [26] developed a method of texture classification named the binary rotation invariant and noise tolerant (BRINT) method. Similar to CLBP, the BRINT method includes three descriptors: sign, magnitude, and central value. However, the number of sampled points is restricted to eight on any radius for the sign descriptor. Magnitude descriptors were computed with a locally varying threshold. Moreover, these descriptors were calculated with uniform patterns inspired by the LBP ri P ,R [27] method. Therefore, the robustness of the proposed approach is to process effectively for image rotation and noise. To overcome sensitivity to a noisy image under LBP, the authors in [28] developed the median robust extended LBP (MRELBP) by comparing medians of regional images rather than raw image intensities.
However, the LBP methods only used two-valued codes (0 and 1) with a threshold value at exactly the central pixel, so they are sensitive to noise. Therefore, the authors in [29] introduced a generalization of LBP named local ternary pattern (LTP) to increase the discriminant and decrease the sensitivity to noise in uniform regions. Some recent research was based on LTP in order to develop texture descriptors for the pattern recognition problem. Some attempts proposed extensions of LTP to solve the texture classification problem [30]- [35]. Some texture descriptors based LTP were proposed for face recognition, such as prominent LTP [36], relaxed LTP [37], adaptive LTP [38], co-occurrence of adjacent sparse LTP [39], improved gradient LTP [40]. Other research was based on LTP to develop texture descriptors for smoke detection, medical image analysis, fingerprint vitality, fall detection, and blur detection and segmentation [41]- [48]. Different from LBP, traditional LTP features 2 EAI Endorsed Transactions on Industrial Networks and Intelligent Systems 11 2021 -04 2022 | Volume 9 | Issue 30 | e1 Extending Color Properties for Texture Descriptor Based on Local Ternary Patterns to Classify Rice Varieties are calculated from three-valued code (-1, 0, and 1). However, these three-valued codes could be assigned to other values, as they were in [30,32,37] when comparing the value between a central pixel and a neighbor pixel against a threshold value. Besides, selection of the suitable threshold value can improve the accuracy rate in some research. The authors in [38] set the threshold value of LTP for face recognition automatically according to Weber's Law. In [33], the threshold value is the median absolute deviation of the gray-level values in a 3×3 local region. While LTP features are calculated with only sign components, the authors in [42] proposed completed LTP (CLTP) approaches calculated from sign components, magnitude components, and central components for more discrimination. Instead of comparing values of central pixels and neighbors in a 3×3 local region to obtain the binary pattern as LTP method, the authors in [35] apply Frei-Chen masks in eight directions, and the second derivative of a Gaussian filter for the central pixel, and then calculated the lower and upper components of the local directional ternary pattern (LDTP). The authors in [39] combined LTP and a GLCM to form a texture descriptor named CoALTP for texture and face image retrieval. The approach applies LTP to the given image, and then uses a GLCM in four directions for upper and lower components of LTP. To enrich texture information for uniform LTP, the authors in [44] computed feature histograms from the upper and lower binary patterns from eight orientations of 0, 45, 90, 135, 180, 225, 270, and 315 degrees to form eight histograms. Then, a feature vector is formed by concatenating these eight histograms.
A histogram of patterns is considered as extracted features in these texture descriptors, but it usually has computational complexity and is sensitive to rotation. Therefore, the proposed method, called an improved local ternary pattern (ILTP) [49] applies a uniformity measure, as done in [50] to the binary pattern, so this method could be resistant to rotation sensitivity, and significantly decrease the total number of extracted features. However, ILTP is considered for gray-scale intensity while the color components of an image contain a lot of useful content for classification. Therefore, our study is based on ILTP to extend color properties for rice images. The proposed method considers not only gray-scale intensity but also R, G, and B components to build a descriptor for rice variety classification.

Main Contributions
In this study, an SVM is applied to texture features for classification of 17 rice varieties planted in Vietnam. This descriptor considers gray-scale intensity as well as R, G, and B components to enhance the accuracy of rice variety classification. In addition, this descriptor resists noise and sensitivity to image rotation. The framework of this study is illustrated in Fig. 1. Rice images are pre-processed to separate them into individual rice grains. A proposed texture method is applied to each rice grain to generate a feature set of each rice variety. These feature sets are used with the SVM to classify 17 rice varieties. To compare the classification ability of the proposed descriptor with other texture descriptors, experiments were carried out using the SVM and some other texture descriptors from the literature for classification of the 17 rice varieties.
Our main contributions are presented as follows.
• An image dataset of the 17 Vietnam rice varieties is collected and acquired by scanner.
• We develop a texture descriptor based on the ILTP method for not only gray-scale intensity but also color properties of the 17 rice varieties.
• Experiments are performed to select parameters for the texture descriptor, including a threshold value and the radius of the neighborhood.
• Experiments are performed for classification of these 17 rice varieties by an SVM using the proposed descriptor and other texture descriptors from the literature. The results show the proposed approach achieved higher performance in comparison with the literature approaches.
The rest of this paper is organized as follows. LBP and LTP is presented in Section 2. The proposed texture color descriptor for rice varieties is presented in Section 3. Use of the SVM is detailed in Section 4 and its performance is evaluated. Section 5 presents the image dataset of the rice varieties under study, while the experiment results are discussed in Section 6, followed by conclusions in Section 7.

LBP
LBP describes the spatial structure of a local image texture and was first introduced by Ojala et al. [51]. This descriptor is independent of rotation, has a simple implementation, and extracts proper features to achieve high classification accuracy [17]. LBP code of a given pixel is only based on sign information computed by comparing the pixel value with its neighbors, as seen in equation (1): where g c is the gray-scale intensity of the central pixel, g p is the value of its neighbors, P is the number of neighboring points, and R is the radius of the neighborhood. The sign function s(x) generates binary values and is defined in equation (2): The output binary values are multiplied by the weights according to indexed neighbors to obtain a decimal number for the LBP. This operator is carried out for all local patterns of the image, and then, LBP values are constructed to form the histogram. However, the LBP descriptor is variant to rotation, because when an image is rotated, indexing of neighboring pixels changes. In order to eliminate the operator's sensitivity to rotation, a method for a rotation-invariant LBP ri P ,R [27] was proposed. The bit rotation is applied to the output binary pattern, and an intermediate LBP value is computed for each rotation. Finally, the LBP value is determined as the minimum value of all immediate LBP values. The rotation operator is computed with equation (3): where ri represents the rotation invariant operator, and ROR(LBP P ,R , i) denotes the local binary pattern rotated circularly to the right i times. However, this approach has a high computational complexity, especially when increasing the number of neighboring points. Therefore, another improvement by Ojala et al., named uniform MLBP [50] was proposed. In that approach, a uniformity, U , represents the number of changes from 0 to 1 (and vice versa) in the output binary pattern. U is calculated as seen in (4). For example, the U value equals 0 in 00000000 and 11111111 patterns but equals 4 in the 11110110 pattern.
Relying on the U value, the patterns are categorized into two groups: uniform patterns and non-uniform patterns. Uniform patterns include patterns that have a U value less than or equal to U T , whereas non-uniform patterns have a U value more than U T . Up next, labels 0 to P are assigned to uniform patterns, and P + 1 is assigned to non-uniform patterns, as seen in equation (5):

LTP
Although LBP is a simple method, and is rotation invariant, it is sensitive to noise. Therefore, LTP [29] is an extension of LBP to improve robustness in local coding. Instead of using two-valued codes, 0 and 1, like LBP, LTP uses three-valued codes (-1, 0, and 1) to assign to neighbors when comparing their values to the central pixel, as seen in equations (6) and (7): where s_LT P (x) is the threshold function, and t is the threshold defined by the user.
By using three-valued codes, LTP code has more discriminative features than LBP. However, the dimension of the built LTP histogram is very large when using 3 n value codes. For simplicity, the ternary code is divided into the upper and lower codes in equations (8) and (9): where s L (x) and s U (x) denote the threshold functions to calculate lower and upper LTP patterns, respectively. These two histograms of upper and lower LTP patterns are connected to form a single histogram. In order to decrease the number of bins in the histogram, the authors in [49] proposed an improved local ternary pattern (ILTP) texture descriptor for bark classification. After the ternary code is divided into upper and lower codes, this method is based on uniformity measure U to label both upper and lower LTP patterns, as in the uniform MLBP method [50]. Therefore, upper patterns are classified into two groups: uniform and non-uniform. Similarity, lower patterns are classified as uniform and non-uniform. In the results, upper patterns are labeled 0 to P + 1. Also, lower patterns are labeled 0 to P + 1. Finally, feature vectors are calculated as the occurrence probability of the labels of the upper and lower LTP patterns. An example of calculating the ILTP pattern with R = 1 is presented in Fig. 2.

Proposed texture color descriptor for rice varieties classification
The ILTP descriptor can improve classification accuracy compared to other texture methods for bark classification [49]. However, authors have only applied this approach to analysis of texture features for gray-scale intensity, but have not yet considered the color properties of images. Therefore, the descriptor proposed in this study not only inherits the attributes of ILTP but also analyzes color properties of rice grains. The procedure with the texture color under the proposed method is illustrated in Fig. 3.
Color images of each rice grain are converted to gray, R, G, and B components, called I k , where k = 0, 1, 2, 3. The ILTP method is applied to each gray, R, G, and B component individually. Consider a local region with P neighbors at central pixel (i, j), and radius R for image component I k . Let Lb P ,R (h) and U b P ,R (h), respectively, denote the lower binary code and the upper binary code of the pattern, where h = 0, 1, 2, ..., P − 1. Lb P ,R (h) and U b P ,R (h) are calculated with the following equations: and for which s U and s L are computed with equation (8) and (9).
Uniformity measure U is computed for upper and lower patterns individually, as seen in equations (12) and (13): Up next, upper patterns and lower patterns are labeled from 0 to P + 1 with equations (14) and (15): and .., f l(P +1) ) and .., f u(P +1) ), respectively, be the occurrence probability of labels assigned to lower and upper patterns. Elements of feature vectors f le and f ue , where e = 0, 1, 2, ..., P + 1, are calculated with equations (16) and (17): and f ue = N c ue N total (17) where N c le and N c ue denote the number of lower and upper patterns, respectively, that are assigned label e of component image I k , and N total is the total number of pixels in the rice grain image. The output feature vector of each image component I k is generated by concatenating the two feature vectors F L_I k P ,R and F U _I k P ,R . Also, output of the gray, R, G, and B components are concatenated into a vector that describes the total texture color features of each rice grain.
In the ILTP procedure, uniform patterns are labeled 0 to P while all non-uniform ones are labeled P + 1. In order to increase the distinction between texture patterns, the number of non-uniform patterns should occur in a smaller proportion. We refer readers to [52], where the authors selected a value for U T equal to P /4, 5 EAI Endorsed Transactions on Industrial Networks and Intelligent Systems 11 2021 -04 2022 | Volume 9 | Issue 30 | e1 and where the number of patterns labeled P + 1 was very low. For selecting threshold value t and number of neighbors P , the trial-and-error method is used in our work, which improves the classification accuracy of the rice varieties.
Regarding the dimension of the feature vector, the number of features decreased owing to applied uniformity measure U for ternary patterns. In the proposed method, the number of features equals 2(P + 1) for each component image. Therefore, the number of features in the total vector equals 8(P + 1). The proposed descriptor is calculated in Algorithm 1.
The advantages of the proposed descriptor are as follows.
• Inheriting the advantages of the LTP and ILTP methods, the proposed method is invariant to rotation, decreases sensitivity to local impulse noise, decreases the number of extraction features, and extracts high discriminative features.
• Besides extracting texture features for gray-scale intensity, the proposed method extends texture analysis to color properties.
• Extracted features from the proposed method are quite effective for classification of rice varieties, which is presented in the results section and in comparisons with the literature.

Support vector machine
The support vector machine [53,54] is a classifier that uses a decision boundary, namely, a hyperplane, to separate data into two classes. Two sides of these two classes are generated by support vectors, and a margin is defined as the shortest distance from one side to the other. The equation of the hyperplane is presented as (18 for i = 0 to (M − 2R) 5: for j = 0 to (N − 2R) 6: Calculate lower binary code Lb P ,R , 7: upper binary code U b P ,R . 8: Calculate U (U LT P P ,R ), U (LLT P P ,R ). 9: if U (U LT P P ,R ) ≤ U U T 10:

12:
if U (LLT P P ,R ) ≤ U LT 13: Lb P ,R (h). 14: else L_I(i, j) k P ,R = P + 1. where P = {p 1 , p 2 , ..., p v } is a weight vector, in which v is the number of attributes; Z is an input vector, and q is a scalar. Let two input attributes be A 1 and A 2 , and training tuples are 2-D, where z 1 , z 2 , respectively, are the values of attributes A 1 and A 2 for Z. If q is considered an additional weight, p 0 , equation (18) can be rewritten as follows: The support vectors of each class can be written as and where y i corresponds to the respective class. The best hyperplane and support vectors are searched so that the margin is achieved at the maximum value. The optimal hyperplane is searched by minimizing equation (22): such that The maximum marginal hyperplane can be written as where y i is the class label, Z T is a test sample, α i and q 0 are numeric parameters, and l is the number of support vectors.
When the data are linearly inseparable, the original training data are transformed into a higher-dimension space to find a linear separating hyperplane by applying nonlinear mapping. In order to decrease computation complexity in the higher dimension, kernel function K(Z i , Z j ) is used as follows: where φ(Z i ) is the nonlinear mapping function. The radial-basis kernel function is applied in our work, as seen in equation (26).
where Z i is the center, and σ is a free parameter defined by the user.
A one-versus-all approach can be applied in cases of multi-class classification. n SVM binary classifiers are applied to train for n classes. A positive value is assigned to class i, and a negative value is assigned to the rest through training. The positive distance is computed for each testing sample. Finally, a sample is categorized into the class that has the largest positive distance.

Evaluation of classifier performance
Accuracy classification. In order to evaluate classification performance, overall accuracy (ACC) is applied in this study: Confusion matrix. A confusion matrix is used to evaluate the ability to recognize samples from classes of the classifier. Table 1 is an example of a confusion matrix with two classes named C 1 and C 2 . where CM ij indicates the number of samples in class C i that were labeled by the classifier as class C j . 7 EAI Endorsed Transactions on Industrial Networks and Intelligent Systems 11 2021 -04 2022 | Volume 9 | Issue 30 | e1

Image acquisition
In some research applying image processing for classification of rice varieties, the authors used a camera to photograph the rice. But that method usually requires setting up a chamber for capturing the images. Therefore, the authors in [55,56] used a high-resolution scanner to capture rice grain images. In our work, a scanner set to a resolution of 2400 dots per inch (dpi) was used to scan images of the 17 varieties of rice.

Image pre-processing
Visual Studio C++ and the Open Source Computer Vision (OpenCV) library were used for image processing in this study. Rice images were scanned containing five rice grains of the same variety. Because this study extracted features for each grain, the rice regions were extracted from a blue background using chroma separation, and then, each grain was separated into an individual rice image. Rice image and results after the pre-processing steps were presented in Fig. 5.

Classification accuracy for the 17 rice varieties
To evaluate classification ability of the proposed descriptor, extracted features were combined with SVM to classify the 17 rice varieties. 3400 rice samples are divided randomly into 50% for training and 50% for testing. Parameters of the SVM were selected to enhance classification accuracy with the rice varieties. The cost parameter was selected from 10 values of 2 d , where d = 1, 2,...,9. The kernel function is a radial function.
In order to find a suitable radius for R parameter of the proposed descriptor, an experiment was carried out with the R value ranging between 1 and 7. Classification accuracy with the 17 varieties of rice is presented in Table 2. As seen in Table 2, the lowest accuracy rate obtained was 72.94% with R = 1, and this rate significantly increased when the radius of the neighborhood became larger. With the considered R values, the highest accuracy achieved was 95.47% with R = 6.
Because the proposed method extracts texture features for gray, R, G, and B components, the number of features is four times that of the ILTP method. Despite increasing the number of features, compared to the original ILTP method, color properties of rice varieties are considered in the proposed descriptor. In addition, the threshold value used to calculate lower and upper binary code can be affected for discrimination of the feature sets. Therefore, to enhance classification accuracy of the rice varieties under the proposed method, a selected threshold value for t was considered for the best result. The experiment was carried out with t = 3, 5, 7, 9, and 11 for the proposed method, with R = 6. The obtained descriptors used the SVM to classify the 17 rice varieties. The results are in Table 3. With the threshold values in Table 3, the highest classification accuracy was 95.53% with a threshold value of 9. From results of the above experiments on the proposed method, we selected 6 as the radius of neighborhood R and t = 9 for the threshold value. Fig.  6 is the confusion matrix from classification of the 17 rice varieties when using the proposed texture color descriptor and the SVM. In the confusion matrix, the classification accuracy of OM5451 and IR6976 was 83% and 87%, respectively, lower than other varieties. The number of samples of class OM5451 was mislabeled to OM429, IR6976, and IR50404 quite much, respectively ten, three, and three samples. Similarity, five and six samples of class IR6976 were incorrectly assigned to OM4218, OM6976, respectively. As can be seen in Fig.   8 EAI Endorsed Transactions on Industrial Networks and Intelligent Systems 11 2021 -04 2022 | Volume 9 | Issue 30 | e1  4, the external observation of OM5451 was quite similar to OM429, IR6976, and IR50404, while IR6976 was a lot like OM4218 and OM6976. Therefore, the number of mislabeled samples of these two varieties were quite higher than others.

Comparing with other machine learning methods for classification the 17 rice varieties
Other machine learning methods like Random Forest [57], Naïve Bayes [58], Decision Tree [59], and Knearest neighbor [60] were combined with the proposed features for classification of seventeen rice varieties. The results were showed in Table 4. In Table 4, the classification accuracy of Random Forest, Naïve Bayes, Decision Tree, and K-nearest neighbor was 85.9%, 61.3%, 54.6%, and 66.4%, respectively. The results show that using SVM combined with the proposed features gives higher ACC than other machine learning methods for classification of the seventeen rice varieties. The results were achieved due to good performance of SVM, combined with selecting parameters of SVM in the experiment so that the classification accuracy was improved.

Comparisons in the literature
To evaluate the effectiveness of the proposed descriptor compared to other methods in the literature for rice variety classification, we used some texture descriptors from the literature with the SVM to classify the 17 rice varieties. In these experiments, the parameters of the SVM were just as they were with the proposed method. However, in some extended LTP methods, the threshold values were not reported in the articles, so we used a threshold value of 5 for those methods.
Comparison of the classification accuracies is presented in Table 5. As seen in Table 5, the accuracy with the proposed method was higher than the other methods from the literature. Regarding the number of features, although our proposed descriptor has more than the ILTP method, they are significantly fewer than other methods, such as CoALTP and CLTP. From  enhance the accuracy in classification of the rice varieties compared to ILTP. When increasing the radius of neighborhood R to 6, the accuracy from ILTP was 90.06%, lower than that of the proposed descriptor. Regarding classification of the rice varieties, a comparison with research from the literature in this area is shown in Table 6. The classification accuracy and the number of classes in our work are higher than other research except for [21]- [23]. Although the authors had more classes in [21], the classification accuracy is lower. The classification accuracy in [22,23] was higher, but the research was conducted with only six and eight classes, respectively.

Conclusion
In this study, texture analysis based on ILTP is extended for color properties to classify 17 rice varieties. The proposed descriptor considers gray, R, G, and B components of the rice varieties. The extracted features are combined with SVM for classification of 17 rice varieties commonly planted in Vietnam. To improve classification accuracy, the radius of neighborhood R and threshold value t were considered in the pattern coding procedure of the proposed method. The classification accuracy from the SVM when using the proposed descriptor was 95.53%, higher than ILTP or other texture descriptors from the literature. The result shows that the proposed method can enhance classification accuracy of rice varieties when extended for color properties of rice grains. In the future, the proposed descriptor will be combined with shape and morphological features to enhance the accuracy of rice varieties classification.