Comparison of Different Neural Network Training Algorithms with Application to Face Recognition

Research in the field of face recognition has been popular for several decades. With advances in technology, approaches to solving this problems haves changed. Main goal of this paper was to compare different training algorithms for neural networks and to apply them for face recognition as it is a nonlinear problem. Algorithm that we have used for face recognition problem was the Eigenface algorithm that belongs to the Principal Component Analysis (PCA) algorithms. Percentage of recognition for all the used training functions is above 90%.


Introduction
People have used different body traits such as voice, face, gait cycle etc.for centuries, in order to recognize other people.Although face recognition is quite intuitive for most human beings, it rather too complex for a machine vision system to be pervasively applied [1].
Two major parts of all face recognition algorithms are: (1) face detection and normalization and (2) face identification.Fully automatic algorithms are those that include both previously mentioned parts, while partially automatic algorithms only have face identification [2].Detection and segmentation of the face areas from the background is the first step in this process [3].With this successfully done, one of the available algorithms (or developed new) is applied in order to determine whether a face belongs to the known or not known faces of the available database.In case of a new face, a new folder with person's photo and information is created while in case of a known face, the database needs to be updated.The results are followed by appropriate information displayed to the user (Fig. 1).

EAI Endorsed Transactions on Industrial Networks and Intelligent Systems
from the basic parts of a face (e.g.eyes, nose, mouth, and chin).After that, gathered key information and converted into a feature vector.Yullie and Cohen [4] used deformable templates in order to extract the contour of face images.The second method is based on PCA method.In this method, key information is derived from the entire face image.Kirby and Sirovich [5], [6] have shown that any particular face can be expressed using the coordinate system named "eigenfaces".They can be described as the Eigen functions of the average covariance of the ensemble of faces.Turk and Pentland [7] proposed the eigenface approach for face recognition problem.Artificial Neural Network (ANN) was used for further classification.ANN can be used for prediction for seen and unseen data.Different models of ANN have been used for face recognition, and reason for that is ability of those models to resemble the way in which neurons function in human brain.Researches [8], [9] have been using ANN for face detection and recognition on video stream.Common problem is distinction between different skin colors.Mohamed et al. [10] proposed face detection system for image segmentation that depends on skin color.
The application of face recognition can be divided into two parts: law enforcement application and commercial application.Law enforcement applications would include identifying wanted individuals, while the commercial applications range from static matching of photographs on credit cards, ATM cards, passports, driver's licenses, and photo ID to real-time matching with still images or video image sequences for access control [2].
All these form the motivation behind the challenging task of recognizing a person's identity using (artificial neural networks) ANN for nonlinear problem of identification.Our main goal is therefore to compare different training algorithms for face recognition in order to obtain gather valuable information about the process of recognition.

Materials and Methods
Material for this paper was AT&T database which consists of grayscale photos of 40 people, where each person was photographed 10 times (400 photos on the whole).These 10 photos of each person were made at different times, varying the lighting conditions, facial expressions (open / closed eyes, smiling / not smiling) and facial details (glasses / no glasses).All the images were taken while subjects were standing and facing forward.Background for all of the images is dark and homogeneous.Size of each image is 112 x 92 pixels with 256 levels of grey.
First, it was necessary to load the database and each photo, since it is represented with matrix, convert to a column vector.Therefore, the database is converted to a matrix that consists of 400 columns and 10304 rows.Next step is for user to choose the photo of a person that should be recognized and start the program.
Algorithm used in this paper belongs to the Principal Component Analysis algorithms (PCA) -Eigenface algorithm.General steps in eigenface algorithms include computation of the average face and subtraction of the average face from the training faces.First, we need to calculate the average image.Then, that image has to be subtracted from each original image.Next step is calculation of the eigenvectors and eigenvalues of the covariance matrix.Each eigenvector has the same dimensionality as the original images, and can be seen as an image.They represent the directions in which the images in the database deviate from the mean image.This is a computationally expensive step, but in order to apply this method we need to compute the eigenvectors of covariance matrix efficiently, without ever computing covariance matrix explicitly by sorting the eigenvalues in descending order and arrange eigenvectors accordingly.The number of principal components k is determined arbitrarily by setting a threshold ε on the total variance.Final step is the decision itself where the program needs to distinguish between the cases: This analysis allows for reduction of the calculation.In practice, the training set of face images is rather small, and that makes calculations manageable.However, since the algorithm itself assumes that the problem is linear, authors [11] indicate that the results obtained are not satisfying.Therefore, we introduced neural networks which are used for nonlinear problems.Moreover, because as the distance from the mean face for each individual becomes more densely distributed it becomes difficult for eigenfaces algorithm to distinguish in between so the result becomes more erroneous.Whereas in the case of neural networks, with more training and more complex neuron structure, its performance does not deteriorate so rapidly.In some cases performance is getting close to perfectness.The ANN is first trained on images from the face database, and then it is used to identify the face images given to it.We have used feed forward back propagation neural network with two hidden layers (50 and 90 neurons in the hidden layers).Training parameters that were used are shown in Table 1.parameters.As previously mentioned our implementation of Eigenface algorithm has been successful in more than 90% of images used.Those results are similar to the results that can be found in [12], [13].

Conclusion
There are number or different approaches to face recognition problem.In this paper we have shown the combination of Eigenface algorithm and neural networks with the use of different training functions.As evidenced by [14], accuracy deteriorates with changes in lighting, illumination and other factors.Whereas reflectance and shape are elemental properties of a face object, the appearance of a face is also subject to several other factors, including the facial expression and facial pose.In addition to these, various imaging parameters, such as exposure time, aperture, lens aberrations, and sensor spectral response also increase subject variations [11].Nevertheless, the number of examples per person, that are available for learning, is often much smaller than the dimensionality of the image space.A system that was trained using so few examples may not work well to unseen instances of the face [11].
All training functions have percentage of recognition above 90%.However, only small database of 400 images has been tested.Further research would include creating bigger database and testing again to see if the results will be as good as with database used in this paper.In this approach we treat this problem as a twodimensional (2D) problem, although it is 3D problem.One way in which future work may go is looking at this problem as 3D problem and use neural networks recognition between 3D face models.

Figure 1 .
Figure 1.Phases in the process of face recognition There are two basic methods for face recognition.The first method is based on the use of deformable templates and extensive mathematics for extraction of the feature vectors (i) It is a new face -update the database with the new face and inform the user that it is a new face (ii) It is a known face -inform the user what are the possibilities and probabilities for correct recognition.
Figure 2 and 3 are examples of successful face recognition.

Figure 2 .
Figure 2. Face recognition example using traincgb as training function

Figure 3 .
Figure 3. Face recognition example using trainrp as training function Based on Figure 2 and 3 and the percentage of successful recognition we can say that used algorithm and neural network have shown good results so far.This leaves room for further testing with bigger image database and different training Endorsed Transactions on Industrial Networks and Intelligent Systems 12 2017 -01 2018 | Volume 4 | Issue 12 | e3

Table 1 .
Neural Network parameters Comparison of Different Neural Network Training Algorithms with Application to Face Recognition 3 With all used training function percentage of recognition was above 90%.The results for each training function are shown in Table 2. Considering number of unrecognized photos training functions trainrp and traingdx have shown the best results with successful recognition in 95.75% cases.Considering number of persons where all 10 photos are recognized, function traingcb performed best with completely recognizing 16 out of 40 persons.Second best function was trainrp with 15 persons.Based on the results shown in Table 2 we can say that training functions traincgb and trainrp have performed best for previously mentioned database and training parameters.