Rating Prediction Method for Item-based Collaborative Filtering Recommender Systems Using Formal Concept Analysis

The recommender systems are used to mainly suggest recommendations to the online users by utilizing the user preferences recorded during the item purchase. No matter how, the performance of the recommendation quality seems to be inevitable and far satisfactory. In this paper, a new approach based on the mathematical model, Formal Concept Analysis (FCA) is used to improve the rating prediction of the unknown users which can certainly overcome the issues of the existing approaches like data sparsity, high dimensionality of data, performance of the recommendation generated for top n recommendation. The FCA method is applied using Boolean Matrix Factorization (ie. optimal formal concepts) in predicting the rating of the unknown users in the available user-item interaction matrix which proves to be more efficient in tackling the problem of computational complexity managing the high dimensionality of data. The proposed method is applied using item-based collaborative filtering technique and the experiment is conducted on the Movielens dataset which shows the satisfactory results. The experiments results are evaluated using the related error metrics and performance metrics. The experimental results are also compared with existing item-based Collaborative Filtering techniques which demonstrate that the performance of recommendation quality gradually improved with state-of-the-art existing techniques.


Introduction
Mobile devices like mobile phones, tablets, laptops, and many other smart devices have vibrant influence over the modern digital society across the globe. With this advancement, the online markets in the e-commerce industry has the wider power to promote the products and services via email, social networking sites, search engine optimization, display advertising, Google AdWords and so on. The availability of information becomes more and more abundant for the internet users which cause people to pay out more time in looking at the necessary information through various streaming and social media sites.
From this inclination, many online platforms has involved in developing the recommender systems which could track the user behavior and record the user preferences to provide customized recommendation. The recommender systems are prevalent tool for dealing with the huge torrent of information over internet, which indeed causes the information overload problem, and an excellent technique that gives additional benefits to the online customers by generating personalized recommendation based on the past history of the online users [1]. Majority of the Ecommerce sites like Netflix, YouTube, Amazon, Flipkart, eBay and so on are using recommender systems in generating the useful recommendation to its customers based on their purchase patterns. All these Ecommerce sites run their online businesses successfully by relying on the various EAI Endorsed Transactions on Energy Web 03 2021 -05 2021 | Volume 8 | Issue 33 | e11 Chemmalar Selvi G. and Lakshmi Priya G.G.
2 recommendation filtering techniques which is backbone of the online business services.
Thus, the recommender systems work based on the different recommendation filtering techniques which can be broadly classified into content-based filtering, collaborative filtering and hybrid filtering [2]. The content-based filtering extracts features of the content which is used to recommend the similar kind of content to the users [3]. The collaborative filtering makes use of user history and finds the recommendation based on the user profile similarity [4]. The technique of combining one or more filtering approach to overcome the short comes of the above filtering technique and improve the power of recommendation is known as hybrid technique [5].
Collaborative Filtering (CF) is most popular and widely been practiced recommender technique in most of the E-Commerce industry [6]. The working principle of CF is to find the similar group of users to the actual user and to recommend items which were most liked by those similar groups of users to the actual user. There exist two algorithms of CF namely the memory-based CF and modelbased CF algorithms [7]. Memory-based CF algorithm computes the similarity between the users from which the most similar users to the actual user are found and the rating prediction is performed based on the similarity scores. Model-based CF algorithm builds the model based on the user preference available from the user-item rating matrix of which the rating prediction is determined and the relevant recommendation is generated. However, the main challenges of CF are data sparsity where the user-item rating matrix used in CF are extremely sparse in nature which in turn degrade the performance of rating prediction or recommendation of CF and scalability in which the number of users and items in the recommender systems tremendously grow at an exponential rate.
Although there are several research work going on to unravel the highlighted challenges of CF, still there is a need to walk a long way to settle down the problem with ease of solutions. The major problem studied in the recommender systems are data sparsity, high dimensionality of data and poor quality and performance of recommender systems. Thus the identified problem statement is to enhance the rating prediction method which is the baseline for improving the recommender systems performance without losing the originality of the input data. In this regard, the paper introduces a new method from the theory of Formal Concept Analysis (FCA) which is studied carefully to tackle the problem solution to certain extend. The main aim of this paper is to enhance the memory-based collaborative filtering technique using FCA which is a mathematical model and an exploratory data analysis tool [8]. The concept of FCA is been successfully applied to many research domains like information retrieval, knowledge discovery, security analysis, text mining, web services and so on. In this paper, the authors have applied the theory of FCA in recommender system which is based on the memory-based collaborative filtering algorithm to improve the accuracy of the rating prediction and performance of the recommendation task.
The paper is organized as follows: section 1 was introduction which describes the overview of the recommender systems. Section 2 explains the background study in FCA theory and memory-based CF. Section 3 explains the workflow of the FCA-based item recommendation model in CF. Section 4 presents the experimental results and discussion based on the benchmarking publicly available movie dataset. Finally, the paper is completed with the conclusion of the work and future directions.

FCA Theory
The foundations of FCA concept were introduced in the early1980's by a German professor R.Wille which is formed from applied lattice theory and set theory. The input data for the FCA method is represented in the form of matrix where the rows denote the objects and columns denote the attributes. The elements of the input data matrix can consider only Boolean values, meaning whether an object have any attribute or not. If the object has the attribute(s), then a mark(X) is placed in the respective element cell, otherwise, a blank is left in the respective element cell. The input data matrix is called as formal context upon which the data analysis is performed. The output of the FCA can be produced in two forms. The first is the concept lattice which is the hierarchical structure exhibiting the relationship between the identified concepts. The second one is the attribute implications which shows the interdependencies associated among the various attributes in the formal context.
FCA processes the data in the structured format of objects, attributes and relationship between them. This relationship is expressed as formal context in the form of  A X X X B X X X X C X X X X X D X X X X X E X X X X F X X X X X Rating Prediction Method for Item-based Collaborative Filtering Recommender Systems Using Formal Concept Analysis 3 G X X X X H X X X X

Background Study
Belohlavek et al [25] proposed a successful FCA-based Boolean Matrix Factorization (BMF) by decomposing the boolean matrix I into two smaller boolean matrices AF and BF. The paper well discussed about the formation of number of factors which should be as minimal and as optimal for fitting to the problem of matrix decomposition. The number of factors plays an essential role in factorizing the matrix which typically lowers the dimensionality space of the considered matrix. There are certain theorems used to accomplish the task of BMF as mentioned below.

Theorem 1 (Universality of formal concepts as factors).
For

Theorem 2 (Optimality of formal concepts as factors). Let I = A • B for n ×k and k ×m binary matrices A and B.
Then there exists a set F ⊆ B(X, Y , I) of formal concepts of I with |F| ≤ k such that for the n × |F| and |F| ×m binary matrices AF and BF we have The above mentioned theorems are proved in the paper [25] and algorithms are proposed to solve the computational complexity in decomposing the boolean matrix. In this paper, the algorithm that was used (Algorithm 2 in [25]) determines all the formal concepts first and then finds the optimal factors which makes the computation process very faster with the time estimation O(K|G||M| 3 ). Here, k is the number of optimal factors determined, |G| is the number of objects and |M| is the number of attributes.
For any given matrix with rows showing the set of users and columns showing the set of items and the ratings recorded between the scale 1~5 is transformed to a boolean matrix I which is shown in the below example.  In the above example, it illustrates that the algorithm used optimally determines the number of factors which eventually reduces the dimensionality of the given data matrix.
The application of FCA in recommender systems was not well studied research problem. However, there are few researchers who have done their research by applying FCA in the field of recommendation. Ryan et al have proposed two algorithms namely feature-based and entity-based collaborative recommender to find the user neighbor by constructing the concept lattice and demonstrated that the algorithm can solve the high dimensionality problem, also promising result with no loss of information and accuracy [9]. Ignatov et al [10] presented a new approach of collaborative filtering algorithm based on FCA by constructing the BMF which decomposes the binary-scaled matrix into two small dimensional matrices. The authors have shown their experimental results with the MovieLens dataset [11] and compared the performance and accuracy of the collaborative filtering algorithm based on FCA which proves the same result as yielded when applying the nonscaled data.
Hence, it is evident that FCA if applied to recommender systems can be successful to certainly solve the major challenges of the recommender systems along with promising results of good performance and improving quality of recommendation.

Memory-based CF
The memory-based CF uses the user-item rating matrix to calculate the user or item similarity score based on which the rating prediction and top-n recommendation is generated. Basically, the memory-based CF also known as neighbor-based CF is classified into two approaches: userbased and item-based CF. In user-based CF approach, the similarity is computed between the similar users to the target user based on the rating given for the similar items among them of which the target user has not rated any item whereas in item-based CF approach, the item-to-item similarity is determined for which the target user has rated and finally the prediction is made based on the similar items [12].

User-based CF
Item-based CF The item-based CF approach is practiced in most of the ecommerce sites like Amazon because it is more efficient and scalable in real-time scenario. Linden et al [13] have developed an item-to-item collaborative filtering which aims at addressing the issue of data scalability and enhancing the performance of the recommendation. The key for the proposed algorithm is that, it computes the item-to-item similarity matrix offline which improves the faster computation of rating prediction for extremely large data sets while generating the recommendation online. Sarwar et al [14] has presented the item-based CF algorithm by computing the item-to-item similarity based on different similarity measures like cosine and pearson coefficient. Their experimental result evaluated the comparison of userbased vs item-based and concluded that item-based CF algorithm give more desired result in terms of performance quality than the user-based CF algorithm.
Deshpande et al [15] built the item-based CF for generating the top-n recommendation by computing the item similarity and presenting the highly interpolated top-n recommendation. The authors have used different similarity measures like cosine and conditional probability measures for item similarity computation. Based on the similarity score, the neighborhood algorithm is applied based on which the most similar items are grouped. Finally, the paper discusses the experimental evaluation conducted on the MovieLens dataset and presented the performance of the top-n recommendation using evaluation metrics like Hit Ratio (HR), Average Reciprocal Hit Ratio (ARHR). The authors of Cremonesi et al [16] , discussed the importance of evaluation metrics while evaluating the performance of topn recommendation. The paper explains about the general scheme of item-based CF in generating the top-n recommendation using the latent factor model like matrix factorization [17] and illustrates the result using the error metrics, Mean Absolute Error (MAE).
With these research advancements, the traditional itembased technique is further explored to boost its performance and recommendation quality in managing the large sparse real-time datasets. In the paper, Ning et al [19] proposed an efficient algorithm, Sparse Linear Method (SLIM) for top-n recommender which tackles the issue of matrix factorization by scaling the user matrix as binary data and optimizing the aggregation matrix using l1 and l2 norm minimization problem. The result of the proposed work is compared with the traditional item-based CF and found that SLIM outperforms by scaling the data and addressing the data sparsity problem. The authors Kabbur et al [20] put forth Factored Item Similarity Model (FISM) which projects the item-to-item similarity matrix as a factored representation into two low-dimensional latent factor models using the state-of-the-art matrix factorization. Their experimental evaluation illustrates that FISM well addressed the sparsified matrix with no item-bias and compared with other existing traditional item-based techniques using the evaluation metrics HR and ARHR [21].
Although there are good number of models available based on item-based CF, recently it is captured that using factor-based technique for learning the model relation promotes addressing the vital challenges of recommender systems. The following section will discuss about the new method of a factor-based technique called BMF which takes the advantage from the concept driven of FCA. It further discuss about the work flow architecture carried out in constructing the model and evaluating the performance of top-n item-based recommendation.

A new FCA rating prediction for itembased recommendation
FCA is a powerful data analysis tool and it is capable of biclustering the data fed in the form of matrix representation using the derivation operation of set theory. By utilizing the advantages from FCA, the Boolean matrix factorization [22] technique is applied by scaling the data elements to 1 or 0 based on the user-item interaction matrix. Hence, the interaction matrix with 1's and 0's are processed to complete the unknown or 0's element value appropriately. The intuition behind the FCA-based BMF technique is to determine the number of factors based on the dimensionality of the interaction matrix whereas in the traditional MF technique the number of factors is decided by the programmer. The factors are the hidden features which can be expressed as the user preferences with the more informed way of interaction between the users and items [23].

FCA-based Boolean Matrix Factorization
The main idea in choosing the BMF technique for the recommender systems is to solve the challenge of addressing the binary-rated data which is so called as implicit feedback [24]. Now-a-days, most of the users do not show interest in giving the feedback and hence the most of the CF systems start to utilize this implicit feedback which is recorded from the user's depth of page scrolls, web history, clickthrough, purchases and so on. In this way, the degree of the user preferences is managed by performing the matrix factorization technique which is based on binaryrated values. In the paper, we scaled the data of the original matrix with the ratings 1~5 into 1's and 0's inorder to show improvement in the faster computation time of the recommendation model.
Given the Boolean matrix F with m rows and n columns and there exist binary relation Imn ∈ {0,1} and is decomposed into two small boolean matrix product of the form A ͦ B with Aml ∈ {0,1} and Bln ∈ {0,1} and m x f and f x n respectively such that f becomes as small as possible. Thus, the boolean matrix or binary matrix is a matrix which can be represented as below: in which ˅ denotes the conjunction and disjunction operation. From this, the original matrix F is expressed as the boolean matrix with the binary relation Imn ∈ {0,1} between the set of objects (users) G and set of attributes (items) M which forms the triplet (G,M,I) called formal context. The collection of all formal concepts are illustrated as the concept lattice Ć (G,M,I) which is used in decomposing the original matrix F into AF and BF and is denoted as: , ∉ Hence, the boolean matrix I is decomposed into as small as smaller matrices AF and BF where AF exhibit the relationship between the user base and factor base and similarly, BF exhibit the relationship between the item base and factor base. The factor based are the hidden features of the user and item profile which can be studied well to solve the problem of rating prediction in recommender systems.

Figure 4. Workflow of Proposed Work
Hence, the BMF technique can be used to predict the rating of the unknown users which aim to improve the quality of recommendation performed. In Figure 4, the proposed method, BMF-based rating prediction is shown. Initially, with the available user-item rating data, the dataset needs to be transformed into matrix representation with rows as users, columns as items and call values as ratings. The formal context is constructed in step 1 which is used to determine the set of formal concepts as in step 2. Based on the formal concepts, the optimal factors are identified using the properties of object-induced and attribute-induced concepts in step 3. With these optimal factors, the matrix is decomposed into 0's and 1's and the rating prediction formula referred in equation 3.1 is applied to predict the rating of the unknown users.
The rating prediction is calculated by: � where Ʀ = new rating for user i and item j F = number of optimal factors = latent factor of user i = latent factor of item j Thus, the incomplete matrix is constructed with the completed matrix values which can be applied on the recommender model to generate the possible recommendation to the users.

Item-Based Top-N Recommendation
Item-based collaborative filtering algorithm is a class of memory-based technique in which the item similarity is computed to the target user based on the similarity measures. The item-to-item similarity offline which makes the algorithm to improve on the model construction time for extremely large dataset. The similarities of the items are computed using the cosine and pearson coefficient similarity measures [26]. In this approach, the cosine similarity is used to compute the itemto-item similarity table as illustrated in figure 5 and 6. The figure 1 shows there are 9 users and 6 items whose item preference are fixed at the scale from 1 ~ 5. The cosine similarity is calculated by using the equation 3. In the collaborative filtering algorithm, the unknown value of a particular user is predicted by calculating the ratings of items that are previously rated by other users.
Hence, the rating prediction R u,i is calculated for the target user u and item i and given in equation 3.3: where k is most similar items to the target user which is examined based on k-Nearest Neighbor (KNN) algorithm in this case, s ( , ′ ) is the similarity score found in the item similarity table between item and target item ′ and , ′ is the ratings given by the other users similar to the items listed in the k group to the items of target user. Thus the predicted rating R u,i is used in generating the recommendation by assuming the threshold value for the target user. When the top n recommendation is performed, all the items in the item set are considered for recommendation without neglecting the k-most similar items in the frequent item set. By this, the better recommendation can be achieved by covering all the items in the item set for the target user.

Experimental Results and Discussion
In this section, the experimental evaluation for FCA-based top n recommendation algorithm (Item-Based CF) is conducted. The experiments were conducted in NVIDIA GPU Xp architecture -Pascal and with system configuration 32 GB RAM and 64-bit windows operating system. The following discussion shows the dataset used in conducting the experiment followed by the experimental evaluation of the FCA-based method in top n recommendation.

Dataset
The experiment is evaluated with the publically available real dataset MovieLens 1 [27] whose features are summarized in the table 1. The MovieLens dataset is preprocessed to improve the consistency of the users and items present in the dataset. There were about 1 lakh transactions which is vital information showing the user preference history for the set of items in the dataset. The condition to filter the number of records were based on every user should have rated atleast 20 movies such that percentage level of sparsity will be reduced.

Experimental Discussion
In the experimental work, the concept of FCA was carefully studied and evaluated in terms of error metrics and time taken to build the model and time taken to produce the recommendation. To the fact that the experimental results shown in figure 7, illustrates that the error metrics measured using Root Mean Square Error (RMSE) [24] and Mean Absolute Eror (MAE) [25] is satisfactory which can enhance the quality of recommendation as expected. Also, the experimental result is evident that the time consumed in building the model by applying FCA in item-based CF yields good result. The figure 8 shows the time taken to build the recommendation model and time considered for generating the suitable recommendation when applied with top n recommender. The model and recommendation time is measured in seconds and it is compared with the model time of traditional item-based CF which still proves that the new 3.  One of the advantages of FCA-based Item CF was that it decomposed the sparse user-item matrix using BMF with optimal number of factors which had significantly reduced the higher dimensionality and sparsity of the real dataset. Further to add, it examined all possible set of combinations called formal concepts which is a pair (A,B) with the similar set of users A who have co-rated the similar items in the item set B. The kNN algorithm is used in the experiment to find the k-most similar items which are rated by other users to compare against the items rated by the target user. During the experiment, the different k values were used to seek the quality of the recommendation. The evaluation metrics used to test the recommendation quality was Hit Ratio (HR) and Average Reciprocal Hit-Rank (ARHR) [28]. The HR is given as follows: where | | is the items recommended in the top n recommendation list to the number of users and | | is the total number of users used.
The next evaluation metric AVHR is described as: where AVHR is the weighted metric of HR showing the weight as the reciprocal of the position posi of item i which is a hit in the recommendation list of top n recommendation to the target user. The figure 9 presents the comparison graph between the k neighbors with the HR and found that the value of HR gradually increased when the number of k size was eventually increased. The reason is when the k size is increased, then the number of items in the item set will be improved with the different combinations of item features and likely chance of receiving the number of hits existing in the top n list will also be improved. For generating the top n recommendation list, the threshold value used was 0.5 which presumes that, from the rating prediction discussed in section 3 which was calculated from the similarity table also discussed in figure 4 of section3, if the predicted rating score is <0.5 then remove the set of items from the recommendation list as it does not show the stronger similarity among the co-rated items in the item set; otherwise if the predicted rating score is ≥ 0.5 then add it to the top n recommendation list since it exhibit the stronger relationship among all the co-rated items in the item set.

Comparison of Top N recommendation algorithm
The performance of the recommendation algorithm is evaluated using the two evaluation metrics: HR and AVHR.
There are several existing algorithms which were used to compare the result of the new FCA method with the existing top n recommendation algorithm. Among them, the following existing algorithms were used for comparison which comprises of one neighbor-based CF method EAI Endorsed Transactions on Energy Web 03 2021 -05 2021 | Volume 8 | Issue 33 | e11 (ItemKNN), two MF-based CF method (PureSVD and WRMF), two ranking-based CF methods (BPRMF and BPRKNN), one nuclear norm based MF (SLIM) and one optimization-based MF (ALM). The table 2 shows the comparison of the several item-based CF top n recommendation algorithms experimented using the MovieLens dataset. From the comparison table, the result is very clear that the new FCA method for item-based CF algorithm performs well than other existing algorithms both in terms of HR and AVHR evaluation measures. The performance of top n recommendation is achieved on an average by 23.7% compared with the other existing techniques which are quite substantial. On the other hand, the average metric of AVHR is 11.6% which is more notable performance achieved by the FCA-based CF method than any other existing methods. Hence, the new approach of applying FCA method in recommender system ie. Item-based CF technique seems to be successful with the high coverage of the data sparseness and dealing with the high dimensionality of the extremely large dataset without any loss of data and accuracy of information.

Conclusion
In this paper, the new approach of item-based collaborative filtering technique is presented based on the mathematical framework named FCA. The presented method completes the user-item rating matrix by solving the Boolean Matrix Factorization in FCA. To constantly improve the computational time of the recommendation model, the optimal formal concepts are carefully examined without any loss of information from the high dimensional and sparse dataset. Further, the authors conducted the experimental evaluation on the benchmarking datasets and presented the results by comparing the performance of top n recommendation with the existing state-of-the-art techniques. Hence to conclude, the application of FCA in recommender systems yields good result by enhancing the recommendation quality and also to resolve the computational complexity in managing such extremely large datasets in real-time use case.