1st International Conference on Collaborative Computing: Networking, Applications and Worksharing

Research Article

Performance evaluation of rating aggregation algorithms in reputation systems

  • @INPROCEEDINGS{10.1109/COLCOM.2005.1651235,
        author={Zhengqiang Liang and Weisong Shi},
        title={Performance evaluation of rating aggregation algorithms in reputation systems},
        proceedings={1st International Conference on Collaborative Computing: Networking, Applications and Worksharing},
        publisher={IEEE},
        proceedings_a={COLLABORATECOM},
        year={2006},
        month={7},
        keywords={Computational modeling  Computer networks  Computer science  Context modeling  Costs  Humans  Immune system  Inference algorithms  State feedback  Vehicle dynamics},
        doi={10.1109/COLCOM.2005.1651235}
    }
    
  • Zhengqiang Liang
    Weisong Shi
    Year: 2006
    Performance evaluation of rating aggregation algorithms in reputation systems
    COLLABORATECOM
    IEEE
    DOI: 10.1109/COLCOM.2005.1651235
Zhengqiang Liang1,*, Weisong Shi1,*
  • 1: Department of Computer Science, Wayne State University
*Contact email: sean@wayne.edu, weisong@wayne.edu

Abstract

Ratings (also known as recommendations, referrals, and feedbacks) provide an efficient and effective way to build trust relationship amongst peers in open environments. The key to the success of ratings is the rating aggregation algorithm. Several rating aggregation algorithms have been proposed, however, all of them are evaluated in an ad-hoc fashion so that it is difficult to compare the effects of these schemes. In this paper, we argue that what is missing is to evaluate different aggregation schemes in the same context. We first classify all state-of-the-art aggregating algorithms into five categories, and then comprehensively evaluate them in the context of a general decentralized trust inference model with respect to their resistance to different factors, such as dynamic behavior of peers and raters, dishonest ratings, and so on. The simulation results show that complicated algorithms are not always a good choice if we take the implementation cost and resistance to bad raters into consideration.