8th IEEE International Conference on Collaborative Computing: Networking, Applications and Worksharing

Research Article

Evolutionary Study of Web Spam: Webb Spam Corpus 2011 versus Webb Spam Corpus 2006

Download78 downloads
  • @INPROCEEDINGS{10.4108/icst.collaboratecom.2012.250689,
        author={De Wang and Danesh Irani and Calton Pu},
        title={Evolutionary Study of Web Spam: Webb Spam Corpus 2011 versus Webb Spam Corpus 2006},
        proceedings={8th IEEE International Conference on Collaborative Computing: Networking, Applications and Worksharing},
        publisher={IEEE},
        proceedings_a={COLLABORATECOM},
        year={2012},
        month={12},
        keywords={web spam evolutionary spam corpus},
        doi={10.4108/icst.collaboratecom.2012.250689}
    }
    
  • De Wang
    Danesh Irani
    Calton Pu
    Year: 2012
    Evolutionary Study of Web Spam: Webb Spam Corpus 2011 versus Webb Spam Corpus 2006
    COLLABORATECOM
    ICST
    DOI: 10.4108/icst.collaboratecom.2012.250689
De Wang1,*, Danesh Irani1, Calton Pu1
  • 1: Georgia Institute of Technology
*Contact email: wang6@gatech.edu

Abstract

With over 2.5 hours a day spent browsing websites online[1] and with over a billion pages[2], identifying and detecting web spam is an important problem. Although large corpora of legitimate web pages are available to researchers, the same cannot be said about web spam or spam web pages.

We introduce the Webb Spam Corpus 2011 -- a corpus of approximately 330,000 spam web pages -- which we make available to researchers in the fight against spam. By having a standard corpus available, researchers can collaborate better on developing and reporting results of spam filtering techniques. The corpus contains web pages crawled from links found in over 6.3 million spam emails. We analyze multiple aspects of this corpus including redirection, HTTP headers and web page content.

We also provide insights into changes in web spam since the last Webb Spam Corpus was released in 2006. These insights include: 1) spammers manipulate social media in spreading spam; 2) HTTP headers also change over time (e.g. hosting IP addresses of web spam appear in more IP ranges); 3) Web spam content has evolved but the majority of content is still scam.