The challenges of service-side personalized spam filtering: scalability and beyond

Spam filtering of the email stream at the enterprise level poses many challenges especially at the scale of large Email Service Providers (ESPs). The problem is compounded if filtering is to be done on a personal level, with different configurations being adapted on a per-user basis. Commonly, the cost and performance issues are avoided by pushing personalized filtering to the client machine owned by the user, but this changes the user experience depending on the client used to access the mailbox. When inplementing personal spam filters as a services, the benefits stemming from increased spam-detection accuracy need to be carefully balanced with the associated costs, especially in view of a large users population and co-existence with user-independent detection engines. The paper describes the challenges associated with implementing large-scale personalized spam-filtering service ranging from the need to scale with the user population to the challenge of being constrained by a fixed budget.