Electronic Healthcare. 4th International Conference, eHealth 2011, Málaga, Spain, November 21-23, 2011, Revised Selected Papers

Research Article

Syndromic Classification of Twitter Messages

Download39 downloads
  • @INPROCEEDINGS{10.1007/978-3-642-29262-0_27,
        author={Nigel Collier and Son Doan},
        title={Syndromic Classification of Twitter Messages},
        proceedings={Electronic Healthcare. 4th International Conference, eHealth 2011, M\^{a}laga, Spain, November 21-23, 2011, Revised Selected Papers},
        proceedings_a={E-HEALTH},
        year={2012},
        month={5},
        keywords={epidemic intelligence social networking machine learning natural language processing},
        doi={10.1007/978-3-642-29262-0_27}
    }
    
  • Nigel Collier
    Son Doan
    Year: 2012
    Syndromic Classification of Twitter Messages
    E-HEALTH
    Springer
    DOI: 10.1007/978-3-642-29262-0_27
Nigel Collier1,*, Son Doan1
  • 1: National Institute of Informatics
*Contact email: collier@nii.ac.jp

Abstract

Recent studies have shown strong correlation between social networking data and national influenza rates. We expanded upon this success to develop an automated text mining system that classifies Twitter messages in real time into six syndromic categories based on key terms from a public health ontology. 10-fold cross validation tests were used to compare Naive Bayes (NB) and Support Vector Machine (SVM) models on a corpus of 7431 Twitter messages. SVM performed better than NB on 4 out of 6 syndromes. The best performing classifiers showed moderately strong F1 scores: respiratory = 86.2 (NB); gastrointestinal = 85.4 (SVM polynomial kernel degree 2); neurological = 88.6 (SVM polynomial kernel degree 1); rash = 86.0 (SVM polynomial kernel degree 1); constitutional = 89.3 (SVM polynomial kernel degree 1); hemorrhagic = 89.9 (NB). The resulting classifiers were deployed together with an EARS C2 aberration detection algorithm in an experimental online system.