sis 16(10): e1

Research Article

A scalable app for measuring autism risk behaviors in young children: A technical validity and feasibility study

Download1217 downloads
  • @ARTICLE{10.4108/eai.14-10-2015.2261939,
        author={Jordan Hashemi and Kathleen Campbell and Kimberly Carpenter and Adrianne Harris and Qiang Qiu and Mariano Tepper and Steven Espinosa and Jana Schaich Borg and Samuel Marsan and Robert Calderbank and Jeffery Baker and Helen Egger and Geraldine Dawson and Guillermo Sapiro},
        title={A scalable app for measuring autism risk behaviors in young children: A technical validity and feasibility study},
        journal={EAI Endorsed Transactions on Scalable Information Systems},
        keywords={autism, automatic behavioral coding, facial affect coding system, integrated app, scalability, natural environments},
  • Jordan Hashemi
    Kathleen Campbell
    Kimberly Carpenter
    Adrianne Harris
    Qiang Qiu
    Mariano Tepper
    Steven Espinosa
    Jana Schaich Borg
    Samuel Marsan
    Robert Calderbank
    Jeffery Baker
    Helen Egger
    Geraldine Dawson
    Guillermo Sapiro
    Year: 2015
    A scalable app for measuring autism risk behaviors in young children: A technical validity and feasibility study
    DOI: 10.4108/eai.14-10-2015.2261939
Jordan Hashemi1,*, Kathleen Campbell2, Kimberly Carpenter3, Adrianne Harris4, Qiang Qiu1, Mariano Tepper1, Steven Espinosa1, Jana Schaich Borg3, Samuel Marsan3, Robert Calderbank1, Jeffery Baker5, Helen Egger3, Geraldine Dawson2, Guillermo Sapiro1
  • 1: Department of Electrical and Computer Engineering, Duke University, USA
  • 2: School of Medicine, Duke University, USA
  • 3: Department of Psychiatry and Behavioral Sciences, Duke University, USA
  • 4: Department of Clinical Psychology, Duke University, USA
  • 5: Department of Pediatrics, Duke University, USA
*Contact email:


In spite of recent advances in the genetics and neuroscience of early childhood mental health, behavioral observation is still the gold standard in screening, diagnosis, and outcome assessment. Unfortunately, clinical observation is often subjective, needs significant rater training, does not capture data from participants in their natural environment, and is not scalable for use in large populations or for longitudinal monitoring. To address these challenges, we developed and tested a self-contained app designed to measure toddlers' social communication behaviors in a primary care, school, or home setting. Twenty 16-30 month old children with and without autism participated in this study. Toddlers watched the developmentally-appropriate visual stimuli on an iPad in a pediatric clinic and in our lab while the iPad camera simultaneously recorded video of the child's behaviors. Automated computer vision algorithms coded emotions and social referencing to quantify autism risk behaviors. We validated our automatic computer coding by comparing the computer-generated analysis of facial expression and social referencing to human coding of these behaviors. We report our method and propose the development and testing of measures of young children's behaviors as the first step toward development of a novel, fully integrated, low-cost, scalable screening tool for autism and other neurodevelopmental disorders of early childhood.