9th IEEE International Conference on Collaborative Computing: Networking, Applications and Worksharing

Research Article

Non-intrusive process-based monitoring system to mitigate and prevent VM vulnerability explorations

Download697 downloads
  • @INPROCEEDINGS{10.4108/icst.collaboratecom.2013.254107,
        author={Chun-Jen Chung and Jingsong Cui and Pankaj Khatkar and Dijiang Huang},
        title={Non-intrusive process-based monitoring system to mitigate and prevent VM vulnerability explorations},
        proceedings={9th IEEE International Conference on Collaborative Computing: Networking, Applications and Worksharing},
        publisher={ICST},
        proceedings_a={COLLABORATECOM},
        year={2013},
        month={11},
        keywords={software defined networking attack graph intrusion detection countermeasure selection virtual machine introspection},
        doi={10.4108/icst.collaboratecom.2013.254107}
    }
    
  • Chun-Jen Chung
    Jingsong Cui
    Pankaj Khatkar
    Dijiang Huang
    Year: 2013
    Non-intrusive process-based monitoring system to mitigate and prevent VM vulnerability explorations
    COLLABORATECOM
    IEEE
    DOI: 10.4108/icst.collaboratecom.2013.254107
Chun-Jen Chung1, Jingsong Cui2, Pankaj Khatkar1, Dijiang Huang1,*
  • 1: Arizona State University
  • 2: Wuhan University
*Contact email: dijiang@asu.edu

Abstract

Cloud is gaining momentum but its true potential is hampered by the security concerns it has raised. Having vulnerable virtual machines in a virtualized environment is one such concern. Vulnerable virtual machines are an easy target and existence of such weak nodes in a network jeopardizes its entire security structure. Resource sharing nature of cloud favors the attacker, in that, compromised machines can be used to launch further devastating attacks. First line of defense in such case is to prevent vulnerabilities of a cloud network from being compromised and if not, to prevent propagation of the attack. To create this line of defense, we propose a hybrid intrusion detection framework to detect vulnerabilities, attacks, and their carriers, i.e. malicious processes in the virtual network and virtual machines. This framework is built on attack graph based analytical models, VMM-based malicious process detection, and reconfigurable virtual network-based countermeasures. The proposed framework leverages Software Defined Networking to build a monitor and control plane over distributed programmable virtual switches in order to significantly improve the attack detection and mitigate the attack consequences. The system and security evaluations demonstrate the efficiency and effectiveness of the proposed solution.