International Conference on Security and Privacy in Communication Networks. 10th International ICST Conference, SecureComm 2014, Beijing, China, September 24-26, 2014, Revised Selected Papers, Part I

Research Article

Control Flow Obfuscation Using Neural Network to Fight Concolic Testing

Download
265 downloads
  • @INPROCEEDINGS{10.1007/978-3-319-23829-6_21,
        author={Haoyu Ma and Xinjie Ma and Weijie Liu and Zhipeng Huang and Debin Gao and Chunfu Jia},
        title={Control Flow Obfuscation Using Neural Network to Fight Concolic Testing},
        proceedings={International Conference on Security and Privacy in Communication Networks. 10th International ICST Conference, SecureComm 2014, Beijing, China, September 24-26, 2014, Revised Selected Papers, Part I},
        proceedings_a={SECURECOMM},
        year={2015},
        month={11},
        keywords={Software obfuscation Malware analysis Reverse engineering Concolic testing Neural network},
        doi={10.1007/978-3-319-23829-6_21}
    }
    
  • Haoyu Ma
    Xinjie Ma
    Weijie Liu
    Zhipeng Huang
    Debin Gao
    Chunfu Jia
    Year: 2015
    Control Flow Obfuscation Using Neural Network to Fight Concolic Testing
    SECURECOMM
    Springer
    DOI: 10.1007/978-3-319-23829-6_21
Haoyu Ma1,*, Xinjie Ma1, Weijie Liu1, Zhipeng Huang1, Debin Gao2,*, Chunfu Jia1,*
  • 1: Nankai University
  • 2: Singapore Management University
*Contact email: ma.haoyu@mail.nankai.edu.cn, dbgao@smu.edu.sg, cfjia@nankai.edu.cn

Abstract

Concolic testing is widely regarded as the state-of-the-art technique in dynamic discovering and analyzing trigger-based behavior in software programs. It uses symbolic execution and an automatic theorem prover to generate new concrete test cases to maximize code coverage for scenarios like software verification and malware analysis. While malicious developers usually try their best to hide malicious executions, there are also circumstances in which legitimate reasons are presented for a program to conceal trigger-based conditions and the corresponding behavior, which leads to the demand of control flow obfuscation techniques. We propose a novel control flow obfuscation design based on the incomprehensibility of artificial neural networks to fight against reverse engineering tools including concolic testing. By training neural networks to simulate conditional behaviors of a program, we manage to precisely replace essential points of a program’s control flow with neural network computations. Evaluations show that since the complexity of extracting rules from trained neural networks easily goes beyond the capability of program analysis tools, it is infeasible to apply concolic testing on code obfuscated with our method. Our method also incorporates only basic integer operations and simple loops, thus can be hard to be distinguished from regular programs.