3rd International ICST Conference on COMmunication System SoftWAre and MiddlewaRE

Research Article

On Implementing Security at the Transport Layer

  • @INPROCEEDINGS{10.1109/COMSWA.2008.4554433,
        author={Swaminathan Pichumani and Sneha Kasera},
        title={On Implementing Security at the Transport Layer},
        proceedings={3rd International ICST Conference on COMmunication System SoftWAre and MiddlewaRE},
  • Swaminathan Pichumani
    Sneha Kasera
    Year: 2008
    On Implementing Security at the Transport Layer
    DOI: 10.1109/COMSWA.2008.4554433
Swaminathan Pichumani1,*, Sneha Kasera2,*
  • 1: Juniper Networks
  • 2: School of Computing, University of Utah
*Contact email: spichumani@juniper.net, kasera@cs.utah.edu


We design a framework that implements security at the TCP layer to meet the necessity for a practical and truly end-to-end security solution. We call our framework TCPsec. TCPsec is a security extension to TCP and implemented in the kernel. Applications may use TCPsec through regular TCP sockets by setting special socket options. TCPsec uses a Secure Socket Layer (SSL)-like handshake to set up a secure session. It is interoperable with Network Address Translators. The use of TCPsec will also require both application and kernel-level changes. In order to address this concern, we explore two approaches - one that uses application layer proxies to avoid any changes in the applications and another that uses a kernel sandboxing framework to ease kernel upgrading. We implement TCPsec in the FreeBSD 4.7 kernel and evaluate its performance. Our implementation and evaluation show that TCPsec incurs only a modest overhead as compared to TCP and performs competitively with SSL. We also provide a formal verification of our protocol state machine.