sis 15(6): e1

Research Article

Using Deduplicating Storage for Efficient Disk Image Deployment

Download213 downloads
  • @ARTICLE{10.4108/icst.tridentcom.2015.259963,
        author={Xing Lin and Mike Hibler and Eric Eide and Robert Ricci},
        title={Using Deduplicating Storage for Efficient Disk Image Deployment},
        journal={EAI Endorsed Transactions on Scalable Information Systems},
        volume={2},
        number={6},
        publisher={EAI},
        journal_a={SIS},
        year={2015},
        month={8},
        keywords={deduplication, image deployment},
        doi={10.4108/icst.tridentcom.2015.259963}
    }
    
  • Xing Lin
    Mike Hibler
    Eric Eide
    Robert Ricci
    Year: 2015
    Using Deduplicating Storage for Efficient Disk Image Deployment
    SIS
    EAI
    DOI: 10.4108/icst.tridentcom.2015.259963
Xing Lin1, Mike Hibler1, Eric Eide1,*, Robert Ricci1
  • 1: University of Utah
*Contact email: eeide@cs.utah.edu

Abstract

Many clouds and network testbeds use disk images to initialize local storage on their compute devices. Large facilities must manage thousands or more images, requiring significant amounts of storage. At the same time, to provide a good user experience, they must be able to deploy those images quickly. Driven by our experience in operating the Emulab site at the University of Utah---a long-lived and heavily-used testbed---we have created a new service for efficiently storing and deploying disk images. This service exploits the redundant data found in similar images, using deduplication to greatly reduce the amount of physical storage required. In addition to space savings, our system is also designed for highly efficient image deployment---it integrates with an existing highly-optimized disk image deployment system, Frisbee, without significantly increasing the time required to distribute and install images. In this paper, we explain the design of our system and discuss the trade-offs we made to strike a balance between efficient storage and fast disk image deployment. We also propose a new chunking algorithm, called AFC, which enables fixed-size chunking for deduplicating allocated disk sectors. Experimental results show that our system reduces storage requirements by up to 3x while imposing only a negligible runtime overhead on the end-to-end disk-deployment process.