• Summary

  The rise of ubiquitous computing has prompted the need of ubiquitous information storage and access. Supporting the availability, survivability, persistence, confidentiality and integrity of information is becoming more and more crucial. This calls for secure and reliable data storage systems that distribute information over networks, enabling users to store and access critical data in a continuously available and highly trustable fashion. The goal of this Hydra project is to design and implement a general platform for data storage systems to meet such objectives.

  A Hydra platform consists of n networked nodes. A user data is stored (or Hydra-ized) on the Hydra platform using a proper (n, k) MDS error correcting code. A user data is hence totally recoverable in presence of up to (n - k) node failures due to natural disasters and/or malicious attacks.

  See a few screen shots of a simple demo here .

  The Hydra platform provides flexible (n, k) coding schemes together with a set of basic operations on data files. Such a platform can be incorporated, modified, adapted and extended in other research, design and development of distributed storage systems (whether clustered or wide area data grids) that can survive natural disasters and malicious attacks. Such systems can also be used for long-lived data archiving mechanisms.

  In this project, a few applications will also be developed on top of the Hydra platform to demonstrate its potentials in data security.

• Design

  A Hydra platform is designed to meet the following objectives:

  • The platform should not be restricted to either a particular operating system (OS) or a particular file system (FS). It should be easily portable to any OS and FS.
  • The platform should not be dependent on any particular hardware.
  • UNIX file system semantics should be maintained for UNIX clients.
  • The platform's overhead on a host file system should be negligibly small, in terms of both performance and storage usage.
  • The implementation should be transport-independent.
  • It should be easily installable, configurable, scalable, flexible, maintainable and automatable.

  For a quick launch, our current Hydra platform is built on top of NFS. It will later be integrated into proper file systems.

• Documents and Softwares
  • An Efficient XOR-Scheduling Algorithm for Erasure Codes Encoding ( pdf ), Proc. of DSN 2009, June 29 to July 2 2009, Estoril, Lisbon, Portugal.

  • A Performance Evaluation and Examination of Open-Source Erasure Coding Libraries for Storage ( pdf ), Proc. of FAST 2009, Feb. 24 to 27, 2009, San Francisco, CA

  • HyFS: A Highly Available Distributed File System ( abstract || slides || poster ), FAST 2008, Feb. 2008

  • SCAN: An Efficient Sector Failure Recovery Algorithm for RAID-6 Codes ( pdf ), Tech Report, Dec. 2007

  • Quantifying Benefit and Cost of Erasure Code Based File Systems ( pdf ), Tech Report, Dec. 2007

  • Efficient Erasure Decoding for Generalized Reed Solomon Codes ( pdf ), Tech Report, Jan. 2007

  • Optimizing Cauchy Reed-Solomon Codes for Fault-Tolerant Network Storage Applications ( pdf ) Proc. of The 5th IEEE International Symposium on Network Computing and Applica tions (IEEE NCA06), Cambridge, MA, July, 2006. ( Best Paper for Network Computing )
    
    
  • STAR: An Efficient Coding Scheme for Correcting Triple Storage Node Failures ( pdf ) Proc. of FAST 2005, December 14 to 16, San Francisco, CA
    
    
  • Hydra: A Platform for Survivable and Secure Data Storage Systems ( pdf ) Proc. of StorageSS 2005, FairFax, Virginia, USA, Nov. 2005.
    
    
  • On the Erasure Recoverability of MDS Codes under Concurrent Updates ( pdf ) Proc. of ISIT 05, Adelaide, Australia, Sep. 2005.
    
    
  • Using Erasure Codes Efficiently for Storage in a Distributed System(pdf), Proc. DSN 2005, Yokohama, Japan, June-July 2005.
    
    
• Acknowledgments

  This project is being funded by the National Science Foundation .

Back to the Network and Information Systems Lab .