Adaptive Network Coded Clouds: High Speed Downloads and Cost-Effective Version Control

Research output: Contribution to journalResearch articleContributedpeer-review

Contributors

  • Marton Sipos - , Budapest University of Technology and Economics, Aalborg University (Author)
  • Janus Heide - , Aalborg University, Steinwurf ApS (Author)
  • Daniel E. Lucani - , Aalborg University, Chocolate Cloud ApS (Author)
  • Morten V. Pedersen - , Aalborg University (Author)
  • Frank H.P. Fitzek - , Deutsche Telekom Chair of Communication Networks, Aalborg University (Author)
  • Hassan Charaf - , Budapest University of Technology and Economics (Author)

Abstract

Although cloud systems provide a reliable and flexible storage solution, the use of a single cloud service constitutes a single point of failure, which can compromise data availability, download speed, and security. To address these challenges, we advocate for the use of multiple cloud storage providers simultaneously using network coding as the key enabling technology. Our goal is to study two challenges of network coded storage systems. First, the efficient update of the number of coded fragments per cloud in a system aggregating multiple clouds in order to boost the download speed of files. We developed a novel scheme using recoding with limited packets to trade-off storage space, reliability, and data retrieval speed. Implementation and measurements with commercial cloud providers show that up to 9x less network use is needed compared to other network coding schemes, while maintaining similar download speeds and reliability. Second, the ability to update coded fragments from a linear erasure code when the original file is modified. We exploit code structure to provide efficient representations of the evolution of the file. Evaluations using file changes on software library repositories show that a five-order of magnitude reduction in network and storage use is possible compared to state-of-the-art.

Details

Original languageEnglish
Article number7274724
Pages (from-to)19-33
Number of pages15
JournalIEEE transactions on cloud computing
Volume7
Issue number1
Publication statusPublished - 1 Jan 2019
Peer-reviewedYes

External IDs

ORCID /0000-0001-8469-9573/work/161891241

Keywords

Keywords

  • Distributed systems, error control codes, information storage and retrieval