Active Replication at (Almost) No Cost

Research output: Contribution to conferencesPaperContributedpeer-review

Contributors

Abstract

MapReduce has become a popular programming paradigm in the domain of batch processing systems. Its simplicity allows applications to be highly scalable and to be easily deployed on large clusters. More recently, the MapReduce approach has been also applied to Event Stream Processing (ESP) systems. This approach, which we call StreamMapReduce, enabled many novel applications that require both scalability and low latency. Another recent trend is to move distributed applications to public clouds such as Amazon EC2 rather than running and maintaining private data centers. Most cloud providers charge their customers on an hourly basis rather than on CPU cycles consumed. However, many applications, especially those that process online data, need to limit their CPU utilization to conservative levels (often as low as 50%) to be able to accommodate natural and sudden load variations without causing unacceptable deterioration in responsiveness. In this paper, we present a new fault tolerance approach based on active replication for StreamMapReduce systems. This approach is cost effective for cloud consumers as well as cloud providers. Cost effectiveness is achieved by fully utilizing the acquired computational resources without performance degradation and by reducing the need for additional nodes dedicated to fault tolerance.

Details

Original languageEnglish
Pages21-30
Number of pages10
Publication statusPublished - 2011
Peer-reviewedYes

Conference

Title2011 30th IEEE International Symposium on Reliable Distributed Systems
Abbreviated titleSRDS 2011
Conference number30
Duration4 - 7 October 2011
Degree of recognitionInternational event
CityMadrid
CountrySpain

External IDs

Scopus 83155189012

Keywords

Research priority areas of TU Dresden

DFG Classification of Subject Areas according to Review Boards

Sustainable Development Goals

Keywords

  • repliication, Fault Tolerance, energy efficiency, mapreduce