Scalable and Low-Latency Data Processing with StreamMapReduce
Publikation: Beitrag zu Konferenzen › Paper › Beigetragen › Begutachtung
Beitragende
Abstract
We present StreamMapReduce, a data processing approach that combines ideas from the popular MapReduce paradigm and recent developments in Event Stream Processing. We adopted the simple and scalable programming model of MapReduce and added continuous, low-latency data processing capabilities previously found only in Event Stream Processing systems. This combination leads to a system that is efficient and scalable, but at the same time, simple from the user's point of view. For latency-critical applications, our system allows a hundred-fold improvement in response time. Notwithstanding, when throughput is considered, our system offers a ten-fold per node throughput increase in comparison to Hadoop. As a result, we show that our approach addresses classes of applications that are not supported by any other existing system and that the MapReduce paradigm is indeed suitable for scalable processing of real-time data streams.
Details
Originalsprache | Englisch |
---|---|
Seiten | 48-58 |
Seitenumfang | 11 |
Publikationsstatus | Veröffentlicht - 2011 |
Peer-Review-Status | Ja |
Konferenz
Titel | 3rd IEEE International Conference on Cloud Computing Technology and Science |
---|---|
Kurztitel | CloudCom 2011 |
Veranstaltungsnummer | 3 |
Dauer | 29 November - 1 Dezember 2011 |
Webseite | |
Bekanntheitsgrad | Internationale Veranstaltung |
Stadt | Athen |
Land | Griechenland |
Externe IDs
Scopus | 84857167165 |
---|
Schlagworte
Forschungsprofillinien der TU Dresden
DFG-Fachsystematik nach Fachkollegium
Schlagwörter
- Programming, Fault Tolerance, Fault tolerant systems, Scalability, Databases, Ral time systems, Instruction sets, data handling, distributed algorithms, event stream processing, complex event processing, MapReduce