Auto-scaling Techniques for Elastic Data Stream Processing (Demo)

Research output: Contribution to conferencesPaperContributedpeer-review

Contributors

Abstract

Typical use cases like financial trading or monitoring of manufacturing equipment pose huge challenges regarding end to end latency as well as throughput towards existing data stream processing systems. Established solutions like Apache S4 or Storm need to scale out to a large set of hosts to meet these challenges. An ideal system can react to workload changes by on demand acquisition or release of hosts. Thereby, it can handle unexpected peak loads as well as improve the average utilization of the system. This property is called elasticity.

The major challenge for an elastic scaling system is to find the right point in time to scale in or out. To determine this right point is difficult, because it depends on constantly changing system and workload characteristics. In this demonstration, we apply three alternative auto-scaling techniques known from other domains on top of an existing elastic data stream processing system. A user of the demonstration can experience the influence of the chosen auto-scaling technique on the latency and the system utilization using a real-world use case based on different workloads from the Frankfurt stock exchange.

Details

Original languageEnglish
Pages318-321
Number of pages4
Publication statusPublished - 2014
Peer-reviewedYes

Conference

Title8th ACM International Conference on Distributed Event-Based Systems (DEBS '14), ACM, 2014
Abbreviated titleDEBS'14
Conference number
Duration26 - 29 May 2014
Degree of recognitionInternational event
Location
CityNew York
CountryUnited States of America

External IDs

Scopus 84903197105

Keywords

Research priority areas of TU Dresden

DFG Classification of Subject Areas according to Review Boards