Latency-aware Elastic Scaling for Distributed Data Stream Processing Systems

Publikation: Beitrag zu KonferenzenPaperBeigetragenBegutachtung

Beitragende

Abstract

Elastic scaling allows a data stream processing system to react to a dynamically changing query or event workload by automatically scaling in or out. Thereby, both unpredictable load peaks as well as underload situations can be handled. However, each scaling decision comes with a latency penalty due to the required operator movements. Therefore, in practice an elastic system might be able to improve the system utilization, however it is not able to provide latency guarantees defined by a service level agreement (SLA).

In this paper we introduce an elastic scaling system, which optimizes the utilization under certain latency constraints defined by a SLA. Specifically, we present a model, which estimates the latency spike created by a set of operator movements. We use this model to built a latency-aware elastic operator placement algorithm, which minimizes the number of latency violations. We show that our solution is able to reduce the 90th percentile of the end to end latency by up to 30% and reduce the number of latency violations by 50%. The achieved system utilization for our approach is comparable to a scaling strategy, which does not use latency as optimization target.

Details

OriginalspracheEnglisch
Seiten13-22
Seitenumfang10
PublikationsstatusVeröffentlicht - 2014
Peer-Review-StatusJa

Konferenz

Titel8th ACM International Conference on Distributed Event-Based Systems (DEBS '14), ACM, 2014
KurztitelDEBS'14
Veranstaltungsnummer
Dauer26 - 29 Mai 2014
BekanntheitsgradInternationale Veranstaltung
Ort
StadtNew York
LandUSA/Vereinigte Staaten

Externe IDs

Scopus 84903161696

Schlagworte

Forschungsprofillinien der TU Dresden

DFG-Fachsystematik nach Fachkollegium