Online Parameter Optimization for Elastic Data Stream Processing
Publikation: Beitrag in Buch/Konferenzbericht/Sammelband/Gutachten › Beitrag in Konferenzband › Beigetragen › Begutachtung
Beitragende
Abstract
Elastic scaling allows data stream processing systems to dynamically scale in and out to react to workload changes. As a consequence, unexpected load peaks can be handled and the extent of the overprovisioning can be reduced. However, the strategies used for elastic scaling of such systems need to be tuned manually by the user. This is an error prone and cumbersome task, because it requires a detailed knowledge of the underlying system and workload characteristics. In addition, the resulting quality of service for a specific scaling strategy is unknown a priori and can be measured only during runtime. In this paper we present an elastic scaling data stream processing prototype, which allows to trade off monetary cost against the offered quality of service. To that end, we use an online parameter optimization, which minimizes the monetary cost for the user. Using our prototype a user is able to specify the expected quality of service as an input to the optimization, which automatically detects significant changes of the workload pattern and adjusts the elastic scaling strategy based on the current workload characteristics. Our prototype is able to reduce the costs for three real-world use cases by 19% compared to a naive parameter setting and by 10% compared to a manually tuned system. In contrast to state of the art solutions, our system provides a stable and good trade-off between monetary cost and quality of service.
Details
Originalsprache | Englisch |
---|---|
Titel | SoCC '15: Proceedings of the Sixth ACM Symposium on Cloud Computing |
Herausgeber (Verlag) | Association for Computing Machinery (ACM), New York |
Seiten | 276-287 |
Seitenumfang | 12 |
ISBN (Print) | 978-1-4503-3651-2 |
Publikationsstatus | Veröffentlicht - 2015 |
Peer-Review-Status | Ja |
Publikationsreihe
Reihe | MOD: International Conference on Management of Data (SoCC) |
---|
Externe IDs
Scopus | 84959036559 |
---|
Schlagworte
Forschungsprofillinien der TU Dresden
DFG-Fachsystematik nach Fachkollegium
Schlagwörter
- Distributed applications, Distributed Data Stream Processiong, Load Balancing, Elasticity, Parameter Optimization, Distributed data Stream Processing, Load Balancing, Elasticity, parameter optimization