Auto-scaling Techniques for Elastic Data Stream Processing
Research output: Contribution to book/Conference proceedings/Anthology/Report › Conference contribution › Contributed › peer-review
Contributors
Abstract
An elastic data stream processing system is able
to handle changes in workload by dynamically scaling out and
scaling in. This allows for handling of unexpected load spikes
without the need for constant overprovisioning. One of the
major challenges for an elastic system is to find the right point
in time to scale in or to scale out. Finding such a point is
difficult as it depends on constantly changing workload and
system characteristics. In this paper we investigate the application
of different auto-scaling techniques for solving this problem.
Specifically: (1) we formulate basic requirements for an auto-
scaling technique used in an elastic data stream processing
system, (2) we use the formulated requirements to select the
best auto scaling techniques, and (3) we perform evaluation of
the selected auto scaling techniques using the real world data.
Our experiments show that the auto scaling techniques used in
existing elastic data stream processing systems are performing
worse than the strategies used in our work.
to handle changes in workload by dynamically scaling out and
scaling in. This allows for handling of unexpected load spikes
without the need for constant overprovisioning. One of the
major challenges for an elastic system is to find the right point
in time to scale in or to scale out. Finding such a point is
difficult as it depends on constantly changing workload and
system characteristics. In this paper we investigate the application
of different auto-scaling techniques for solving this problem.
Specifically: (1) we formulate basic requirements for an auto-
scaling technique used in an elastic data stream processing
system, (2) we use the formulated requirements to select the
best auto scaling techniques, and (3) we perform evaluation of
the selected auto scaling techniques using the real world data.
Our experiments show that the auto scaling techniques used in
existing elastic data stream processing systems are performing
worse than the strategies used in our work.
Details
| Original language | English |
|---|---|
| Title of host publication | 9th International Workshop on Self-Managing Database Systems (SMDB 2014) |
| Publisher | IEEE Computational Intelligence Society (CIS) |
| Number of pages | 7 |
| Publication status | Published - 2014 |
| Peer-reviewed | Yes |