Elastic Complex Event Processing under Varying Query Load
Research output: Other contribution › Other › Contributed › peer-review
Contributors
Abstract
Distributed data stream processing systems, like Twitter
Storm or Yahoo! S4, have been primarily focusing on adapt-
ing to varying event rates. However, as these systems are
becoming increasingly multi-tenant, adaptation to the vary-
ing query load is becoming an equally important problem.
In this paper we present FUGU – an elastic allocator for
Complex Event Processing systems. FUGU uses bin pack-
ing to allocate continuous queries to a varying set of nodes.
Driven by elasticity requirements FUGU maximizes the over-
all system utilization while trying to maintain stable process-
ing latencies.
The specific contributions of this paper are: (1) introduc-
tion of a re-balancing scheme for bin packing allowing FUGU
to increase overall system utilization by six percent and (2)
a detailed study of achievable system utilization and latency
under real-life workload from Frankfurt Stock Exchange.
Storm or Yahoo! S4, have been primarily focusing on adapt-
ing to varying event rates. However, as these systems are
becoming increasingly multi-tenant, adaptation to the vary-
ing query load is becoming an equally important problem.
In this paper we present FUGU – an elastic allocator for
Complex Event Processing systems. FUGU uses bin pack-
ing to allocate continuous queries to a varying set of nodes.
Driven by elasticity requirements FUGU maximizes the over-
all system utilization while trying to maintain stable process-
ing latencies.
The specific contributions of this paper are: (1) introduc-
tion of a re-balancing scheme for bin packing allowing FUGU
to increase overall system utilization by six percent and (2)
a detailed study of achievable system utilization and latency
under real-life workload from Frankfurt Stock Exchange.
Details
Original language | English |
---|---|
Number of pages | 6 |
Publication status | Published - 2013 |
Peer-reviewed | Yes |