Grand Challenge: Real-time Social Network Graph Analysis using StreamMine3G

Publikation: Beitrag zu KonferenzenPaperBeigetragenBegutachtung

Beitragende

Abstract

In this paper, we present our approach for solving the DEBS Grand Challenge 2016 using StreamMine3G, a distributed, highly scalable, elastic and fault tolerant event stream processing (ESP) system. We first provide an overview about StreamMine3G with regards to its programming model and architecture, followed by thorough description of the implementation for the two queries that provide up-to-date information about (i) the top-3 active posts and (ii) the top-k comments with the largest maximum cliques. Novel aspects of our implementation include (i) highly optimized data structures that lower the amount of lookups and traversals, and a (ii) deterministic data partitioning and processing scheme that allows the system to scale without bounds in an elastic fashion while still guaranteeing semantic transparency. In order to better utilize nowadays many-core machines, we furthermore propose a pipelining scheme in addition to data partitioning. Finally, we present a brief performance evaluation of our system.

Details

OriginalspracheEnglisch
Seiten322-329
Seitenumfang8
PublikationsstatusVeröffentlicht - 2016
Peer-Review-StatusJa

Konferenz

Titel10th ACM International Conference on Distributed Event-Based Systems
KurztitelDEBS '16
Dauer20 - 24 Juni 2016
BekanntheitsgradInternationale Veranstaltung
StadtIrvine
LandUSA/Vereinigte Staaten

Externe IDs

Scopus 84978698318

Schlagworte

Forschungsprofillinien der TU Dresden

DFG-Fachsystematik nach Fachkollegium

Schlagwörter

  • CEP, ESP, complex event processing, event stream processing, fault tolerance, migration, scalability, state management