Minimizing Latency in Fault-Tolerant Distributed Stream Processing Systems

Publikation: Beitrag zu KonferenzenPaperBeigetragenBegutachtung

Beitragende

Abstract

Event stream processing (ESP) applications target the real-time processing of huge amounts of data. Events traverse a graph of stream processing operators where the information of interest is extracted. As these applications gain popularity, the requirements for scalability, availability, and dependability increase. In terms of dependability and availability, many applications require a precise recovery, i.e., a guarantee that the outputs during and after a recovery would be the same as if the failure that triggered recovery had never occurred. Existing solutions for precise recovery induce prohibitive latency costs, either by requiring continuous checkpoint or logging (in a passive replication approach) or perfect synchronization between replicas executing the same operations (in an active replication approach). We introduce a novel technique to guarantee precise recovery for ESP applications while minimizing the latency costs as compared to traditional approaches. The technique minimizes latencies via speculative execution in a distributed system. In terms of scalability, the key component of our approach is a modified software transactional memory that provides not only the speculation capabilities but also optimistic parallelization for costly operations

Details

OriginalspracheEnglisch
Seiten173-182
Seitenumfang10
PublikationsstatusVeröffentlicht - 2009
Peer-Review-StatusJa

Konferenz

Titel2009 29th IEEE International Conference on Distributed Computing Systems
KurztitelICDCS 2009
Veranstaltungsnummer29
Dauer22 - 26 Juni 2009
BekanntheitsgradInternationale Veranstaltung
StadtMontreal
LandKanada

Externe IDs

Scopus 70350222222

Schlagworte

Forschungsprofillinien der TU Dresden

DFG-Fachsystematik nach Fachkollegium