Pipeline Group Optimization on Disaggregated Systems

Publikation: Beitrag zu KonferenzenPaperBeigetragen

Beitragende

Abstract

While hardware disaggregation is considered the "next big thing" providing unique opportunities for database systems, the pipeline-based execution model is state-of-the-art in modern query engines on monolithic systems. Within this paper, we propose a lightweight way of adapting this pipeline-based model to disaggregated memory systems to soften the inherent overhead induced by arbitrary memory accesses. Instead of executing pipelines in strict isolation including a pipeline-local data transfer, we group pipelines with similar data access characteristics of concurrently running queries into pipeline groups. Each such pipeline group is then executed separately, but shared data across pipelines within each group is only transferred once from memory resources to compute resources and potentially re-used multiple times. This method dramatically reduces redundant data transfers and - in combination with a suitable caching strategy as well as a fast communication layer - increases the performance significantly in comparison to traditional pipeline-based execution of multiple queries.

Details

OriginalspracheEnglisch
PublikationsstatusVeröffentlicht - 2023
Peer-Review-StatusNein

Konferenz

Titel13th Annual Conference on Innovative Data Systems Research
KurztitelCIDR 2023
Veranstaltungsnummer13
Dauer8 - 11 Januar 2023
Webseite
OrtMövenpick Hotel Amsterdam City Centre & Online
StadtAmsterdam
LandNiederlande

Externe IDs

Scopus 85166217989