ATUN-HL: Auto Tuning of Hybrid Layouts Using Workload and Data Characteristics

Publikation: Beitrag in Buch/Konferenzbericht/Sammelband/GutachtenBeitrag in KonferenzbandBeigetragenBegutachtung

Beitragende

  • Rana Faisal Munir - , Professur für Datenbanken, UPC Universitat Politècnica de Catalunya (Barcelona Tech) (Autor:in)
  • Alberto Abelló - , UPC Universitat Politècnica de Catalunya (Barcelona Tech) (Autor:in)
  • Oscar Romero - , UPC Universitat Politècnica de Catalunya (Barcelona Tech) (Autor:in)
  • Maik Thiele - , Professur für Datenbanken (Autor:in)
  • Wolfgang Lehner - , Professur für Datenbanken (Autor:in)

Abstract

Ad-hoc analysis implies processing data in near real-time. Thus, raw data (i.e., neither normalized nor transformed) is typically dumped into a distributed engine, where it is generally stored into a hybrid layout. Hybrid layouts divide data into horizontal partitions and inside each partition, data are stored vertically. They keep statistics for each horizontal partition and also support encoding (i.e., dictionary) and compression to reduce the size of the data. Their built-in support for many ad-hoc operations (i.e., selection, projection, aggregation, etc.) makes hybrid layouts the best choice for most operations. Horizontal partition and dictionary sizes of hybrid layouts are configurable and can directly impact the performance of analytical queries. Hence, their default configuration cannot be expected to be optimal for all scenarios. In this paper, we present ATUN-HL (Auto TUNing Hybrid Layouts), which based on a cost model and given the workload and the characteristics of data, finds the best values for these parameters. We prototyped ATUN-HL for Apache Parquet, which is an open source implementation of hybrid layouts in Hadoop Distributed File System, to show its effectiveness. Our experimental evaluation shows that ATUN-HL provides on average 85% of all the potential performance improvement, and 1.2x average speedup against default configuration.

Details

OriginalspracheEnglisch
TitelAdvances in Databases and Information Systems - 22nd European Conference, ADBIS 2018, Proceedings
Redakteure/-innenAndras Benczur, Tomas Horvath, Bernhard Thalheim
Herausgeber (Verlag)Springer, Berlin [u. a.]
Seiten200-215
Seitenumfang16
ISBN (Print)9783319983974
PublikationsstatusVeröffentlicht - 2018
Peer-Review-StatusJa

Publikationsreihe

ReiheLecture Notes in Computer Science, Volume 11019
ISSN0302-9743

Konferenz

Titel22nd East-European Conference on Advances in Databases and Information Systems, ADBIS 2018
Dauer2 - 5 September 2018
StadtBudapest
LandUngarn

Externe IDs

Scopus 85051071595
ORCID /0000-0001-8107-2775/work/142253506

Schlagworte

Schlagwörter

  • Auto tuning, Big data, Hybrid storage layouts, Parquet

Bibliotheksschlagworte