Towards efficient multi-domain data processing

Research output: Contribution to book/Conference proceedings/Anthology/ReportConference contributionContributedpeer-review

Contributors

  • Johannes Luong - , TUD Dresden University of Technology (Author)
  • Dirk Habich - , TUD Dresden University of Technology (Author)
  • Thomas Kissinger - , TUD Dresden University of Technology (Author)
  • Wolfgang Lehner - , TUD Dresden University of Technology (Author)

Abstract

Economy and research increasingly depend on the timely analysis of large datasets to guide decision making. Complex analysis often involve a rich variety of data types and special purpose processing models. We believe, the database system of the future will use compilation techniques to translate specialized and abstract high level programming models into scalable low level operations on efficient physical data formats. We currently envision optimized relational and linear algebra languages, a flexible data flow language(A language inspired by the programming models of popular data flow engines like Apache Spark (spark.apache.org) or Apache Flink (flink.apache.org).) and scaleable physical operators and formats for relational and array data types. In this article, we propose a database system architecture that is designed around these ideas and we introduce our prototypical implementation of that architecture.

Details

Original languageEnglish
Title of host publicationData Management Technologies and Applications - 5th International Conference, DATA 2016, Revised Selected Papers
EditorsMarkus Helfert, Chiara Francalanci
PublisherSpringer Verlag
Pages47-64
Number of pages18
ISBN (print)9783319629100
Publication statusPublished - 2017
Peer-reviewedYes
Externally publishedYes

Publication series

SeriesCommunications in Computer and Information Science
Volume737
ISSN1865-0929

Conference

Title5th International Conference on Data Management Technologies and Applications, DATA 2016
Duration24 - 26 July 2016
CityColmar
CountryFrance

External IDs

ORCID /0000-0001-8107-2775/work/142253524

Keywords