Experimental Evaluation of Optimizing Memory Consumption in SAP HANA using PEOopt

Research output: Contribution to book/Conference proceedings/Anthology/ReportConference contributionContributedpeer-review

Contributors

Abstract

Materialized intermediate results dominate memory consumption in query processing, which is a challenge in the age of Big Data workloads. Modern query processors use pipelines to produce materialized intermediate results. The order by which pipelines are executed influence the lifetime of materialized intermediate results, directly impacting the overall consumed memory. By scheduling the execution of pipelines, we can therefore save memory. Our previous work provided a proof-of-concept for pipeline execution ordering to reduce memory consumption during runtime using a simple join query workload. This paper presents the experimental evaluation of an end-to-end implementation prototype called PEOopt within SAP HANA, which generalizes our previous work. For long running queries our prototype reduces the memory footprint by up to 36% without compromising any execution time performance. Hence, we prove the effectiveness of saving memory in real world database systems like SAP HANA.

Details

Original languageEnglish
Title of host publicationSIGMOD-Companion 2025 - Companion of the 2025 International Conference on Management of Data
EditorsAmol Deshpande, Ashraf Aboulnaga, Babak Salimi, Badrish Chandramouli, Bill Howe, Boon Thau Loo, Boris Glavic, Carlo Curino, Daisy Zhe Wang, Dan Suciu, Daniel Abadi, Divesh Srivastava, Eugene Wu, Faisal Nawab, Ihab Ilyas, Jeffrey Naughton, Jennie Rogers, Jignesh Patel, Joy Arulraj, Jun Yang, Karima Echihabi, Kenneth Ross, Khuzaima Daudjee, Laks Lakshmanan, Minos Garofalakis, Mirek Riedewald, Mohamed Mokbel, Mourad Ouzzani, Oliver Kennedy, Oliver Kennedy, Paolo Papotti, Peter Alvaro, Peter Bailis, Renee Miller, Senjuti Basu Roy, Sergey Melnik, Stratos Idreos, Sudeepa Roy, Theodoros Rekatsinas, Viktor Leis, Wenchao Zhou, Wolfgang Gatterbauer, Zack Ives
PublisherAssociation for Computing Machinery
Pages499-511
Number of pages13
ISBN (electronic)979-8-4007-1564-8
Publication statusPublished - 22 Jun 2025
Peer-reviewedYes

Conference

Title2025 ACM SIGMOD/PODS International Conference on Management of Data
Abbreviated titleSIGMOD/PODS 2025
Duration22 - 27 June 2025
Website
LocationIntercontinental Berlin
CityBerlin
CountryGermany

External IDs

ORCID /0000-0001-8107-2775/work/194824062

Keywords

ASJC Scopus subject areas

Keywords

  • main memory database system, memory optimization, pipeline, query optimization, query processing