Small Selectivities Matter: Lifting the Burden of Empty Samples

Research output: Contribution to book/Conference proceedings/Anthology/ReportConference contributionContributedpeer-review

Contributors

  • Axel Hertzschuch - , Chair of Databases (Author)
  • Guido Moerkotte - , University of Mannheim (Author)
  • Wolfgang Lehner - , Chair of Databases (Author)
  • Norman May - , SAP Research (Author)
  • Florian Wolf - , SAP Research (Author)
  • Lars Fricke - , SAP Research (Author)

Abstract

Every year more and more advanced approaches to cardinality estimation are published, using learned models or other data and workload specific synopses. In contrast, the majority of commercial in-memory systems still relies on sampling. It is arguably the most general and easiest estimator to implement. While most methods do not seem to improve much over sampling-based estimators in the presence of non-selective queries, sampling struggles with highly selective queries due to limitations of the sample size. Especially in situations where no sample tuple qualifies, optimizers fall back to basic heuristics that ignore attribute correlations and lead to large estimation errors. In this work, we present a novel approach, dealing with these 0-Tuple Situations. It is ready to use in any DBMS capable of sampling, showing a negligible impact on optimization time. Our experiments on real world and synthetic data sets demonstrate up to two orders of magnitude reduced estimation errors. Enumerating single filter predicates according to our estimates reveals 1.3 to 1.8 times faster query responses for complex filters.

Details

Original languageEnglish
Title of host publicationSIGMOD '21: Proceedings of the 2021 International Conference on Management of Data
PublisherAssociation for Computing Machinery, Inc
Pages697-709
Number of pages13
Publication statusPublished - 2021
Peer-reviewedYes

Conference

Title2021 International Conference on Management of Data, SIGMOD 2021
Duration20 - 25 June 2021
CityVirtual, Online
CountryChina

External IDs

Scopus 85108979368
ORCID /0000-0001-8107-2775/work/142253441

Keywords

ASJC Scopus subject areas

Keywords

  • beta distribution, filter predicate ordering, in-memory, olap, sampling, small selectivity