LCTL - Lightweight Compression Template Library.

Research output: Contribution to conferencesPaperContributedpeer-review

Contributors

Abstract

For fast and efficient data processing, a common approach in many application domains is to mainly store and process data in form of arrays of integers offering several benefits. For example, with the help of some additional lightweight computations for lossless integer compression, the necessary memory space can be dramatically reduced. Moreover, compressed integer values offer advantages for data processing such as increasing the effective bandwidth to reduce the memory wall effect. Thus, a large corpus of lightweight integer compression formats has been developed. Unfortunately, there is no approach available that allows to define a specific format in a consistent way and to generate executable code for compression and decompression from it. To overcome that, we present the Lightweight Compression Template Library (LCTL) in this paper. As we are going to show, LCTL allows (i) the implementation of a variety of compression formats in an abstract way, (ii) the generation of efficient executable code for compression, and (iii) the automatic derivation of the decompression routines out of a compression format. Thus, LCTL offers a unique approach to comprehensively and systematically implement the large corpus of lightweight integer compression formats.

Details

Original languageEnglish
Pages2966-2975
Number of pages10
Publication statusPublished - 2021
Peer-reviewedYes

External IDs

Scopus 85125343705
ORCID /0000-0001-8107-2775/work/142253420

Keywords

Keywords

  • code generation, library, abstraction, template metaprogramming, compression, integer data