dCSR: A Memory-Efficient Sparse Matrix Representation for Parallel Neural Network Inference.

Publikation: Beitrag in Buch/Konferenzbericht/Sammelband/GutachtenBeitrag in KonferenzbandBeigetragenBegutachtung

Beitragende

Abstract

Reducing the memory footprint of neural networks is a crucial prerequisite for deploying them in small and low-cost embedded devices. Network parameters can often be reduced significantly through pruning. We discuss how to best represent the indexing overhead of sparse networks for the coming generation of Single Instruction, Multiple Data (SIMD)-capable microcontrollers. From this, we develop Delta-Compressed Storage Row (dCSR), a storage format for sparse matrices that allows for both low overhead storage and fast inference on embedded systems with wide SIMD units. We demonstrate our method on an ARM Cortex-M55 MCV prototype with M-Profile Vector Extension (MVE). A comparison of memory consumption and throughput shows that our method achieves competitive compression ratios and increases throughput over dense methods by up to 2.9x for sparse matrix-vector multiplication (SpMV)-based kernels and 1.06x for sparse matrix-matrix multiplication (SpMM). This is accomplished through handling the generation of index information directly in the SIMD unit, leading to an increase in effective memory bandwidth.

Details

OriginalspracheEnglisch
Titel2021 40th IEEE/ACM International Conference on Computer-Aided Design, ICCAD 2021 - Proceedings
Seiten1-9
Seitenumfang9
PublikationsstatusVeröffentlicht - 2021
Peer-Review-StatusJa

Externe IDs

Scopus 85124129859

Schlagworte

Forschungsprofillinien der TU Dresden

Schlagwörter

  • Compression, Embedded systems, Pruning, Simd, Sparse neural networks