Prototyping of Low-Cost Configurable Sparse Neural Processing Unit with Buffer and Mixed-Precision Reshapeable MAC Array

Research output: Contribution to book/conference proceedings/anthology/reportConference contributionContributedpeer-review

Contributors

Abstract

More recently, it has become possible to run deep learning algorithms on edge devices such as microcontrollers due to continuous improvements in neural network optimization algorithms such as quantization and neural architecture search. Nonetheless, most of the embedded hardware available today still falls short of the requirements of running deep neural networks. As a result, specialized processors have emerged to improve the inference efficiency of deep learning algorithms. However, most are not for edge applications that require efficient and low-cost hardware. Therefore, we design and prototype a low-cost configurable sparse Neural Processing Unit (NPU). The NPU has a built-in buffer and a reshapable mixed-precision multiply-accumulator (MAC) array. The computing and memory resources of the NPU are parameterized, and different NPUs can be derived. Besides, users can also conFigure the NPU at runtime to fully utilize the resources. In our experiments, the 200MHz NPU with only 32 MACs is more than 32 times faster than the 400MHzSTM32H7 when inferring MobileNet-Vl. Besides, the yielded NPUs can achieve roofline or even beyond roofline performance. The buffer and reshapeable MAC array push the NPU's attainable performance to the roofline, while the feature of supporting sparsity allows the NPU to obtain performance beyond the roofline.

Details

Original languageEnglish
Title of host publicationProceedings - 2022 IEEE 28th International Conference on Parallel and Distributed Systems, ICPADS 2022
Place of PublicationNanjing
Pages712-719
Number of pages8
ISBN (electronic)978-1-6654-7315-6
Publication statusPublished - 2023
Peer-reviewedYes

Publication series

SeriesInternational Conference on Parallel and Distributed Systems (ICPADS)
ISSN1521-9097

External IDs

Scopus 85152928335

Keywords

ASJC Scopus subject areas

Keywords

  • configurable, low-cost, mixed-precision, neural processing unit, sparsity

Library keywords