An online guided tuning approach to run CNN pipelines on edge devices

Publikation: Beitrag in Buch/Konferenzbericht/Sammelband/GutachtenBeitrag in KonferenzbandBeigetragenBegutachtung

Beitragende

  • Pirah Noor Soomro - , Chalmers University of Technology (Autor:in)
  • Mustafa Abduljabbar - , Chalmers University of Technology (Autor:in)
  • Jeronimo Castrillon - , Professur für Compilerbau (cfaed) (Autor:in)
  • Miquel Pericàs - , Chalmers University of Technology (Autor:in)

Abstract

Modern edge and mobile devices are equipped with powerful computing resources. These are often organized as heterogeneous multi-cores, featuring performance-asymmetric core clusters. This raises the question on how to effectively execute the inference pass of convolutional neural networks (CNN) on such devices. Existing CNN implementations on edge devices leverage offline profiling data to determine a better schedule for CNN applications. This approach requires a time consuming phase of generating a performance profile for each type of representative kernel on various core configurations available on the device, coupled with a search space exploration. We propose an online tuning technique which utilizes compile time hints and online profiling data to generate high throughput CNN pipelines. We explore core heterogeneity and compatible core-layer configurations through an online guided search. Unlike exhaustive search, we adopt an evolutionary approach with a guided starting point in order to find the solution. We show that by pruning and navigating through the complex search space using compile time hints, 79% of the tested configurations turn out to be near-optimal candidates for a throughput maximizing pipeline on NVIDIA Jetson TX2 platform.

Details

OriginalspracheEnglisch
TitelCF '21: Proceedings of the 18th ACM International Conference on Computing Frontiers
Herausgeber (Verlag)Association for Computing Machinery, Inc
Seiten45-53
Seitenumfang9
ISBN (elektronisch)978-1-4503-8404-9
PublikationsstatusVeröffentlicht - 11 Mai 2021
Peer-Review-StatusJa

Publikationsreihe

ReiheCF: Computing Frontiers Conference

Konferenz

Titel18th ACM International Conference on Computing Frontiers 2021, CF 2021
Dauer11 - 13 Mai 2021
StadtVirtual, Online
LandItalien

Externe IDs

ORCID /0000-0002-5007-445X/work/141545523

Schlagworte

Forschungsprofillinien der TU Dresden

ASJC Scopus Sachgebiete

Schlagwörter

  • CNN pipelines, design space exploration, edge devices, evolutionary algorithm, heterogeneous core clusters, online tuning, task moldability, task parallel runtimes