Efficient Post-training Augmentation for Adaptive Inference in Heterogeneous and Distributed IoT Environments

Research output: Contribution to book/Conference proceedings/Anthology/ReportConference contributionContributedpeer-review

Contributors

Abstract

Early Exit Neural Networks (EENNs) achieve enhanced efficiency compared to traditional models, but creating them is challenging due to the many additional design choices required. To address this, we propose an automated augmentation flow that converts existing models into EENNs, making all necessary design decisions for deployment on heterogeneous or distributed embedded targets. Our framework is the first to perform all these steps, including EENN architecture construction, subgraph mapping, and decision mechanism configuration. We evaluated our approach on embedded Deep Learning scenarios, achieving significant performance improvements. Our solution reduced latency by 65.95% on a speech command detection problem and mean operations per inference by 78.3% on an ECG classification task. This showcases the potential for EENNs in embedded applications.

Details

Original languageEnglish
Title of host publicationEmbedded Computer Systems: Architectures, Modeling, and Simulation
EditorsLuigi Carro, Francesco Regazzoni, Christian Pilato
PublisherSpringer Science and Business Media B.V.
Pages99-108
Number of pages10
ISBN (electronic)978-3-031-78380-7
ISBN (print)978-3-031-78379-1
Publication statusPublished - 2025
Peer-reviewedYes

Publication series

SeriesLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume15227 LNCS
ISSN0302-9743

Conference

Title24th International Conference on Embedded Computer Systems: Architectures, Modeling, and Simulation
Abbreviated titleSAMOS 2024
Conference number24
Duration29 June - 4 July 2024
Website
LocationDoryssa Seaside Resort
CityPythagorion
CountryGreece

Keywords

Keywords

  • Deep Learning, Early Exit Neural Networks, Network Architecture Search