Efficient Post-training Augmentation for Adaptive Inference in Heterogeneous and Distributed IoT Environments
Research output: Contribution to book/Conference proceedings/Anthology/Report › Conference contribution › Contributed › peer-review
Contributors
Abstract
Early Exit Neural Networks (EENNs) achieve enhanced efficiency compared to traditional models, but creating them is challenging due to the many additional design choices required. To address this, we propose an automated augmentation flow that converts existing models into EENNs, making all necessary design decisions for deployment on heterogeneous or distributed embedded targets. Our framework is the first to perform all these steps, including EENN architecture construction, subgraph mapping, and decision mechanism configuration. We evaluated our approach on embedded Deep Learning scenarios, achieving significant performance improvements. Our solution reduced latency by 65.95% on a speech command detection problem and mean operations per inference by 78.3% on an ECG classification task. This showcases the potential for EENNs in embedded applications.
Details
| Original language | English |
|---|---|
| Title of host publication | Embedded Computer Systems: Architectures, Modeling, and Simulation |
| Editors | Luigi Carro, Francesco Regazzoni, Christian Pilato |
| Publisher | Springer Science and Business Media B.V. |
| Pages | 99-108 |
| Number of pages | 10 |
| ISBN (electronic) | 978-3-031-78380-7 |
| ISBN (print) | 978-3-031-78379-1 |
| Publication status | Published - 2025 |
| Peer-reviewed | Yes |
Publication series
| Series | Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) |
|---|---|
| Volume | 15227 LNCS |
| ISSN | 0302-9743 |
Conference
| Title | 24th International Conference on Embedded Computer Systems: Architectures, Modeling, and Simulation |
|---|---|
| Abbreviated title | SAMOS 2024 |
| Conference number | 24 |
| Duration | 29 June - 4 July 2024 |
| Website | |
| Location | Doryssa Seaside Resort |
| City | Pythagorion |
| Country | Greece |
Keywords
ASJC Scopus subject areas
Keywords
- Deep Learning, Early Exit Neural Networks, Network Architecture Search