From Microbenchmarks to LLM Inference an End-To-End Analysis on the Energy Efficiency of the Grace Hopper Superchip
Publikation: Beitrag in Buch/Konferenzbericht/Sammelband/Gutachten › Beitrag in Konferenzband › Beigetragen › Begutachtung
Beitragende
Abstract
The Nvidia Grace Hopper Superchip has seen a wide adoption across both HPC sites and AI data centers. This hybrid architecture places the Grace CPU and Hopper GPU on the same board, with a shared and adjustable power budget. We present an end-to-end analysis of its energy efficiency, from a verification of power sensor accuracy using microbenchmarks, an investigation of power knob effects to an energy-efficiency and TCO investigation for selected Large Language Model (LLM) inference use cases. Our study shows that the internal power sensors provide precise readings when compared to an external power meter, unlike contemporary x86 processors. Some power budget configurations are not enforced by the system, the power draw can exceed the configured limit. This behavior is not documented by Nvidia. We analyzed the energy efficiency of LLM inference for prefill and decoding-dominated workloads, with the latter having emerged as the driving factor. Increasing the module power limit while maintaining a constant GPU power did not affect the system's energy efficiency. According to our TCO analysis, no cost savings can be expected from reduced power limits for output lengths of up to 2048 tokens for our LLM setup.
Details
| Originalsprache | Englisch |
|---|---|
| Titel | Proceedings - 2026 25th International Symposium on Parallel and Distributed Computing, ISPDC 2026 |
| Herausgeber (Verlag) | Institute of Electrical and Electronics Engineers (IEEE) |
| Seiten | 121-129 |
| Seitenumfang | 9 |
| ISBN (elektronisch) | 979-8-3195-3432-3 |
| ISBN (Print) | 979-8-3195-3433-0 |
| Publikationsstatus | Veröffentlicht - 23 Juli 2026 |
| Peer-Review-Status | Ja |
Publikationsreihe
| Reihe | International Symposium on Parallel and Distributed Computing |
|---|
Konferenz
| Titel | 25th International Symposium on Parallel and Distributed Computing |
|---|---|
| Kurztitel | ISPDC 2026 |
| Veranstaltungsnummer | 25 |
| Dauer | 1 - 3 Juli 2026 |
| Webseite | |
| Ort | Deutsches Elektronen-Synchrotron (DESY) |
| Stadt | Hamburg |
| Land | Deutschland |
Externe IDs
| ORCID | /0000-0002-8491-770X/work/224856192 |
|---|---|
| ORCID | /0000-0002-2730-0308/work/224857327 |
Schlagworte
Ziele für nachhaltige Entwicklung
- SDG 7 – Erschwingliche und saubere Energie - benchmarking, Energy Efficiency, Nvidia Grace Hopper
ASJC Scopus Sachgebiete
- Computernetzwerke und -kommunikation - benchmarking, Energy Efficiency, Nvidia Grace Hopper
- Hardware und Architektur - benchmarking, Energy Efficiency, Nvidia Grace Hopper
- Informationssysteme und -management - benchmarking, Energy Efficiency, Nvidia Grace Hopper
Schlagwörter
- benchmarking, Energy Efficiency, Nvidia Grace Hopper