From Microbenchmarks to LLM Inference an End-To-End Analysis on the Energy Efficiency of the Grace Hopper Superchip
Research output: Contribution to book/Conference proceedings/Anthology/Report › Conference contribution › Contributed › peer-review
Contributors
Abstract
The Nvidia Grace Hopper Superchip has seen a wide adoption across both HPC sites and AI data centers. This hybrid architecture places the Grace CPU and Hopper GPU on the same board, with a shared and adjustable power budget. We present an end-to-end analysis of its energy efficiency, from a verification of power sensor accuracy using microbenchmarks, an investigation of power knob effects to an energy-efficiency and TCO investigation for selected Large Language Model (LLM) inference use cases. Our study shows that the internal power sensors provide precise readings when compared to an external power meter, unlike contemporary x86 processors. Some power budget configurations are not enforced by the system, the power draw can exceed the configured limit. This behavior is not documented by Nvidia. We analyzed the energy efficiency of LLM inference for prefill and decoding-dominated workloads, with the latter having emerged as the driving factor. Increasing the module power limit while maintaining a constant GPU power did not affect the system's energy efficiency. According to our TCO analysis, no cost savings can be expected from reduced power limits for output lengths of up to 2048 tokens for our LLM setup.
Details
| Original language | English |
|---|---|
| Title of host publication | Proceedings - 2026 25th International Symposium on Parallel and Distributed Computing, ISPDC 2026 |
| Publisher | Institute of Electrical and Electronics Engineers (IEEE) |
| Pages | 121-129 |
| Number of pages | 9 |
| ISBN (electronic) | 979-8-3195-3432-3 |
| ISBN (print) | 979-8-3195-3433-0 |
| Publication status | Published - 23 Jul 2026 |
| Peer-reviewed | Yes |
Publication series
| Series | International Symposium on Parallel and Distributed Computing |
|---|
Conference
| Title | 25th International Symposium on Parallel and Distributed Computing |
|---|---|
| Abbreviated title | ISPDC 2026 |
| Conference number | 25 |
| Duration | 1 - 3 July 2026 |
| Website | |
| Location | Deutsches Elektronen-Synchrotron (DESY) |
| City | Hamburg |
| Country | Germany |
External IDs
| ORCID | /0000-0002-8491-770X/work/224856192 |
|---|---|
| ORCID | /0000-0002-2730-0308/work/224857327 |
Keywords
Sustainable Development Goals
ASJC Scopus subject areas
Keywords
- benchmarking, Energy Efficiency, Nvidia Grace Hopper