Close

Presentation

Input-Dependent Power Usage in GPUs
DescriptionGPUs are known to be power-hungry, and due to the boom in artificial intelligence, they are the major contributors to the high power demands of datacenters. Most GPU usage in these popular workloads consists of large general matrix-matrix multiplications (GEMMs), which have therefore been optimized to achieve high utilization of hardware resources.

We show that modifying the input data to GEMMs, while maintaining the matrix shapes and sizes can notably change the power consumption of these kernels. We experiment with four kinds of input variations: value distribution, bit similarity, placement, and sparsity, across different data types. Our findings indicate that these variations can change the GPU power usage during GEMM by almost 40%.

We hypothesize that input-dependent power usage variations occur due to changes in the number of bit flips in the GPUs. We propose leveraging this property through compiler and scheduler optimizations to manage power and reduce energy consumption.
Event Type
Workshop
TimeSunday, 17 November 20244:30pm - 4:50pm EST
LocationB312
Tags
Energy Efficiency
HPC Infrastructure
Sustainability
Registration Categories
W