Thesis Part 1: Profiling LLM Inference Energy

Sun, 26 Apr 2026 00:00:00 +0000

Recently I’ve been writing my MSc thesis at the EPCC on the energy cost of memory stalls during LLM inference. It was a self-proposed thesis, and I did a bunch of experiments and research to try and find a real literature gap. Although my time at the EPCC has focused on HPC and programming performance and efficiency at scale, I have enjoyed modern MlSys research. I wanted to help contribute to that field and begin research in this direction. So, the premise is fairly straightforward: when a GPU runs out of local HBM, and has to fetch the KV cache over PCIe, the SMs sit idle for microseconds at a time, fully energised, drawing power for no useful work. I wanted to measure that wasted energy directly. That’s not what this post is about, though.

GPU on Akeel Ather Medina

Thesis Part 1: Profiling LLM Inference Energy