Keynote 2
Circuits and Packaging for Next Generation AI Datacenters

Tom Gray , NVIDIA

AI applications on GPU systems have exploded with an almost unlimited appetite for compute. Single-chip inference performance has increased more than 50,000X over the last 13 years with improvements ranging from architecture to circuits. Hundreds of thousands of datacenter connected GPUs are needed for training and inference of state-of-the-art agentic AI models. Interconnect, power delivery, and thermals are strained at all levels from on-die to datacenter with bandwidth density requirements increase on the order of 2x in each generation. This creates a critical need for future research addressing scaling and power reduction in all areas including compute/memory, communication, power delivery, and heat removal. In this talk, we will explore what is ultimately needed for design and deployment of these systems at the circuit and hardware level including compute, optical and electrical interconnect, power delivery, thermal management, memory system design, packaging and stacking. We will also explore how the AI enabled by these datacenters enables the design and implementation of future circuits and systems.

C. Thomas Gray received the B.S. degree in computer science and mathematics from the Mississippi College, Clinton, MS, and the M.S. and Ph.D. degrees in computer engineering from North Carolina State University, Raleigh, NC. He has held various design and architecture positions at IBM, Cadence Design Systems, Artisan, ARM, and Nethra Imaging. His work experience includes digital signal processing design and CMOS implementation of DSP blocks, as well as high-speed serial-link communication systems, architectures, and implementation. In 2011, he joined NVIDIA, Durham, NC where he is currently the Sr. Director of Circuits Research leading activities related to high-speed electrical and photonic signaling, security circuits, low-energy and resilient memories, circuits for AI computation, power delivery, and thermal management.