EPEPS 2020: San Jose, CA
Tutorial I
Tutorial on Optimizing Machine Learning for Hardware

Warren Gross and Brett Meyer, McGill University

Machine learning (ML) is a key technology, with wide and growing application. Every layer in the computer system design stack has been touched by machine learning: from ML-enabled applications, to ML accelerators in hardware, to ML-enabled design tools, learning algorithms are leaving an indelible mark on computing.

This tutorial, presented by Professors Warren J. Gross and Brett H. Meyer from the Department of Electrical and Computer Engineering at McGill University, will provide an introduction to machine learning and ML algorithm optimization.

Professor Gross will begin by introducing the basics of neural networks and deep learning. He will then describe methodologies for the design of deep-learning accelerators and will give an overview of the main techniques used to efficiently map the computations used in deep learning to hardware. He will then describe recent work in custom hardware accelerators for deep learning.

Professor Meyer will continue by introducing how machine learning algorithms, and in particular, deep learning, can be optimized for more efficient execution. He'll discuss the typical constraints of hardware running ML applications, and the related metrics used for optimization. He will then revisit deep learning hyperparameters, introduce hyperparameter optimization, and present recent results optimizing computer vision and natural language processing tasks.

Warren J. Gross received the B.A.Sc. degree in electrical engineering from the University of Waterloo, Waterloo, ON, Canada, in 1996, and the M.A.Sc. and Ph.D. degrees from the University of Toronto, Toronto, ON, Canada, in 1999 and 2003, respectively. He is a Professor and Louis-Ho Faculty Scholar in Technological Innovation in the Department of Electrical and Computer Engineering, McGill University, Montreal, QC, Canada. He currently serves as Chair of the Department. His research interests are in the design and implementation of signal processing systems and custom computer architectures. Dr. Gross served as the Chair for the IEEE Signal Processing Society Technical Committee on Design and Implementation of Signal Processing Systems. He served as the General Co-Chair for the IEEE GlobalSIP 2017 and the IEEE SiPS 2017 and the Technical Program Co-Chair for SiPS 2012. He served as an Associate Editor for the IEEE Transactions on Signal Processing and as a Senior Area Editor. He is a recipient of the William and Rhea Seath Award in Engineering Innovation. He is a Licensed Professional Engineer in the Province of Ontario

Brett H. Meyer is an Associate Professor in the Department of Electrical and Computer Engineering at McGill University. He received his MS and PhD in Electrical and Computer Engineering from Carnegie Mellon University in 2005 and 2009, respectively. He received his BS in Electrical Engineering, Computer Science and Math from the University of Wisconsin- Madison in 2003. After receiving his PhD, Meyer worked as a post-doctoral research associate in the Computer Science Department at the University of Virginia. He has been on the faculty at McGill since 2011. Meyer's research interests are diverse, but tend toward architectures and design algorithms for embedded computer systems; embedded system security and machine learning are new areas of interest. Meyer's research has been recognized with Best Paper in Session awards at SRC TECHCON (2007 and 2013), nominations for Best Paper at ASPDAC 2014 and GLSVLSI 2015, and a Best Paper Award at DFT 2015. He received the William and Rhea Seath Award in Engineering Innovation in 2018. He has served on the technical program committees for CASES, CODES+ISSS, DAC, DATE, GLSVLSI, ICCAD, ISLPED, among others, and as TPC chair for AHS 2015