Deep learning applications are actively used in various types of applications. The amount of computation required for inference and training is increasing rapidly, and at the same time, the demand for deploying these applications on mobile devices is also increasing. Deep learning optimization is becoming more and more important. In this course, we will learn advanced optimization techniques from the perspective of both hardware and software. By learning various studies about how to optimize deep learning algorithms, we could acquire knowledge that will be helpful in future research.

CSED442, CSED311

Mid-term (35) Final (35) Attendance (10) Paper Assignment (10) Q&A (10)

Introduction, DNN basics, GPU architecture
Class introduction, Deep learning basics (convolutional neural network)
DNN vision & language applications and corrsponding optimizations
GPU architecture and CNN Acceleration
Transfer learning and Online training
Selective execution
Neural Architecture search
Quantization, Pruning, and Low rank approximation (LRA)
Hardware accelerators

After giving lectures in each chapter, we will review the state-of-the-art studies.