The current prevalence of machine-learning and deep-learning applications deeply relies on its engineering aspect, i.e., "how" they are expressed and executed on computing systems.
This includes accessible language and programming models (e.g., TensorFlow, PyTorch), architectural innovations (e.g., GPUs, NPUs), and highly-optimized libraries (e.g., cuDNN, MKL).
As both applications and hardware for deep learning are increasingly complex and diverse, the role of compilers bridging the gap of abstractions levels between them is more crucial than ever.
In this class, we will explore recent research trends in deep learning programming and compiler frameworks (TensorFlow, TVM, MLIR, etc.) and discuss future research directions.

CSED423 (undergraduate compiler) and CSED702R (graduate compiler) are recommended prerequisites.

Participation: 10%
Discussion lead: 40%
Project proposal: 15%
Final project: 35%

Week 1: Introduction
Overview - DL frameworks and compilers
Week 2: IR overview
Front-end optimizations
Week 3: No class (Chuseok holiday)
Week 4: Back-end code generation
Auto-tuning
Week 5: TensorFlow XLA in depth
Week 6: TVM in depth
Week 7: MLIR in depth
Week 8: Project proposal presentation (midterm week)
Week 9: Paper discussion
Week 10: Paper discussion
Week 11: Paper discussion
Week 12: Paper discussion
Week 13: Paper discussion
Week 14: Paper discussion
Week 15: Review
Week 16: Final project presentation

- This class will have different structures for pre-midterm and post-midterm sections. During the first half of the semester, we will focus on understanding common components of deep learning compilers (e.g., front-end and back-end optimizations, IRs) and studying some of the systems in detail (e.g., TensorFlow XLA, TVM) through lectures. The second half of the semester will consist of student-led paper discussion sessions. In these sessions, a discussion leader briefly summarizes paper(s), shares their thoughts, and leads a classroom (online) discussion with comments and questions. Thus, active participation from the entire class is strongly encouraged.
- A list of discussion papers, organized as pairs, will be provided. Given a topic, each pair includes a recent notable work and earlier related research for understanding them in a historical context.
- There are no exams (midterm or final). Instead, students will work on a semester-long research project investigating and improving previous work in the related areas, including, but not limited to, domain-specific languages and compilers, graph-level optimizations, auto-tuning, and compiler support for specialized accelerators. I will provide a list of potential topics and papers and have individual meetings with students to confirm the subject. The project will be evaluated by its proposal presentation, final report, and final presentation.
- All talks and reports should be in English unless noted otherwise.
- To facilitate discussion sessions, students are advised to turn on cameras and speak up through mics during the sessions.