This course explores the theoretical foundations underlying modern machine learning systems, with particular emphasis on large language models. We begin by reviewing key concepts in statistical learning theory, including risk minimization, generalization bounds, and concentration inequalities. Building on this foundation, we examine theoretical perspectives on alignment via preference learning in LLMs, test-time scaling, reinforcement learning with verifiable rewards, and the role of the base model in exploration. Throughout, the course emphasizes mathematical clarity, conceptual insight, and the relationship between theory and empirical practice, enabling students to understand not only how modern systems work, but why they work.

Machine Learning

1. Introduction to learning theory
2. Backgrounds: concentration inequalities
3. Backgrounds: supervised learning
4. Backgrounds: convexity
5. Interactive machine learning
6. LLM alignment: RLHF and DPO
7. Theory of RLHF and DPO
8. Unified theory
9. Test-time scaling
10. Theory of imperfect process verifiers
11. Representation-Based Exploration for Language Models
12. The Computational Role of the Base Model in Exploration
13. Student presentation
14. Student presentation
15. Student presentation
16. Student presentation
(This class is experimental; the schedule and content listed above are subject to changes)

- Lecture Type: Lecture, Student Presentation
- Academic Integrity: Penalty for cheating – Removal from course with failing (F) grade
- This course consists of two main parts: lectures and student presentations. The lecture part provides in-depth knowledge and technologies required to analyze modern machine learning algorithms and develop novel ones. In the presentation part, enrolled students present latest articles published in top system conferences, such as ICML, NeurIPS, ICLR, COLT, AISTATS, UAI, AAAI, etc. A list of recommended papers for the presentation will be given, but students are free to choose papers outside the list. For each student presentation, there will be 2-3 other students assigned to write a paper critique about the paper being presented, and the presenter must address questions raised by these critiques.