This course aims at studying basic theory and practical algorithms of reinforcement learning (RL) so that being capable of understanding research papers on RL, applying RL techniques to problems in other fields, and, hopefully, formulating/solving research problems in RL.

Mandatory: machine learning, calculus, probability & statistics
Recommend: optimization, programming

The grade is based on assignments (35%), midterm exam (20%), paper review (20%), proposal (20%), participation (5%), while there is policy of 5-strike out for attendance, i.e., 5 absences lead to F, no matter what (email me in advance of inevitable absence).

There will be no official textbook. However, most of contents are based on the following books:
- “Bandit algorithm” by T. Lattimore and C. Szepesvari
- “Reinforcement Learning: An Introduction”, by R. S. Sutton and A. G. Barto, MIT Press, 2018, (link to draft)

[Tentative Syllabus]
https://docs.google.com/spreadsheets/d/18p6JMZ76PCBQzcQnlnpOfe2_MuHv31ICtBGkBjMRynU/edit?usp=sharing

1. Introduction to RL
2. Multi-Armed Bandit
3. Regret Analysis in MAB
4. Sample Complexity in MAB
5. Markov Decision Process (MDP)
6. Dynamic Programming
7. Regret Minimization in MDP
8. Sampling Schemes
9. Temporal Difference learning
10. n-step Bootstrapping
11. Function Approximation
12. Deep Q-Network
13. Eligibility trace
14/15. Policy gradient method (Buddha's birthday / Childern's day - video lecture or scheduling make-up class)
16. Midterm Exam (9:30am~12:30pm )
17. Optimization techniques for RL
18. Scaling RL
19. Exploration via Intrinsic Motivation
20. Partially Observable RL
21. Bayesian and meta RL
22. Multitask and Hierarchical RL
23. Multi-agent RL
24. Adversarial Search: Alphago
25. Imitation learning and inverse RL
26. Application of RL
27-30. Final project presentation and guest lecture

Online.

.