2022-Fall Information and Coding Theory (EECE577-01) The course syllabus

1.Course Information

Course No. EECE577 Section 01 Credit 3.00
Category Major elective Course Type Classroom Course prerequisites
Postechian Core Competence
Hours TUE, THU / 14:00 ~ 15:15 / Elec Bldg[102]Lecture Room Grading Scale G

2. Instructor Information

Kim Yongjune Name Kim Yongjune Department Dept. of Electrical Eng.
Email address yongjune@postech.ac.kr Homepage https://iil.postech.ac.kr
Office INFORMATION & INTELLIGENCE LAB Office Phone 054-279-2233
Office Hours By appointment

3. Course Objectives

This course offers an introduction to the information theory and its applications to reliable and efficient information systems.

1. Understand concepts such as entropy, divergence, mutual information;
2. Understand lossless data compression and lossy data compression;
3. Understand channel capacity and reliable data transmission;
4. Understand connections between information theory and machine learning.

4. Prerequisites & require

5. Grading

Homework: 20%
Class participation: 10%
Midterm exam: 30%
Final exam: 40%

6. Course Materials

Title Author Publisher Publication
Year/Edition		 ISBN
Elements of Information Theory (2nd ed.) T. Cover & J. Thomas Wiley 2005 978-0471241959

7. Course References

8. Course Plan

Week 1
Introduction and entropy
Week 2
Divergence, mutual information, and Fano’s inequality
Week 3
AEP and entropies of stochastic process
Week 4
Data compression and Huffman codes
Week 5
Lossless compression
Week 6
Channel capacity
Week 7
Channel coding theorem
Week 8
Mid-term exam
Week 9
Strong coding theorem and error exponents
Week 10
Joint source channel coding
Week 11
Differential entropy and entropy maximization
Week 12
Additive Gaussian noise channel
Week 13
Parallel Gaussian channels and water-filling
Week 14
Rate distortion theory and Blahut-Arimoto algorithm
Week 15
Applications (machine learning, etc.)
Week 16
Final exam

9. Course Operation

이론강의

10. How to Teach & Remark

정보이론의 중요한 개념들이 통신, 신호처리, 기계학습, 인공지능 분야에 어떻게 적용되는지에 대해 강의할 예정입니다.

11. Supports for Students with a Disability

- Taking Course: interpreting services (for hearing impairment), Mobility and preferential seating assistances (for developmental disability), Note taking(for all kinds of disabilities) and etc.

- Taking Exam: Extended exam period (for all kinds of disabilities, if needed), Magnified exam papers (for sight disability), and etc.

- Please contact Center for Students with Disabilities (279-2434) for additional assistance

Copy The course syllabus
No. Semester Name Course No. Section Course Name