Deep Learning Natural Language Processing (DNLP) -
This course will cover a cutting-edge research knowledge in deep learning natural language processing. Through lectures, students will learn the deep neural network models for various NLP problems such as word embedding/contextual word embedding, text classification, syntactic parsing, recurrent language modeling, machine translation, question answering, natural language/code generation, dialog systems, multi-task deep learning models, LLM models, multi-modal foundation models, etc

no required pre-requisites

midterm 35%
final 35%
homework 30%

Dan Jurafsky and James H. Martin. Speech and Language Processing (3rd ed.)
Jacob Eisenstein. Natural Language Processing
Delip Rao and Brian McMahan. Natural Language Processing with PyTorch
Lewis Tunstall, Leandro von Werra, and Thomas Wolf. Natural Language Processing with Transformers
Yoav Goldberg. A Primer on Neural Network Models for Natural Language Processing
Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep Learning
Michael A. Nielsen. Neural Networks and Deep Learning
Eugene Charniak. Introduction to Deep Learning

The course design comes from Stanford NLP with deep learning course with some modification:
the course slides can be downloaded at https://nlp.postech.ac.kr/courses/deep-learning-natural-language-processing

DNLP overview
Word embedding
Text classification/BP
Neural dependency parsing
Neural language modeling (RNN)
Neural machine translation (seq-to-seq)
Some NLP Project Data
Question answering
ConvNet NLP
Sub-word modeling
Contextual word embedding (BERT/Transformer)
LLM Prompting RLHF
NLG (dialog, summarization)
code generation
Co-reference resolution
knowledge in LM
Multi-task DNLP
Multimodal NLP
Tree RNN
Future deep learning NLP

instructor: Gary Geunbae Lee, Eng 2-211, gblee@postech.ac.kr, 279-2254
instruction language: English
2 homeworks: solve deep learning NLP application problems including python programming (written part + coding part)