CS 769 NLP | Schedule

Date	Lecture	Readings	Logistics
Module 1: Introduction to Natural Language Processing
1/26	Lecture #1 : Overview of NLP [ slides ]	SLP3 ch2
1/31	Lecture #2 : Text Classification [ slides \| video ]	SLP3 ch4 E ch2-4 Dani et al. 2017
2/2	Lecture #3 : Language Model [ slides ]	SLP3 ch 3 Chen & Goodman	HW 1 out (2/4)
2/7	Lecture #4 : Deep Learning Basics [ slides \| video ]	Colab example Gradient Descent (Ruder 2016) Dropout (Srivastava et al. 2014)
Module 2: Modeling and Neural Networks
2/9	Lecture #5 : Word Vectors [ slides \| video ]	SLP3 ch 6 E ch 14
2/14	Lecture #6 : Recurrent Networks [ slides \| video ]	Blog Karpathy et al. 2015 Hochreiter et al. 1997
2/16	Lecture #7 : Conditioned Generation [ slides \| video ]	Sutskever et al. 2014 Graves 2012 GENIE WMT	HW 1 due (11:59pm 2/20) HW 2 out (2/20)
2/21	Lecture #8 : Attention and Transformer [ slides \| video ]	Bahdanau et. al 2014 Vaswani et. al 2017
Module 3: Syntatic and Semantic Analysis
2/23	Lecture #9 : Morphology & Sequence Labeling 1 [ slides \| video ]	BPE, Sennrich et al. 2016 SLP3 Collins 2011
2/28	Lecture #10 : Sequence Labeling 2 [ slides ]	Zhu's notes CRF
3/2	Lecture #11 : Syntatic Parsing 1 [ slides \| video ]	SLP3 ch12 SLP3 ch13 Kitaev et. al 2018
3/9	Lecture #12 : Syntatic Parsing 2 [ slides \| video ]	SLP3 ch14 Dozat et al. 2017 Kiperwasser et al. 2016 UD Chen et al. 2014	HW 2 & Initial Interest Survey due (11:59pm 3/11) HW 3 out (3/11)
3/21	Lecture #13 : Semantic Parsing [ slides \| video ]	SLP3 ch15 SLP2 ch18 E ch12 Yale Text-to-SQL Challenge Chen et al. 2021 (Codex) Yin et al. 2017
3/14	No class (Spring break)
3/16	No class (Spring break)
Module 4: Representation Learning and Algorithms
3/7	Lecture #14 : Pre-training and Self-supervised Learning [ slides \| video ]	Devlin et al. 2019 Yang et al. 2019 Survey Lewis et al. 2020
3/23	Lecture #15 : Multi-task and Multi-lingual Learning [ slides \| video ]	Understanding Dataset shift Caruana 1997 Neubig and Hu, 2018 Chu et al. 2017 Johnson et al. 2017 Conneau et al. 2020 Ganin et al. 2015
3/28	Lecture #16 : Prompting and Few-shot Learning [ slides \| video ]	Liu et al. 2021
3/30	Lecture #17 : Modeling Long Sequences [ slides \| video ]	Dai et al. 2019 Child et al. 2019 Kitaev et al. 2020 Wang et al. 2021 Xiong et al. 2021 SLP3 ch21 SLP3 ch22
4/4	Lecture #18 : Structured Learning Algorithms [ slides \| video ]	Karpathy 2016 Goldberg Book ch19-19.3 Williams 1992 Blum and Mitchell 1998 Collins 2002 Taskar et al. 2005 Norouzi et al. 2016
4/6	Lecture #19 : Latent Variable Models [ slides \| video ]	Tutorial on VAE Kingma and Welling et al. 2014 Bowman et al. 2016 Miao and Blunsom 2016 Chen et al. 2017 Jiang et al. 2017 Hu et al. 2017 Yin et al. 2018 Kim et al. 2019	HW 3 due (11:59pm 4/8)
Module 5: NLP Applications
4/11	Lecture #20 : Text-based Question Answering [ slides \| video ]	SLP3 ch23 Weston et al. 2015 Rajpurkar et al. 2016 Seo et al. 2017 Chen et al. 2017 Jia and Liang 2017 Kwiatkowski et al. 2019
4/13	Lecture #21 : Information Extraction and Knowledge-based QA [ slides \| video ]	SLP3 ch17
4/18	Lecture #22 : Machine Translation [ slides \| video ]	IBM Model Moses Johnson et al. 2016 Koehn et al. 2017 Arivazhagan et al. 2019 SLP3 ch10
4/20	Lecture #23 : Dialog System [ slides \| video ]	SLP3 ch24
4/25	Lecture #24 : Computational Social Science, Bias and Fairness [ slides \| video ]	Zhao et al. 2017 Blodgett et al. 2020
4/27	Lecture #25 : Vison-Language Learning [ slides \| video ]	Baltrusaitis et al. 2017 Radford et al. 2021 Ramesh et al. 2021
5/2	Project Presentation
5/4	Project Presentation
5/6	Project report due 11:59pm 5/10 (extended)