CS 769 NLP | Schedule

Date	Lecture	Readings	Logistics
Module 1: Introduction to Natural Language Processing
1/25	Lecture #1 : Overview of NLP [ slides \| video ]	SLP3 ch2	HW 1, 2, 3 and 5 out (1/15)
1/30	Lecture #2 : Text Classification [ slides \| video ]	SLP3 ch4 E ch2-4 Dani et al. 2017
2/1	Lecture #3 : Deep Learning Basics [ slides \| video ]	Colab example Gradient Descent (Ruder 2016) Dropout (Srivastava et al. 2014)
2/6	Lecture #4 : Word Vectors [ slides \| video ]	SLP3 ch 6 E ch 14
Module 2: Modeling and Neural Networks
2/8	Lecture #5 : Language Model [ slides \| video ]	SLP3 ch 3 Chen & Goodman	HW 1 due (11:59pm 2/8)
2/13	Lecture #6 : Recurrent Networks [ slides \| video ]	Blog Karpathy et al. 2015 Hochreiter et al. 1997
2/15	Lecture #7 : Attention and Transformer [ slides \| video ]	Bahdanau et. al 2014 Vaswani et. al 2017
2/20	Lecture #8 : Pre-training and Self-supervised Learning [ slides \| video ]	Devlin et al. 2019 Yang et al. 2019 Survey Lewis et al. 2020
2/22	Lecture #9 : Conditioned Generation [ slides \| video ]	Sutskever et al. 2014 Graves 2012 GENIE WMT
Module 3: Syntatic and Semantic Analysis
2/27	Lecture #10 : Morphology & Sequence Labeling 1 [ slides \| video ]	BPE, Sennrich et al. 2016 SLP3 Collins 2011
3/1	Lecture #11 : Sequence Labeling 2 [ slides \| video ]	Zhu's notes CRF	HW 2 due (11:59pm 3/1)
3/7	Lecture #12 : Syntatic Parsing 1 [ slides \| video ]	SLP3 ch12 SLP3 ch13 Kitaev et. al 2018
3/9	Lecture #13 : Syntatic Parsing 2 [ slides \| video ]	SLP3 ch14 Dozat et al. 2017 Kiperwasser et al. 2016 UD Chen et al. 2014	Project Interest Survey and Team Information due (11:59pm 3/9)
3/13	No class (Spring break)
3/15	No class (Spring break)
3/20	Lecture #14 : Semantic Parsing [ slides \| video ]	SLP3 ch15 SLP2 ch18 E ch12 Yale Text-to-SQL Challenge Chen et al. 2021 (Codex) Yin et al. 2017
Module 4: Advanced Representation Learning and Algorithms
3/22	Lecture #15 : Multi-task Learning in NLP [ slides \| video ]	Understanding Dataset shift Caruana 1997 Neubig and Hu, 2018 Chu et al. 2017 Johnson et al. 2017 Conneau et al. 2020 Ganin et al. 2015	HW 3 (Proposal) due (11:59pm 3/22)
3/27	Lecture #16 : Prompting and Zero-/Few-shot Learning [ slides \| video ]	Liu et al. 2021
3/29	Lecture #17 : Modeling Long Sequences [ slides \| video ]	Dai et al. 2019 Child et al. 2019 Kitaev et al. 2020 Wang et al. 2021 Xiong et al. 2021 SLP3 ch21 SLP3 ch22
4/3	Lecture #18 : Structured Learning Algorithms [ slides \| video ]	Karpathy 2016 Goldberg Book ch19-19.3 Williams 1992 Blum and Mitchell 1998 Collins 2002 Taskar et al. 2005 Norouzi et al. 2016
4/5	Lecture #19 : Latent Variable Models [ slides \| video ]	Tutorial on VAE Kingma and Welling et al. 2014 Bowman et al. 2016 Miao and Blunsom 2016 Chen et al. 2017 Jiang et al. 2017 Hu et al. 2017 Yin et al. 2018 Kim et al. 2019
Module 5: NLP Applications
4/10	Lecture #20 : Text-based Question Answering [ slides \| video ]	SLP3 ch23 Weston et al. 2015 Rajpurkar et al. 2016 Seo et al. 2017 Chen et al. 2017 Jia and Liang 2017 Kwiatkowski et al. 2019
4/12	Lecture #21 : Information Extraction and Knowledge-based QA [ slides \| video ]	SLP3 ch17
4/17	Lecture #22 : Machine Translation [ slides \| video ]	IBM Model Moses Johnson et al. 2016 Koehn et al. 2017 Arivazhagan et al. 2019 SLP3 ch10
4/19	Lecture #23 : Dialog System [ slides \| video ]	SLP3 ch24	HW 4 due (11:59pm 4/21)
4/24	Lecture #24 : Computational Social Science, Bias and Fairness [ slides \| video ]	Zhao et al. 2017 Blodgett et al. 2020
4/26	Lecture #25 : Vison-Language Learning [ slides \| video ]	Baltrusaitis et al. 2017 Radford et al. 2021 Ramesh et al. 2021
5/1	Project Presentation
5/3	Project Presentation
5/9	Project report due 11:59pm 5/9