CS 769 NLP | Schedule

Date	Lecture	Readings	Logistics
Module 1: Introduction to Natural Language Processing
9/4	Lecture #1 : Overview of NLP [ slides \| video ]	SLP3 ch2
9/9	Lecture #2 : Text Classification [ slides \| video ]	SLP3 ch4 E ch2-4 Dani et al. 2017
9/11	Lecture #3 : Deep Learning Basics [ slides \| video ]	Colab example Gradient Descent (Ruder 2016) Dropout (Srivastava et al. 2014)
9/16	Lecture #4 : Word Vectors [ slides \| video ]	SLP3 ch 6 E ch 14
Module 2: Modeling and Neural Networks
9/18	Lecture #5 : Language Model [ slides \| video ]	SLP3 ch 3 Chen & Goodman	HW 1 due & Project survey due (11:59pm 9/18)
9/23	Lecture #6 : Recurrent Networks [ slides \| video ]	Blog Karpathy et al. 2015 Hochreiter et al. 1997
9/25	Lecture #7 : Attention and Transformer [ slides \| video ]	Bahdanau et. al 2014 Vaswani et. al 2017
9/30	Lecture #8 : Pre-training and Self-supervised Learning [ slides \| video ]	Devlin et al. 2019 Yang et al. 2019 Survey Lewis et al. 2020
10/2	Lecture #9 : Conditioned Generation [ slides \| video ]	Sutskever et al. 2014 Graves 2012 GENIE WMT
Module 3: Syntatic and Semantic Analysis
10/7	Lecture #10 : Morphology & Sequence Labeling [ slides \| video ]	BPE, Sennrich et al. 2016 SLP3 Collins 2011 Zhu's notes CRF
10/9	Lecture #11 : Syntatic Parsing [ slides \| video 1 ]	SLP3 ch12 SLP3 ch13 SLP3 ch14 Dozat et al. 2017 Kiperwasser et al. 2016 UD Chen et al. 2014 Kitaev et. al 2018	HW 2 due & Team information due (11:59pm 10/9)
Module 4: Advanced Representation Learning and Algorithms
10/14	Lecture #12 : Parameter-efficient Fine-tuning [ slides \| video ]	He et al. 2021 Dettmers et al. 2023 Hu et al. 2021
10/16	Lecture #13 : Multi-task Learning in NLP [ slides \| video ]	Understanding Dataset shift Caruana 1997 Neubig and Hu, 2018 Chu et al. 2017 Johnson et al. 2017 Conneau et al. 2020 Ganin et al. 2015
10/21	Lecture #14 : Prompting and Zero-/Few-shot Learning [ slides \| video ]	Liu et al. 2021
10/23	Lecture #15 : Modeling Long Sequences [ slides \| video ]	Dai et al. 2019 Child et al. 2019 Kitaev et al. 2020 Wang et al. 2021 Xiong et al. 2021 SLP3 ch21 SLP3 ch22
10/28	Lecture #16 : Structured Learning Algorithms [ slides \| video ]	Karpathy 2016 Goldberg Book ch19-19.3 Williams 1992 Blum and Mitchell 1998 Collins 2002 Taskar et al. 2005 Norouzi et al. 2016
10/30	Lecture #17 : Latent Variable Models [ slides \| video ]	Tutorial on VAE Kingma and Welling et al. 2014 Bowman et al. 2016 Miao and Blunsom 2016 Chen et al. 2017 Jiang et al. 2017 Hu et al. 2017 Yin et al. 2018 Kim et al. 2019	HW 3 (Proposal) due (11:59pm 10/30)
11/4	Lecture #18 : Retrieval-Augmented Generation of LLMs (Guest Lecture by Akari) [ slides \| video ]	Akari's slides
Module 5: NLP Applications
11/6	Lecture #19 : Text-based Question Answering [ slides \| video ]	SLP3 ch23 Weston et al. 2015 Rajpurkar et al. 2016 Seo et al. 2017 Chen et al. 2017 Jia and Liang 2017 Kwiatkowski et al. 2019
11/11	Lecture #20 : LLMs and Knowledge-Graph [ slides \| video ]	SLP3 ch17
11/13	Lecture #21 : Vison-Language Learning (Guest Lecture) [ slides \| video ]	Baltrusaitis et al. 2017 Radford et al. 2021 Ramesh et al. 2021	HW 4 due (11:59pm 11/13)
11/18	Lecture #22 : Machine Translation [ slides \| video ]	IBM Model Moses Johnson et al. 2016 Koehn et al. 2017 Arivazhagan et al. 2019 SLP3 ch10
11/20	Lecture #23 : Dialog System [ slides \| video ]	SLP3 ch24
11/25	Lecture #24 : Computational Social Science, Bias and Fairness [ slides \| video ]	Zhao et al. 2017 Blodgett et al. 2020
11/27	Lecture #25 : Code Generation and LLM Programming [ slides \| video ]	Survey. Jiang et al. 2017
12/2	Project Presentation
12/4	Project Presentation
12/6	Project report due 11:59pm 12/6