COMP 5801H/COMP 4900: Generative AI and Large Language Models (Winter 2026)

Course Information

1. Course Instructor: Sriram Ganapathi Subramanian
   
2. Course Equivalence: This course is equivalent to CSI5388 IG00 at the University of Ottawa.
   
3. Contact: [email protected]
   
4. Course Website: https://brightspace.carleton.ca
   
5. Lecture Times: 16.05 — 17.25 (Tue, Thurs) in-person at Hall No. 100 in St. Patrick’s building
   
6. Office Hours: 11:00 am — 12:00 noon (Tue) in-person at HP 5360
   
7. Discussion Medium: Students will be encouraged to use Piazza whenever possible (use this link to self enrol: https://piazza.com/carleton.ca/winter2026/comp5801h4900a)
   
8. Teaching Assistants: Adnan Khan ([email protected]) & Hoda Vafaeesefat ([email protected])
   

Course Description

This course provides an exhaustive exploration of Generative Artificial Intelligence, with a focus on foundational mathematical principles, recent research breakthroughs, novel model architectures, and applications cutting across a wide range of domains. A variety of modern deep learning architectures, such as Generative Adversarial Networks (GANs), Variational Autoencoders (VAEs), Transformers, and Diffusion Models, will be covered in the course. The emphasis will be on practical aspects pertaining to the training and deployment of large models: data preparation, setting up training pipelines, evaluation of model performance, and strategies for improving training efficiency.

Prerequisites (for COMP 4900 only): COMP 3105 or COMP 3106; must have fourth year standing in BCS.

Textbooks

There is no required textbook for the course. Reading materials associated with each lecture will be provided.

Useful References

1. [Reference 1] I. Goodfellow, Y. Bengio, and A. Courville, Deep Learning, MIT Press, 2016.
   
2. [Reference 2] J. Eisenstein, Introduction to Natural Language Processing, MIT Press, 2019.
   
3. Research papers from arXiv, NeurIPS, ICML, ACL, and EMNLP (will be provided for each lecture).

Course Format

The course has two in-person classes every week. While the lectures will be recorded, all students are highly encouraged to attend the lectures in person as each lecture will have ample scope for discussions and questions.

For the best educational experience, you should attend the lectures in person. The lecture recordings may have occasional quality issues and will not capture any explanations of the instructor when the instructor chooses to use the chalkboard. The primary purpose of the recordings is for revision of concepts and for use when extenuating circumstances (such as an illness) prevent in-person attendance.

Assessments

There will be 3 assignments (each worth 20% of your final grade) and one final project (worth 40% of your final grade). The assignments and the project will expect critical thinking, curiosity, and creativity from students.

Course Project

Instructions for the course project can be found here.

Inquiries

If you have a question (ex: clarification on readings, discussion about something said during class, questions about assignments), you should post them on Piazza so that your classmates can benefit from the discussion. If the question is about your assessments or situation, you may email the instructor or leave a private message on Piazza.

Please add COMP 5801 or COMP 4900 in your email subject line to ensure they are prioritized. Do not post code or assignment answers in the open or in course discussions. Questions about assessments will not be answered within 24 hours of the due date.

You may also schedule an appointment by emailing the instructor or assigned TA (please do not do this unless absolutely necessary).

Course Schedule

All due dates are 11.59 pm ET.

Lecture	Date	Topic	Readings
1	Jan 6	Introduction to Generative AI (slides) (lecture video, passcode:9*Fn+P^5)	None
2	Jan 8	Recurrent Neural Networks (RNNs) – I (slides) (lecture video, passcode:!ffSj1?B)	Chapter 10 of Reference 1, Chapter 6 of Reference 2
3	Jan 13	Recurrent Neural Networks (RNNs) – II (slides) (lecture video, passcode:U9UzK6?%)	“Bidirectional recurrent Neural Networks” by Schuster and Paliwal (1997) “Recurrent Stacking of Layers for Compact Neural Machine Translation Models” by Dabre and Fujita (2019)
4	Jan 15	Attention Mechanism and Transformers – I (slides) (lecture video, passcode:A9jK=a58)	“Attention Is All You Need.” by Vaswani, A., et al. (2017) “Neural Machine Translation by Jointly Learning to Align and Translate.” by Bahdanau, D., Cho, K., & Bengio, Y. (2014) “On the difficulty of training Recurrent Neural Networks.” by Pascanu et al. (2013)
Jan 16	Assignment 1 released	Question
5	Jan 20	Attention Mechanism and Transformers – I (Slides: Previous lecture continued.) (lecture video, passcode: tF0S!Lge)	Same as references in the previous lecture.
6	Jan 22	Attention Mechanism and Transformers – II (Slides) (lecture video, passcode: J61A$$GT)	Same as references in the previous lecture.
7	Jan 27	BERT and GPT – I (Slides) (lecture video, passcode: 7t+qc!VX)	“BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding” by Devlin et al. (2018) “Improving Language Understanding by Generative Pre-Training” by Radford et al. (2018) “Language Models are Few-Shot Learners” by Brown et al. (2020)
8	Jan 29	BERT and GPT – II (Previous lecture continued) (lecture video, passcode: #aS&s8mM)	Same as references in the previous lecture.
Jan 29	Assignment 1 due
Jan 30	Assignment 2 released	Question
9	Feb 3	Variational Autoencoders (VAEs) – I (Slides) (lecture video, passcode: k00!$tQn)	“Reducing the dimensionality of data with neural networks” by Hinton & Salakhutdinov (2006) “An Introduction to Variational Autoencoders” by Kingma & Welling (2019)
10	Feb 5	Variational Autoencoders (VAEs) – I (Previous Lecture Continued) (lecture video, passcode: ?rq@&5tS)	Same as the previous lecture.
11	Feb 10	Variational Autoencoders (VAEs) II (Slides) (lecture video, passcode:9I%2?tVA)	Same as the previous lecture.
12	Feb 12	Variational Autoencoders (VAEs) II (Previous lecture continued) (lecture video, passcode:=%80N.yv)	Same as the previous lecture.
Feb 12	Assignment 2 due
Feb 13	Assignment 3 released	Question
13	Feb 24	Generative Adversarial Networks (GANs) – I (Slides) (lecture video, passcode: qQybsb2+)	“Generative Adversarial Networks” by Goodfellow et al. (2014) “Improved Techniques for Training GANs” by Salimans et al. (2016) “GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium” by Heusel et al. (2018)
14	Feb 26	Generative Adversarial Networks (GANs) – I (Previous lecture continued) (lecture video)	Same as the previous lecture.
Feb 26	Assignment 3 due
Mar 1	(Optional for most students, see Project Instructions) Project proposal due
15	Mar 3	Generative Adversarial Networks (GANs) – II (Slides) (lecture video, passcode: g4p&3Abv)	“Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks” by Radford et al. (2015) “Conditional Generative Adversarial Nets” by Mirza & Ozindero (2014) “Wasserstein GAN” by Arjovsky et al. (2017) “Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks” by Zhu et al. (2020) “A Style-Based Generator Architecture for Generative Adversarial Networks” by Karras et al. (2018)
16	Mar 5	Generative Adversarial Networks (GANs) – II (Previous lecture continued) (lecture video, passcode: fH%Y=M6G)	Same as the previous lecture.
18	Mar 10	Diffusion Models – I (Slides) (lecture video, passcode: 3vKWroH%)	“Deep Unsupervised Learning using Nonequilibrium Thermodynamics” by Dickstein et al. (2015) “Denoising Diffusion Probabilistic Models” by Ho et al. (2020) “Denoising Diffusion Implicit Models” by Song et al. (2021) “Score-Based Generative Modeling through Stochastic Differential Equations” by Song et al. (2021)
19	Mar 12	Diffusion Models – I (Previous lecture continued) (lecture video, passcode: L*T$C7TE)	Same as the previous lecture.
20	Mar 17	Diffusion Models – II (Slides) (lecture video, passcode: R#c#vC9M)	“High-Resolution Image Synthesis with Latent Diffusion Models” by Rombach et al. (2021) “Conditional Image Synthesis with Diffusion Models: A Survey” by Zhan et al. (2024) “Classifier-Free Diffusion Guidance” by Ho and Salimans (2022) “GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models” by Nichol et al. (2021) “High-Resolution Image Synthesis with Latent Diffusion Models” by Rombach et al. (2022)
21	Mar 19	Diffusion Models – II (Previous lecture continued) (lecture video, passcode: U?sjj55Y)	Same as the previous lecture.
22	Mar 24	Reinforcement Learning from Human Feedback (Slides) (Lecture Video, passcode: y^w#St42)	“Deep reinforcement learning from human preferences” by Christiano et al. (2017) “Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback” by Bai et al. (2022) “Training language models to follow instructions with human feedback” by Ouyang et al. (2022) “Direct Preference Optimization: Your Language Model is Secretly a Reward Model” by Rafailov et al. (2023)
23	Mar 26	Reinforcement Learning from Human Feedback (Previous lecture continued) (Lecture Video, passcode: n6#lmu6m)	Same as the previous lecture.
24	Mar 31	Reinforcement Learning from Human Feedback (Previous lecture continued) (Lecture Video, passcode: ?t2lt2%b)	Same as the previous lecture.
25	Apr 2	Ethics and Governance: Bias, fairness, and regulatory frameworks (Slides) (Lecture Video, passcode: 4$z6=rj#)	“Inherent Trade-Offs in the Fair Determination of Risk Scores” by Kleinberg et al. (2016) “On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?” by Bender et al. (2021) “Model Cards for Model Reporting” by Mitchell et al. 2018 “Model Cards for Model Reporting” by Mitchell et al. (2018)
26	Apr 7	Ethics and Governance: Bias, fairness, and regulatory frameworks (Previous lecture continued) (Lecture Video, passcode: jSQ0&Mx@)	Same as the previous lecture.
Apr 8	Project report due

Late Policy

The deadlines for assignments and projects will be strictly enforced, and no late submissions of assignments and projects will be accepted by default. If you need an exception for this, email the instructor at least 24 hours before the deadline. Note that more than one exception will not be made for the same student under any circumstances.

This policy accommodates unexpected circumstances such as technical and personal issues; therefore, no additional extensions will be granted (excepting accommodations provided by university policy).

Generative AI Tools – Course Policy

You are free to use any Generative AI Tool for your assignments or projects, but you bear responsibility for all your submissions. Since Generative AI tools may plagiarize from other sources on the web, you should be extremely careful while using them. They are also prone to producing incorrect information with exaggerated confidence. All prior work should be rigorously cited, and any plagiarism will be considered an academic offence.