Course Information
- Course Instructor: Sriram Ganapathi Subramanian
- Course Equivalence: This course is equivalent to CSI5388 IG00 at the University of Ottawa.
- Contact: [email protected]
- Course Website: https://brightspace.carleton.ca
- Lecture Times: 16.05 — 17.25 (Tue, Thurs) in-person at Hall No. 100 in St. Patrick’s building
- Office Hours: 11:00 am — 12:00 noon (Tue) in-person at HP 5360
- Discussion Medium: Students will be encouraged to use Piazza whenever possible (use this link to self enrol: https://piazza.com/carleton.ca/winter2026/comp5801h4900a)
- Teaching Assistants: Adnan Khan ([email protected]) & Hoda Vafaeesefat ([email protected])
Course Description
This course provides an exhaustive exploration of Generative Artificial Intelligence, with a focus on foundational mathematical principles, recent research breakthroughs, novel model architectures, and applications cutting across a wide range of domains. A variety of modern deep learning architectures, such as Generative Adversarial Networks (GANs), Variational Autoencoders (VAEs), Transformers, and Diffusion Models, will be covered in the course. The emphasis will be on practical aspects pertaining to the training and deployment of large models: data preparation, setting up training pipelines, evaluation of model performance, and strategies for improving training efficiency.
Prerequisites (for COMP 4900 only): COMP 3105 or COMP 3106; must have fourth year standing in BCS.
Textbooks
There is no required textbook for the course. Reading materials associated with each lecture will be provided.
Useful References
- [Reference 1] I. Goodfellow, Y. Bengio, and A. Courville, Deep Learning, MIT Press, 2016.
- [Reference 2] J. Eisenstein, Introduction to Natural Language Processing, MIT Press, 2019.
- Research papers from arXiv, NeurIPS, ICML, ACL, and EMNLP (will be provided for each lecture)
Course Format
The course has two in-person classes every week. While the lectures will be recorded, all students are highly encouraged to attend the lecture as each lecture will have ample scope for discussions and questions.
Assessments
There will be 3 assignments (each worth 20% of your final grade) and one final project (worth 40% of your final grade). The assignments and the project will expect critical thinking, curiosity, and creativity from students.
Course Project
Instructions for the course project can be found here.
Inquiries
If you have a question (ex: clarification on readings, discussion about something said during class, questions about assignments), you should post them on Piazza so that your classmates can benefit from the discussion. If the question is about your assessments or situation, you may email the instructor or leave a private message on Piazza.
Please add COMP 5801 or COMP 4900 in your email subject line to ensure they are prioritized. Do not post code or assignment answers in the open or in course discussions. Questions about assessments will not be answered within 24 hours of the due date.
You may also schedule an appointment by emailing the instructor or assigned TA (please do not do this unless absolutely necessary).
Course Schedule
All due dates are 11.59 pm ET.
| Lecture | Date | Topic | Readings |
| 1 | Jan 6 | Introduction to Generative AI (slides) (lecture video, passcode:9*Fn+P^5) | None |
| 2 | Jan 8 | Recurrent Neural Networks (RNNs) – I (slides) (lecture video, passcode:!ffSj1?B) | Chapter 10 of Reference 1, Chapter 6 of Reference 2 |
| 3 | Jan 13 | Recurrent Neural Networks (RNNs) – II (slides) (lecture video, passcode:U9UzK6?%) | “Bidirectional recurrent Neural Networks” by Schuster and Paliwal (1997) “Recurrent Stacking of Layers for Compact Neural Machine Translation Models” by Dabre and Fujita (2019) |
| 4 | Jan 15 | Attention Mechanism and Transformers – I (slides) (lecture video, passcode:A9jK=a58) | “Attention Is All You Need.” by Vaswani, A., et al. (2017) “Neural Machine Translation by Jointly Learning to Align and Translate.” by Bahdanau, D., Cho, K., & Bengio, Y. (2014) “On the difficulty of training Recurrent Neural Networks.” by Pascanu et al. (2013) |
| Jan 16 | Assignment 1 released | Question | |
| 5 | Jan 20 | Attention Mechanism and Transformers – I (Slides: Previous lecture continued.) (lecture video, passcode: tF0S!Lge) | Same as references in the previous lecture. |
| 6 | Jan 22 | Attention Mechanism and Transformers – II (Slides) (lecture video, passcode: J61A$$GT) | Same as references in the previous lecture. |
| 7 | Jan 27 | BERT and GPT – I (Slides) (lecture video, passcode: 7t+qc!VX) | “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding” by Devlin et al. (2018) “Improving Language Understanding by Generative Pre-Training” by Radford et al. (2018) “Language Models are Few-Shot Learners” by Brown et al. (2020) |
| 8 | Jan 29 | BERT and GPT – II (Previous lecture continued) (lecture video, passcode: #aS&s8mM) | Same as references in the previous lecture. |
| Jan 29 | Assignment 1 due | ||
| Jan 30 | Assignment 2 released | Question | |
| 9 | Feb 3 | Variational Autoencoders (VAEs) – I (Slides) (lecture video, passcode: k00!$tQn) | “Reducing the dimensionality of data with neural networks” by Hinton & Salakhutdinov (2006) “An Introduction to Variational Autoencoders” by Kingma & Welling (2019) |
| 10 | Feb 5 | Variational Autoencoders (VAEs) – I (Previous Lecture Continued) (lecture video, passcode: ?rq@&5tS) | Same as the previous lecture. |
| 11 | Feb 10 | Variational Autoencoders (VAEs) II (Slides) (lecture video, passcode:9I%2?tVA) | Same as the previous lecture. |
| 12 | Feb 12 | Variational Autoencoders (VAEs) II (Previous lecture continued) (lecture video, passcode:=%80N.yv) | Same as the previous lecture. |
| Feb 12 | Assignment 2 due | ||
| Feb 13 | Assignment 3 released | Question | |
| 13 | Feb 24 | Generative Adversarial Networks (GANs) – I (Slides) (lecture video, passcode: qQybsb2+) | “Generative Adversarial Networks” by Goodfellow et al. (2014) “Improved Techniques for Training GANs” by Salimans et al. (2016) “GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium” by Heusel et al. (2018) |
| 14 | Feb 26 | Generative Adversarial Networks (GANs) – I (Previous lecture continued) (lecture video) | Same as the previous lecture. |
| Feb 26 | Assignment 3 due | ||
| Mar 1 | (Optional for most students, see Project Instructions) Project proposal due | ||
| 15 | Mar 3 | Generative Adversarial Networks (GANs) – II (Slides) (lecture video, passcode: g4p&3Abv) | “Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks” by Radford et al. (2015) “Conditional Generative Adversarial Nets” by Mirza & Ozindero (2014) “Wasserstein GAN” by Arjovsky et al. (2017) “Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks” by Zhu et al. (2020) “A Style-Based Generator Architecture for Generative Adversarial Networks” by Karras et al. (2018) |
| 16 | Mar 5 | Generative Adversarial Networks (GANs) – II (Previous lecture continued) (lecture video, passcode: fH%Y=M6G) | Same as the previous lecture. |
| 18 | Mar 12 | Diffusion Models – I (Slides) (lecture video, passcode: 3vKWroH%) | “Deep Unsupervised Learning using Nonequilibrium Thermodynamics” by Dickstein et al. (2015) “Denoising Diffusion Probabilistic Models” by Ho et al. (2020) “Denoising Diffusion Implicit Models” by Song et al. (2021) “Score-Based Generative Modeling through Stochastic Differential Equations” by Song et al. (2021) |
| 19 | Mar 17 | Diffusion Models – I | |
| 20 | Mar 19 | Diffusion Models – II | |
| 21 | Mar 24 | Diffusion Models – II | |
| 22 | Mar 26 | Reinforcement Learning from Human Feedback – I | |
| 23 | Mar 31 | Reinforcement Learning from Human Feedback – II | |
| 24 | Apr 2 | Emerging Research | |
| 25 | Apr 7 | Ethics and Governance: Bias, fairness, and regulatory frameworks | |
| Apr 8 | Project report due |
Late Policy
The deadlines for assignments and projects will be strictly enforced, and no late submissions of assignments and projects will be accepted by default. If you need an exception for this, email the instructor at least 24 hours before the deadline. Note that more than one exception will not be made for the same student under any circumstances.
This policy accommodates unexpected circumstances such as technical and personal issues; therefore, no additional extensions will be granted (excepting accommodations provided by university policy).
Generative AI Tools – Course Policy
You are free to use any Generative AI Tool for your assignments or projects, but you bear responsibility for all your submissions. Since Generative AI tools may plagiarize from other sources on the web, you should be extremely careful while using them. They are also prone to producing incorrect information with exaggerated confidence. All prior work should be rigorously cited, and any plagiarism will be considered an academic offence.