Deep Learning Foundations (AI3DL001)

Theoretical and probabilistic foundations of deep learning, based on Bishop and Bishop’s Deep Learning: Foundations and Concepts (2 credits lecture, 1 credit lab). The course emphasizes the mathematical principles, probabilistic models, and emerging architectures underlying modern AI systems, and bridges Machine Learning (Semester 2) with applied NLP and Computer Vision (Semester 4).

Sessions

• Sessions 1-3: Introduction to Deep Learning — the deep learning revolution, probabilistic foundations, and standard distributions
• Sessions 4-5: Single-layer Networks — regression and classification
• Sessions 6-8: Deep Neural Networks — architecture, optimization and backpropagation, regularization techniques
• Sessions 9-11: Advanced Architectures — convolutional neural networks, transformers and attention, graph neural networks
• Sessions 12-13: Generative Models and Future Directions — latent variable models, diffusion models, emerging trends

Labs

Hands-on sessions at weeks 4, 5, 6, 9, 11, and 12: single-layer networks, MLP playground, CNN filters and feature maps, GNN node classification, and generative models with autoencoders.

Assessment

Resources

• Textbook: Bishop and Bishop (2023). Deep Learning: Foundations and Concepts
• Tools: PyTorch, TensorFlow Probability, Hugging Face Transformers, Google Colab
• Prerequisites: Machine Learning, linear algebra, basic statistics, Python

Full session materials, lab guides, and solutions are shared with enrolled students only.