AIE 604

AIE 604 : Deep Learning Applications

M.Sc in Artificial Intelligence Systems

(by Prof. Mohab Mangoud)

Machine Learning vs Deep Learning - PieceX source code marketplace

  1. The purpose of the Deep Learning applications course is to present to learners more details about one of the latest engineering applications of AI and deep learning in daily basis. This course will rather lead to profitable roles in IT, engineering computer vision, healthcare, FinTech, e-commerce, and other industries. Deep Learning is one of the most highly sought-after skills in AI. In this course, students will also learn the foundations of Deep Learning, understand how to build advanced ANN, successful machine learning projects for engineers. Students will learn Convolutional networks, RNNs, LSTM, Adam, Dropout, BatchNorm, Xavier/He initialization and Generative AI.

Syllabus: More details, about DEEP LEARNING:

  • Variability models, deformation model, stochastic model.
  • Supervised Learning: classification.
  • CNN-Convolutional Neural Networks
  • Properties of CNN representations: invertibility, stability,
  • Localization, Regression, and Embeddings, DrLim, and inverse
  • Object Detection techniques 
  • Sequential NN, RNN / LSTM
  • Deep Generative modeling (AE, VAE, GANs)
  • Examples and Applications with Coding.

Course Grading

  • Survey paper / presentation                     (20%)
  • HWs:                10% +10% +10%+10%=    (40%)                                               
  • Final Project                                                (40%)
 

Course outlines

Lecture 1(w1)  [Feb 20]:  Course overview and Objectives  Introduction to Deep learning + Lab1: Python + TF + Keras + Matlab : (chapter 1) 

Lecture 2(w2) : [Feb 27]: Neural Networks I :  Basics/ Perceptron / MLP / different types of Activation (Chapter 2)

Lecture 3(w3) : [Mar 5 ]  Neural Networks II / Modeling / Training/ Loss function / Back-propagation / improve training speed and accuracy / Stochastic gradient descent /Batches / Xavier/He initialization 

 Final project proposal (*** 5%)

( M : March 11= 1st Ramadan 1443) Start of Ramadan working hours

Lecture 4(w4) : [March 12]    

– HW1 (10%): Build a basic neural network models (6 examples) / Lab1

Lecture 5(w5)  :  [March 19 ] Convolutional Neural Networks (CNN I) CNN model / Compare to fully connected NN / Build a CNN by choosing the grid size, padding, stride, depth, and pooling.  (chapter 3)

Lecture 6(w6) : [March 26 ] Convolutional Neural Networks  II: Pre- trained Models/ Transfer learning/ Data Augmentation /  AlexNet, VGG, GoogLeNet, ResNet, etc / Deep learning literature talks about many image classification topologies like AlexNet, VGG-16 and VGG-19, Inception, and ResNet. This week, learn how these topologies are designed and the usage scenarios for each.  (chapter 4) 

– HW2 (10%) : Build a CNN model / application – / Lab2   

________________________

Sun 31 March – 4 April : Mid-semester break for students

+ Eid Alfitr Holiday (10 April -12 April) – end of Ramadan working hours
 

_____________________

Lecture 7(w7) :  [April 16 ]  Object Detection – Two stage detectors  – R-CNN, SDD, YOLO, one stage detector. 
 
Lecture 8(w8) :  [April 23]  RNN : Recurrent neural networks (RNN) and their application to natural language processing (NLP). LSTM Long short term memory (LSTM).

 

Lecture 9 (w9) : [April 30] Students seminar# 1:  Survey paper
# 1 (5 papers presentations) (20%)

Lecture 10(w10) : [May 7] Deep Generative Modeling 1 (AE, VAE)

Lecture 11(w11) :  [May 14] Deep Generative Modeling 2 (GANs)

  • HW3 (10%) : OD/ RNN/ GANs  – Lab 3 
  • Lecture 14(w12) : [May 21]Deep reinforcement learning I (DRL)
  • HW4 (10%) : DRL  – Lab 5 

*Lecture 15(w13) : [May 28] Deep reinforcement learning II (DRL)

  • HW4 (10%) : DRL  – Lab 5 

Last day of Classes      Thursday May 28

[ June] Final Project presentation + report / final paper submission (40%)

 

____________________________________________________________________

Previous Course Projects (Examples)

Class (2021) 

Class (2022)

Class (2023) 

______________________________________________________________

Tutorials

https://cs231n.github.io/python-numpy-tutorial/

https://github.com/cs231n/gcloud

https://www.tensorflow.org/tutorials

https://deeplearning.cs.cmu.edu/F20/document/recitation/IDL_Recitation1.pdf

Textbooks

This is a selection of optional textbooks you may find useful

Deep Learning
Dive Into Deep LearningBy Aston Zhang, Zachary C. Lipton, Mu Li, and Alexander J. SmolaPDF, 2020
Deep Learning
Deep LearningBy Ian Goodfellow, Yoshua Bengio, Aaron CourvilleOnline book, 2017
 

Software tools

  1. keras.io
  2. tensorflow.org
  3. mathworks.com/products/deep-learning.html

The Neural Network Zoo