Deep Learning and IoT Data Analytics

This is a course on the principles of representation learning in general and deep learning in particular. Deep learning has recently been responsible for a large number of impressive empirical gains across a wide array of applications including most dramatically in object recognition and detection in images, natural language processing and speech recognition. On the other hand, Internet of thing (IoT) and Smart City deployments are generating large amounts of time-series sensor data in need of analysis. Developing deep learning methods to these domains is an important and very timely research topic.

In this course we will explore both the fundamentals advances in the area of deep learning and the recent applications to the field of IoT and in general communications. Our focus will be on recent applications of deep learning to perform data analytics on Internet of Things (IoT) communications, including neural networks, auto-encoders, convolutional neural networks and recurrent networks. We will also consider well-known probabilistic graphical models, including undirected models and directed models that have recently shown promise (e.g. Boltzmann machines, Deep Believe Nets). 

Course announcement

I shall be able to answer questions on your homework assignments every Wednesday in my office A4.19 between 5 p.m. to 6 p.m., but you can also ask questions via e-mail to pablo.piantanida@centralesupelec.fr

Summary of lectures

1. 1.Introduction to Neural Networks: Perceptrons, capacity of a single neuron, Linear and logistic regression, Back-propagation and stochastic gradient optimization.
   

2. 2.Advance Neural Networks: Auto-encoders and variants, hyper-parameters and training tricks for neural networks, difficulty training deep networks and regularization (norms, Dropout, denoising auto-encoders,…), unsupervised learning of representations and pre-training.
   

3. 3.Variations on auto-encoders and probabilistic Graphical Models: Convolutional neural networks, recurrent networks, Variational auto-encoders and Generative Adversarial Nets, Review of directed models (HMMs and mixtures), Undirected models: Markov random fields and Boltzmann machines, Inference, sampling and learning algorithms for such models, Deep belief networks, deep Boltzmann machines.
   

4. 4.Modern architectural variations for communications and IoT data analytics : A quick view of TensorFlow, Typical classification tasks: classifying MNIST digits using logistic regression, Keras: a python deep learning library, Deep learning for solving communication problems.
   

Readings and textbooks

Textbooks and references:

1. Deep Learning, by Ian Goodfellow, Yoshua Bengio and Aaron Courville, Adaptive Computation and Machine Learning series, MIT Press, November 2016, ISBN-10: 0262035618.
   

2. Machine Learning: A Probabilistic Perspective, by Kevin P. Murphy, Adaptive Computation and Machine Learning series, MIT Press, ISBN-10: 0262018020.
   

3. Machine Learning, by Tom M. Mitchell, Mcgraw-Hill Series in Computer Science, McGraw-Hill Education, ISBN-10: 0070428077.
   

4. Deep Learning - Methods and Applications by Li Deng and Dong Yu http://research.microsoft.com/pubs/219984/BOOK2014.pdf
   

5. Reading lists for new LISA students by University of Montreal
   

6. Theano: http://deeplearning.net/software/theano/ in python, works on CPU and GPU and several wrappers
   

7. http://lasagne.readthedocs.org/
   

8. http://keras.io/
   

9. https://www.cs.cmu.edu/~ymiao/pdnntk.html
   

10. http://deeplearning.net/software/pylearn2/
    

11. http://blocks.readthedocs.org/
    

12. Torch7: http://torch.ch/ Lua interface to C/CUDA
    

13. TensorFlow https://www.tensorflow.org API in C++ and Python
    

Lecture notes

Lecture notes will be distributed during the course and available via e-mail to pablo.piantanida@centralesupelec.fr