Deep Learning en détail

Notebooks

(à venir foolbox)

Cours

• Deep Learning course: lecture slides and lab notebooks

• Artificial Intelligence, Revealed (1) : article de blog et vidéos expliquant les différents concepts du deep learning

• colah’s blog (2016/08) blog/cours sur le deep learning

• Tutoriels avec CNTK

• Course notes for CS224N Winter17 (Stanford)

Tutoriels

• Building Autoencoders in Keras

• Image Similarity Ranking using Microsoft Cognitive Toolkit (CNTK)

• Tutoriels avec CNTK : ces notebooks sont bien illlustrés (GAN - Generative Models).

• Tutoriels avec TensorFlow : ce ne sont pas les seuls mais ils ont l’avantage d’être bien illustrés (Adversarial Training).

• Machine Learning is Fun! Part 3: Deep Learning and Convolutional Neural Networks

• Machine Learning is Fun! Part 4: Modern Face Recognition with Deep Learning

• Object detection using Fast R-CNN

• Deep Learning - The Straight Dope : séries de notebooks de difficulté graduelle

Sites

• Tinker With a Neural Network Right Here in Your Browser

• ConvNetJS

• Databricks / Deep Learning

Liens

• Four deep learning trends from ACL 2017 (1)

• Four deep learning trends from ACL 2017 (2)

Articles scientifiques

• LightRNN: Memory and Computation-Efficient Recurrent Neural Networks

• Deep learning architecture diagrams

• Factorized Convolutional Neural Networks

• Deep Residual Learning for Image Recognition

• Deep Learning, Yoshua Bengio, Ian Goodfellow and Aaron Courville

• LeNet5

• mxnet

• Benchmarking State-of-the-Art Deep Learning Software Tools

• Wide & Deep Learning: Better Together with TensorFlow, Wide & Deep Learning for Recommender Systems

• To go deep or wide in learning?

• Three Classes of Deep Learning Architectures and Their Applications: A Tutorial Survey

• Tutorial: Learning Deep Architectures

• Deep Learning (wikipédia)

• Fast R-CNN (voir Object Detection using Fast R CNN)

• Evaluation of Deep Learning Toolkits (2015/12)

• Understanding Deep Learning Requires Rethinking Generalization

• Training Deep Nets with Sublinear Memory Cost

• On the importance of initialization and momentum in deep learning

• TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems

• Foolbox v0.8.0: A Python toolbox to benchmark the robustness of machine learning models

Chiffres, Textes

• One weird trick for parallelizing convolutional neural networks

• ImageNet Classification with Deep Convolutional Neural Networks

• Very Deep Convolutional Networks for Large-Scale Image Recognition

• Multi-Digit Recognition Using A Space Displacement Neural Network

• Space Displacement Localization Neural Networks to locate origin points of handwritten text lines in historical documents

• Neural Network Architectures, Convolutional Neural Networks (CNNs / ConvNets)

• Transfer Learning

Benchmarks

• Benchmarking CNTK on Keras: is it Better at Deep Learning than TensorFlow? (code)

Plus théoriques

• Why Does Unsupervized Deep Learning Work? - A perspective from group theory

• Deep Learning of Representations: Looking Forward

• Why Does Unsupervised Pre-training Help Deep Learning?

Lectures deep text

• Efficient Estimation of Word Representations in Vector Space, Tomas Mikolov, Kai Chen, Greg Corrado, Jeffrey Dean,

• Distributed Representations of Words and Phrases and their Compositionality, Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, Jeff Dean,

• word2vec Parameter Learning Explained, Xin Rong,

• Tutorial on Auto-Encoders, Piotr Mirowski

• Pretrained Character Embeddings for Deep Learning and Automatic Text Generation

Vus dans des conférences

• Fast R-CNN (dotAI)

• Mask R-CNN (dotAI)

• Modèle Tenserflow (modèle adaptés pour du transfer learning : ResNet, Inception) (dotAI)

• Domain-Adversarial Training of Neural Networks (dotAI)

Deep learning embarqué

• TensorFlow sur Android

• TensorFlow sur RasberryPI

Modules - deep learning

• pytorch : design plus simple que tous les autres

• keras

• mxnet

• caffe (installation)

• climin (algorithme de back propagation)

• tensorflow (Google)

• cntk

Modules - à suivre

• chainer

• Federated Learning: Collaborative Machine Learning without Centralized Training Data

• foolbox : trouver des petites perturbations des données qui trompent les réseaux de neurones