2020年最新深度学习模型、策略整理及实现汇总分享

    本资源整理了常见的各类深度学习模型和策略，涉及机器学习基础、神经网路基础、CNN、GNN、RNN、GAN等，并给出了基于TensorFlow或 PyTorch的实现细节，这些实现都是Jupyter Notebooks编写，可运行Debug且配有详细的讲解，可以帮助你体会算法实现的细节。

    资源整理自网络，源地址：https://github.com/rasbt/deeplearning-models

 

    带链接版资源下载地址：

    链接: https://pan.baidu.com/s/1jsqqTPtA33UfEPUIJ0qCmg

    提取码: 97nm

传统机器学习

    •Perceptron

       [TensorFlow 1: GitHub | Nbviewer]

       [PyTorch: GitHub | Nbviewer]

    •Logistic Regression

       [TensorFlow 1: GitHub | Nbviewer]

       [PyTorch: GitHub | Nbviewer]

    •Softmax Regression (Multinomial Logistic Regression)

       [TensorFlow 1: GitHub | Nbviewer]

       [PyTorch: GitHub | Nbviewer]

    •Softmax Regression with MLxtend's plot_decision_regions on Iris

       [PyTorch: GitHub | Nbviewer]

 

多层感知器

    •Multilayer Perceptron

       [TensorFlow 1: GitHub | Nbviewer]

       [PyTorch: GitHub | Nbviewer]

    •Multilayer Perceptron with Dropout

       [TensorFlow 1: GitHub | Nbviewer]

       [PyTorch: GitHub | Nbviewer]

    •Multilayer Perceptron with Batch Normalization

       [TensorFlow 1: GitHub | Nbviewer]

       [PyTorch: GitHub | Nbviewer]

    •Multilayer Perceptron with Backpropagation from Scratch

       [TensorFlow 1: GitHub | Nbviewer]

       [PyTorch: GitHub | Nbviewer]

 

卷积神经网络

    Basic

    •Convolutional Neural Network

       [TensorFlow 1: GitHub | Nbviewer]

       [PyTorch: GitHub | Nbviewer]

    •Convolutional Neural Network with He Initialization

       [PyTorch: GitHub | Nbviewer]

    Concepts

    •Replacing Fully-Connnected by Equivalent Convolutional Layers

       [PyTorch: GitHub | Nbviewer]

    Fully Convolutional

    •Fully Convolutional Neural Network

       [PyTorch: GitHub | Nbviewer]

    LeNet

    •LeNet-5 on MNIST

       [PyTorch: GitHub | Nbviewer]

    •LeNet-5 on CIFAR-10

       [PyTorch: GitHub | Nbviewer]

    •LeNet-5 on QuickDraw

       [PyTorch: GitHub | Nbviewer]

    AlexNet

    •AlexNet on CIFAR-10

       [PyTorch: GitHub | Nbviewer]

    VGG

    •Convolutional Neural Network VGG-16

       [TensorFlow 1: GitHub | Nbviewer]

       [PyTorch: GitHub | Nbviewer]

    •VGG-16 Gender Classifier Trained on CelebA

       [PyTorch: GitHub | Nbviewer]

    •Convolutional Neural Network VGG-19

       [PyTorch: GitHub | Nbviewer]

    DenseNet

    •DenseNet-121 Digit Classifier Trained on MNIST

       [PyTorch: GitHub | Nbviewer]

    •DenseNet-121 Image Classifier Trained on CIFAR-10

       [PyTorch: GitHub | Nbviewer]

    ResNet

    •ResNet and Residual Blocks

       [PyTorch: GitHub | Nbviewer]

    •ResNet-18 Digit Classifier Trained on MNIST

       [PyTorch: GitHub | Nbviewer]

    •ResNet-18 Gender Classifier Trained on CelebA

       [PyTorch: GitHub | Nbviewer]

    •ResNet-34 Digit Classifier Trained on MNIST

       [PyTorch: GitHub | Nbviewer]

    •ResNet-34 Object Classifier Trained on QuickDraw

       [PyTorch: GitHub | Nbviewer]

    •ResNet-34 Gender Classifier Trained on CelebA

       [PyTorch: GitHub | Nbviewer]

    •ResNet-50 Digit Classifier Trained on MNIST

       [PyTorch: GitHub | Nbviewer]

    •ResNet-50 Gender Classifier Trained on CelebA

       [PyTorch: GitHub | Nbviewer]

    •ResNet-101 Gender Classifier Trained on CelebA

       [PyTorch: GitHub | Nbviewer]

    •ResNet-101 Trained on CIFAR-10

       [PyTorch: GitHub | Nbviewer]

    •ResNet-152 Gender Classifier Trained on CelebA

       [PyTorch: GitHub | Nbviewer]

    Network in Network

    •Network in Network CIFAR-10 Classifier

       [PyTorch: GitHub | Nbviewer]

 

归一化层

    •BatchNorm before and after Activation for Network-in-Network CIFAR-10 Classifier

       [PyTorch: GitHub | Nbviewer]

    •Filter Response Normalization for Network-in-Network CIFAR-10 Classifier

       [PyTorch: GitHub | Nbviewer]

 

度量学习

    •Siamese Network with Multilayer Perceptrons

       [TensorFlow 1: GitHub | Nbviewer]

 

自编码器

    Fully-connected Autoencoders

    •Autoencoder (MNIST)

       [TensorFlow 1: GitHub | Nbviewer]

       [PyTorch: GitHub | Nbviewer]

    •Autoencoder (MNIST) + Scikit-Learn Random Forest Classifier

       [TensorFlow 1: GitHub | Nbviewer]

       [PyTorch: GitHub | Nbviewer]

    Convolutional Autoencoders

    •Convolutional Autoencoder with Deconvolutions / Transposed Convolutions

       [TensorFlow 1: GitHub | Nbviewer]

       [PyTorch: GitHub | Nbviewer]

    •Convolutional Autoencoder with Deconvolutions and Continuous Jaccard Distance

       [PyTorch: GitHub | Nbviewer]

    •Convolutional Autoencoder with Deconvolutions (without pooling operations)

       [PyTorch: GitHub | Nbviewer]

    •Convolutional Autoencoder with Nearest-neighbor Interpolation

       [TensorFlow 1: GitHub | Nbviewer]

       [PyTorch: GitHub | Nbviewer]

    •Convolutional Autoencoder with Nearest-neighbor Interpolation -- Trained on CelebA

       [PyTorch: GitHub | Nbviewer]

    •Convolutional Autoencoder with Nearest-neighbor Interpolation -- Trained on Quickdraw

       [PyTorch: GitHub | Nbviewer]

    Variational Autoencoders

    •Variational Autoencoder

       [PyTorch: GitHub | Nbviewer]

    •Convolutional Variational Autoencoder

       [PyTorch: GitHub | Nbviewer]

    Conditional Variational Autoencoders

    •Conditional Variational Autoencoder (with labels in reconstruction loss)

       [PyTorch: GitHub | Nbviewer]

    •Conditional Variational Autoencoder (without labels in reconstruction loss)

       [PyTorch: GitHub | Nbviewer]

    •Convolutional Conditional Variational Autoencoder (with labels in reconstruction loss)

       [PyTorch: GitHub | Nbviewer]

    •Convolutional Conditional Variational Autoencoder (without labels in reconstruction loss)

       [PyTorch: GitHub | Nbviewer]

 

生成对抗网络 (GANs)

    •Fully Connected GAN on MNIST

       [TensorFlow 1: GitHub | Nbviewer]

       [PyTorch: GitHub | Nbviewer]

    •Fully Connected Wasserstein GAN on MNIST

       [PyTorch: GitHub | Nbviewer]

    •Convolutional GAN on MNIST

       [TensorFlow 1: GitHub | Nbviewer]

       [PyTorch: GitHub | Nbviewer]

    •Convolutional GAN on MNIST with Label Smoothing

       [TensorFlow 1: GitHub | Nbviewer]

       [PyTorch: GitHub | Nbviewer]

    •Convolutional Wasserstein GAN on MNIST

       [PyTorch: GitHub | Nbviewer]

 

图神经网络 (GNNs)

    •Most Basic Graph Neural Network with Gaussian Filter on MNIST

       [PyTorch: GitHub | Nbviewer]

    •Basic Graph Neural Network with Edge Prediction on MNIST

       [PyTorch: GitHub | Nbviewer]

    •Basic Graph Neural Network with Spectral Graph Convolution on MNIST

       [PyTorch: GitHub | Nbviewer]

 

循环神经网络(RNNs)

    Many-to-one: Sentiment Analysis / Classification

    •A simple single-layer RNN (IMDB)

       [PyTorch: GitHub | Nbviewer]

    •A simple single-layer RNN with packed sequences to ignore padding characters (IMDB)

       [PyTorch: GitHub | Nbviewer]

    •RNN with LSTM cells (IMDB)

       [PyTorch: GitHub | Nbviewer]

    •RNN with LSTM cells (IMDB) and pre-trained GloVe word vectors

       [PyTorch: GitHub | Nbviewer]

    •RNN with LSTM cells and Own Dataset in CSV Format (IMDB)

       [PyTorch: GitHub | Nbviewer]

    •RNN with GRU cells (IMDB)

       [PyTorch: GitHub | Nbviewer]

    •Multilayer bi-directional RNN (IMDB)

       [PyTorch: GitHub | Nbviewer]

    •Bidirectional Multi-layer RNN with LSTM with Own Dataset in CSV Format (AG News)

       [PyTorch: GitHub | Nbviewer]

    •Bidirectional Multi-layer RNN with LSTM with Own Dataset in CSV Format (Yelp Review Polarity)

       [PyTorch: GitHub | Nbviewer]

    •Bidirectional Multi-layer RNN with LSTM with Own Dataset in CSV Format (Amazon Review Polarity)

       [PyTorch: GitHub | Nbviewer]

    Many-to-Many / Sequence-to-Sequence

    •A simple character RNN to generate new text (Charles Dickens)

       [PyTorch: GitHub | Nbviewer]

 

有序回归

    •Ordinal Regression CNN -- CORAL w. ResNet34 on AFAD-Lite

       [PyTorch: GitHub | Nbviewer]

    •Ordinal Regression CNN -- Niu et al. 2016 w. ResNet34 on AFAD-Lite

       [PyTorch: GitHub | Nbviewer]

    •Ordinal Regression CNN -- Beckham and Pal 2016 w. ResNet34 on AFAD-Lite

       [PyTorch: GitHub | Nbviewer]

 

策略和技巧

    •Cyclical Learning Rate

       [PyTorch: GitHub | Nbviewer]

    •Annealing with Increasing the Batch Size (w. CIFAR-10 & AlexNet)

       [PyTorch: GitHub | Nbviewer]

    •Gradient Clipping (w. MLP on MNIST)

       [PyTorch: GitHub | Nbviewer]

 

迁移学习

    •Transfer Learning Example (VGG16 pre-trained on ImageNet for Cifar-10)

       [PyTorch: GitHub | Nbviewer]

 

PyTorch工作流程和机制

    Custom Datasets

    •Custom Data Loader Example for PNG Files

       [PyTorch: GitHub | Nbviewer]

    •Using PyTorch Dataset Loading Utilities for Custom Datasets -- CSV files converted to HDF5

       [PyTorch: GitHub | Nbviewer]

    •Using PyTorch Dataset Loading Utilities for Custom Datasets -- Face Images from CelebA

       [PyTorch: GitHub | Nbviewer]

    •Using PyTorch Dataset Loading Utilities for Custom Datasets -- Drawings from Quickdraw

       [PyTorch: GitHub | Nbviewer]

    •Using PyTorch Dataset Loading Utilities for Custom Datasets -- Drawings from the Street View House Number (SVHN) Dataset

       [PyTorch: GitHub | Nbviewer]

    •Using PyTorch Dataset Loading Utilities for Custom Datasets -- Asian Face Dataset (AFAD)

       [PyTorch: GitHub | Nbviewer]

    •Using PyTorch Dataset Loading Utilities for Custom Datasets -- Dating Historical Color Images

       [PyTorch: GitHub | Nbviewer]

    Training and Preprocessing

    •Generating Validation Set Splits

    [PyTorch]: GitHub | Nbviewer]

    •Dataloading with Pinned Memory

       [PyTorch: GitHub | Nbviewer]

    •Standardizing Images

       [PyTorch: GitHub | Nbviewer]

    •Image Transformation Examples

       [PyTorch: GitHub | Nbviewer]

    •Char-RNN with Own Text File

       [PyTorch: GitHub | Nbviewer]

    •Sentiment Classification RNN with Own CSV File

       [PyTorch: GitHub | Nbviewer]

    Parallel Computing

    •Using Multiple GPUs with DataParallel -- VGG-16 Gender Classifier on CelebA

       [PyTorch: GitHub | Nbviewer]

    Other

    •Sequential API and hooks

       [PyTorch: GitHub | Nbviewer]

    •Weight Sharing Within a Layer

       [PyTorch: GitHub | Nbviewer]

    •Plotting Live Training Performance in Jupyter Notebooks with just Matplotlib

       [PyTorch: GitHub | Nbviewer]

    Autograd

    •Getting Gradients of an Intermediate Variable in PyTorch

       [PyTorch: GitHub | Nbviewer]

 

TensorFlow工作流程和机制

    Custom Datasets

    •Chunking an Image Dataset for Minibatch Training using NumPy NPZ Archives

       [TensorFlow 1: GitHub | Nbviewer]

    •Storing an Image Dataset for Minibatch Training using HDF5

       [TensorFlow 1: GitHub | Nbviewer]

    •Using Input Pipelines to Read Data from TFRecords Files

       [TensorFlow 1: GitHub | Nbviewer]

    •Using Queue Runners to Feed Images Directly from Disk

       [TensorFlow 1: GitHub | Nbviewer]

    •Using TensorFlow's Dataset API

       [TensorFlow 1: GitHub | Nbviewer]

    Training and Preprocessing

    •Saving and Loading Trained Models -- from TensorFlow Checkpoint Files and NumPy NPZ Archives

       [TensorFlow 1: GitHub | Nbviewer]