Machine Learning

Looking for a Function

• Speech Recognition
• Handwritten Recongnition
• Weather Forecast
• Play video games

Machine Learning Framework

• Models: Hypothesis Function Set
• Training: Pick the best function
• Testing: test the best function

Deep Learning

• Production line: a very complex function
• Deep Model: All function are learned from data
• A Single Neuron: A Single Simple Function

Why Speed Matters

• Traning time: Big data increases the training time
• Inference time: not patient to wait

Why Deeper is Better

• More parameters
• Universality Theorem

How to Apply Deep Learning

• Input domain: Aggregate Information
• Output domain: Classification

Layer Output Relation

Neural Network Formulation

Chain Rule

we can get this:

Backpropagation

Backward Pass And Forward Pass

Momentum

考虑之前数据的惯性作用，避免无法突破 local minimum