机器学习基石 - Linear Regression

机器学习基石上 (Machine Learning Foundations)—Mathematical Foundations
Hsuan-Tien Lin, 林轩田，副教授 (Associate Professor)，资讯工程学系 (Computer Science and Information Engineering)

Linear Regression Problem

• hypothesis: h(x)=wTx$h(\mathbf{x})={\mathbf{w}}^T\mathbf{x}$
• find lines/hyperplanes with small residuals

Linear Regression Algorithm

• Matrix Form of Ein(w)$Ein(\mathbf{w})$

  

• find wLIN${\mathbf{w}}_{LIN}$ such that ∇Ein(wLIN)=0$\mathrm{\nabla }{E}_{in}({\mathbf{w}}_{LIN})=0$

  
  

• 伪逆矩阵 pseudo-inverse

  

  矩阵的平方展开、求导，抓住一个或一行的例子，纵观计算过程，推倒式子

Generalization Issue

• hat matrix H=XX†$H=X{X}^{†}$ , y^=Hy$\hat{y}=Hy$

• Geometric View of Hat Matrix

  

  H$H$将y$y$投影到y^$\hat{y}$

• The Learning Curve

  

  （平均情况）

Linear Regression for Binary Classification

• 可以用回归的方法解决分类的问题
•