原论文地址：https://arxiv.org/pdf/1612.00796.pdf
本博客参考了以下博客的理解https://blog.****.net/dhaiuda/article/details/103967676/
本博客仅是个人对此论文的理解，若有理解不当的地方欢迎大家指正。
本篇论文讲述了一种通过给权重添加正则，从而控制权重优化方向，从而达到持续学习效果的方法。其方法简单来讲分为以下三个步骤，其思想如图所示：
选择出对于旧任务（old task）比较重要的权重
对权重的重要程度进行排序
在优化的时候，越重要的权重改变越小，保证其在小范围内改变，不会对旧任务产生较大的影响
具体方法为：将模型的后验概率拟合为一个高斯分布，其中均值为旧任务的权重，方差为 Fisher 信息矩阵（Fisher Information Matrix）的对角元素的倒数。方差就代表了每个权重的重要程度。
灾难性遗忘（Catastrophic Forgetting ）：在网络顺序训练多个任务的时候，对于先前任务的重要权重无法保留。灾难性遗忘是网络结构的必然特征
持续学习 ：在顺序学习任务的时候，不忘记之前训练过的任务。根据任务A训练网络之后，再根据任务B训练同一个网络，此时对任务A进行测试，还可以维持其性能。
P ( A ∣ B ) = P ( A ∩ B ) P ( B ) P(A|B) = \frac{P(A \cap B)}{P(B)} P ( A ∣ B ) = P ( B ) P ( A ∩ B ) P ( B ∣ A ) = P ( A ∩ B ) P ( A ) P(B|A) = \frac{P(A \cap B)}{P(A)} P ( B ∣ A ) = P ( A ) P ( A ∩ B ) P ( A ∣ B ) P ( B ) = P ( B ∣ A ) P ( A ) P(A|B)P(B)=P(B|A)P(A) P ( A ∣ B ) P ( B ) = P ( B ∣ A ) P ( A ) P ( B ∣ A ) = P ( A ∣ B ) P ( B ) P ( A ) P(B|A) = P(A|B)\frac{P( B)}{P(A)} P ( B ∣ A ) = P ( A ∣ B ) P ( A ) P ( B )
θ \theta θ
θ A ∗ \theta^*_A θ A ∗
D D D
D A D_A D A
D B D_B D B
F F F
H H H
对于网络来讲，给定数据集，目的是寻找一个最优的参数，即P ( θ ∣ D ) P(\theta|D) P ( θ ∣ D ) P ( B ∣ A ) = P ( A ∣ B ) P ( B ) P ( A ) P(B|A) = P(A|B)\frac{P( B)}{P(A)} P ( B ∣ A ) = P ( A ∣ B ) P ( A ) P ( B ) P ( θ ∣ D ) = P ( D ∣ θ ) P ( θ ) P ( D ) P(\theta|D) = P(D|\theta)\frac{P( \theta)}{P(D)} P ( θ ∣ D ) = P ( D ∣ θ ) P ( D ) P ( θ ) log P ( θ ∣ D ) = log ( P ( D ∣ θ ) P ( θ ) P ( D ) ) = log P ( D ∣ θ ) + log P ( θ ) − log P ( D ) \log P(\theta|D) = \log (P(D|\theta)\frac{P( \theta)}{P(D)})=\log P(D|\theta) + \log P( \theta) - \log P(D) log P ( θ ∣ D ) = log ( P ( D ∣ θ ) P ( D ) P ( θ ) ) = log P ( D ∣ θ ) + log P ( θ ) − log P ( D )
如果这是两个任务的顺序学习，旧任务为任务 A，新任务为任务 B，那么可以数据集 D D D D A D_A D A D B D_B D B P ( θ ∣ D A , D B ) = P ( θ , D A , D B ) P ( D A , D B ) = P ( θ , D B ∣ D A ) P ( D A ) P ( D B ∣ D A ) P ( D A ) = P ( θ , D B ∣ D A ) P ( D B ∣ D A ) P(\theta|D_A,D_B)=\frac{P(\theta,D_A,D_B)}{P(D_A,D_B)}=\frac{P(\theta,D_B|D_A)P(D_A)}{P(D_B|D_A)P(D_A)}=\frac{P(\theta,D_B|D_A)}{P(D_B|D_A)} P ( θ ∣ D A , D B ) = P ( D A , D B ) P ( θ , D A , D B ) = P ( D B ∣ D A ) P ( D A ) P ( θ , D B ∣ D A ) P ( D A ) = P ( D B ∣ D A ) P ( θ , D B ∣ D A ) P ( θ , D B ∣ D A ) = P ( θ , D B , D A ) = P ( θ , D A , D B ) P ( D A ) = P ( θ , D A , D B ) P ( θ , D A ) ⋅ P ( θ , D A ) P ( D A ) = P ( D B ∣ θ , D A ) P ( θ ∣ D A ) P(\theta,D_B|D_A)=P(\theta,D_B,D_A)=\frac{P(\theta,D_A,D_B)}{P(D_A)}=\frac{P(\theta,D_A,D_B)}{P(\theta,D_A)} \cdot \frac{P(\theta,D_A)}{P(D_A)}=P(D_B|\theta,D_A)P(\theta|D_A) P ( θ , D B ∣ D A ) = P ( θ , D B , D A ) = P ( D A ) P ( θ , D A , D B ) = P ( θ , D A ) P ( θ , D A , D B ) ⋅ P ( D A ) P ( θ , D A ) = P ( D B ∣ θ , D A ) P ( θ ∣ D A ) P ( θ ∣ D A , D B ) = P ( θ , D B ∣ D A ) P ( D B ∣ D A ) = P ( D B ∣ θ , D A ) P ( θ ∣ D A ) P ( D B ∣ D A ) P(\theta|D_A,D_B)=\frac{P(\theta,D_B|D_A)}{P(D_B|D_A)}=\frac{P(D_B|\theta,D_A)P(\theta|D_A)}{P(D_B|D_A)} P ( θ ∣ D A , D B ) = P ( D B ∣ D A ) P ( θ , D B ∣ D A ) = P ( D B ∣ D A ) P ( D B ∣ θ , D A ) P ( θ ∣ D A ) D A D_A D A D B D_B D B P ( D B ∣ D A ) = P ( D B ) P(D_B|D_A)=P(D_B) P ( D B ∣ D A ) = P ( D B ) P ( D B ∣ θ , D A ) = P ( D B ∣ θ ) P(D_B|\theta,D_A)=P(D_B|\theta) P ( D B ∣ θ , D A ) = P ( D B ∣ θ ) P ( θ ∣ D A , D B ) = P ( D B ∣ θ ) P ( θ ∣ D A ) P ( D B ) P(\theta|D_A,D_B)=\frac{P(D_B|\theta)P(\theta|D_A)}{P(D_B)} P ( θ ∣ D A , D B ) = P ( D B ) P ( D B ∣ θ ) P ( θ ∣ D A ) log P ( θ ∣ D ) = log P ( θ ∣ D A , D B ) = log P ( D B ∣ θ ) + l o g P ( θ ∣ D A ) − log P ( D B ) \log P(\theta|D)=\log P(\theta|D_A,D_B)= \log P(D_B|\theta)+logP(\theta|D_A)-\log P(D_B) log P ( θ ∣ D ) = log P ( θ ∣ D A , D B ) = log P ( D B ∣ θ ) + l o g P ( θ ∣ D A ) − log P ( D B )
在给定整个数据集，我们需要得到一个 θ \theta θ
第一项很明显可以理解为任务B的损失函数，将其命名为 L B ( θ ) L_B(\theta) L B ( θ ) θ \theta θ m a x θ log P ( θ ∣ D ) = m a x θ ( − L B ( θ ) + log P ( θ ∣ D A ) ) \mathop{max}\limits_{\theta}\log P(\theta|D)=\mathop{max}\limits_{\theta}(-L_B(\theta)+\log P(\theta|D_A)) θ ma x log P ( θ ∣ D ) = θ ma x ( − L B ( θ ) + log P ( θ ∣ D A ) ) m i n θ ( L B ( θ ) − log P ( θ ∣ D A ) ) \mathop{min}\limits_{\theta}(L_B(\theta)-\log P(\theta|D_A)) θ min ( L B ( θ ) − log P ( θ ∣ D A ) ) log P ( θ ∣ D A ) \log P(\theta|D_A) log P ( θ ∣ D A )
由于后验概率并不容易进行衡量，所以我们将其先验 log P ( D A ∣ θ ) \log P(D_A|\theta) log P ( D A ∣ θ )
令先验 log P ( D A ∣ θ ) \log P(D_A|\theta) log P ( D A ∣ θ ) P ( D A ∣ θ ) ∼ N ( μ , σ ) P(D_A|\theta) \sim N(\mu, \sigma) P ( D A ∣ θ ) ∼ N ( μ , σ ) P ( D A ∣ θ ) = 1 2 π σ e − ( θ − μ ) 2 2 σ 2 P(D_A|\theta)=\frac{1}{\sqrt{2 \pi}\sigma} e^{-\frac{(\theta-\mu)^2}{2\sigma^2}} P ( D A ∣ θ ) = 2 π σ 1 e − 2 σ 2 ( θ − μ ) 2 log P ( D A ∣ θ ) = log 1 2 π σ − ( θ − μ ) 2 2 σ 2 \log P(D_A|\theta)=\log \frac{1}{\sqrt{2 \pi}\sigma} -\frac{(\theta-\mu)^2}{2\sigma^2} log P ( D A ∣ θ ) = log 2 π σ 1 − 2 σ 2 ( θ − μ ) 2 f ( θ ) = log P ( D A ∣ θ ) f(\theta)=\log P(D_A|\theta) f ( θ ) = log P ( D A ∣ θ ) θ = θ A ∗ \theta = \theta_A^* θ = θ A ∗ f ′ ( θ A ∗ ) = 0 f'(\theta_A^*)=0 f ′ ( θ A ∗ ) = 0 f ( θ ) = f ( θ A ∗ ) + f ′ ( θ A ∗ ) ( θ − θ A ∗ ) + f ′ ′ ( θ A ∗ ) ( θ − θ A ∗ ) 2 2 + o ( θ A ∗ ) f(\theta)=f(\theta_A^*)+f'(\theta_A^*)(\theta-\theta_A^*)+f''(\theta_A^*)\frac{(\theta-\theta_A^*)^2}{2}+o(\theta_A^*) f ( θ ) = f ( θ A ∗ ) + f ′ ( θ A ∗ ) ( θ − θ A ∗ ) + f ′ ′ ( θ A ∗ ) 2 ( θ − θ A ∗ ) 2 + o ( θ A ∗ ) log 1 2 π σ − ( θ − μ ) 2 2 σ 2 ≈ f ( θ A ∗ ) + f ′ ′ ( θ A ∗ ) ( θ − θ A ∗ ) 2 2 \log \frac{1}{\sqrt{2 \pi}\sigma} -\frac{(\theta-\mu)^2}{2\sigma^2}\approx f(\theta_A^*)+f''(\theta_A^*)\frac{(\theta-\theta_A^*)^2}{2} log 2 π σ 1 − 2 σ 2 ( θ − μ ) 2 ≈ f ( θ A ∗ ) + f ′ ′ ( θ A ∗ ) 2 ( θ − θ A ∗ ) 2 log 1 2 π σ \log \frac{1}{\sqrt{2 \pi}\sigma} log 2 π σ 1 f ( θ A ∗ ) f(\theta_A^*) f ( θ A ∗ ) − ( θ − μ ) 2 2 σ 2 = f ′ ′ ( θ A ∗ ) ( θ − θ A ∗ ) 2 2 -\frac{(\theta-\mu)^2}{2\sigma^2}= f''(\theta_A^*)\frac{(\theta-\theta_A^*)^2}{2} − 2 σ 2 ( θ − μ ) 2 = f ′ ′ ( θ A ∗ ) 2 ( θ − θ A ∗ ) 2 μ = θ A ∗ \mu = \theta_A^* μ = θ A ∗ σ 2 = − 1 f ′ ′ ( θ A ∗ ) \sigma^2=-\frac{1}{f''(\theta_A^*)} σ 2 = − f ′ ′ ( θ A ∗ ) 1 P ( D A ∣ θ ) ∼ N ( θ A ∗ , − 1 f ′ ′ ( θ A ∗ ) ) P(D_A|\theta) \sim N(\theta_A^*, -\frac{1}{f''(\theta_A^*)}) P ( D A ∣ θ ) ∼ N ( θ A ∗ , − f ′ ′ ( θ A ∗ ) 1 ) P ( θ ∣ D A ) = P ( P A , θ ) P ( θ ) P ( A ) P(\theta|D_A) = \frac{P(P_A,\theta)P(\theta)}{P(A)} P ( θ ∣ D A ) = P ( A ) P ( P A , θ ) P ( θ ) P ( θ ) P(\theta) P ( θ ) P ( D A ) P(D_A) P ( D A ) P ( θ ∣ D A ) ∼ N ( θ A ∗ , − 1 f ′ ′ ( θ A ∗ ) ) P(\theta|D_A) \sim N(\theta_A^*, -\frac{1}{f''(\theta_A^*)}) P ( θ ∣ D A ) ∼ N ( θ A ∗ , − f ′ ′ ( θ A ∗ ) 1 ) m i n θ ( L B ( θ ) − log P ( θ ∣ D A ) ) \mathop{min}\limits_{\theta}(L_B(\theta)-\log P(\theta|D_A)) θ min ( L B ( θ ) − log P ( θ ∣ D A ) ) m i n θ ( L B ( θ ) − f ′ ′ ( θ A ∗ ) ( θ − θ A ∗ ) 2 2 \mathop{min}\limits_{\theta}(L_B(\theta)- f''(\theta_A^*)\frac{(\theta-\theta_A^*)^2}{2} θ min ( L B ( θ ) − f ′ ′ ( θ A ∗ ) 2 ( θ − θ A ∗ ) 2 m i n θ ( L B ( θ ) − ∑ i f i ′ ′ ( θ A ∗ ) ( θ i − θ A , i ∗ ) 2 2 \mathop{min}\limits_{\theta}(L_B(\theta)- \sum_i f''_i(\theta_A^*)\frac{(\theta_i-\theta_{A,i}^*)^2}{2} θ min ( L B ( θ ) − i ∑ f i ′ ′ ( θ A ∗ ) 2 ( θ i − θ A , i ∗ ) 2 f ′ ′ ( θ A ∗ ) f''(\theta_A^*) f ′ ′ ( θ A ∗ )
Fisher information 是概率分布梯度的协方差。为了更好的说明Fisher Information matrix 的含义，这里定义一个得分函数 S S S S ( θ ) = ∇ log p ( x ∣ θ ) S(\theta)=\nabla \log p(x|\theta) S ( θ ) = ∇ log p ( x ∣ θ ) E p ( x ∣ θ ) [ S ( θ ) ] = E p ( x ∣ θ ) [ ∇ log p ( x ∣ θ ) ] = ∫ ∇ log p ( x ∣ θ ) ⋅ p ( x ∣ θ ) d θ = ∫ ∇ p ( x ∣ θ ) p ( x ∣ θ ) ⋅ p ( x ∣ θ ) d θ = ∫ ∇ p ( x ∣ θ ) d θ = ∇ ∫ p ( x ∣ θ ) d θ = ∇ 1 = 0 \begin{aligned}
\mathop{E}\limits_{p(x|\theta)}[S(\theta)] &=\mathop{E}\limits_{p(x|\theta)}[\nabla \log p(x|\theta)] \\
&= \int \nabla \log p(x|\theta) \cdot p(x|\theta) d\theta \\
&= \int \frac{\nabla p(x|\theta)}{p(x|\theta)} \cdot p(x|\theta) d\theta \\
&= \int \nabla p(x|\theta) d\theta \\
&= \nabla \int p(x|\theta) d\theta \\
& = \nabla 1=0
\end{aligned} p ( x ∣ θ ) E [ S ( θ ) ] = p ( x ∣ θ ) E [ ∇ log p ( x ∣ θ ) ] = ∫ ∇ log p ( x ∣ θ ) ⋅ p ( x ∣ θ ) d θ = ∫ p ( x ∣ θ ) ∇ p ( x ∣ θ ) ⋅ p ( x ∣ θ ) d θ = ∫ ∇ p ( x ∣ θ ) d θ = ∇ ∫ p ( x ∣ θ ) d θ = ∇ 1 = 0 F F F F = E p ( X ∣ θ ) [ ( S ( θ ) − 0 ) ( S ( θ ) − 0 ) T ] F = \mathop{E}\limits_{p(X|\theta)}[(S(\theta)-0)(S(\theta)-0)^T] F = p ( X ∣ θ ) E [ ( S ( θ ) − 0 ) ( S ( θ ) − 0 ) T ] X = { x 1 , x 2 , ⋯ , x n } X = \{x_1,x_2,\cdots ,x_n\} X = { x 1 , x 2 , ⋯ , x n } F = 1 N ∑ i = 1 N ∇ log p ( x i ∣ θ ) ∇ log p ( x i ∣ θ ) T F = \frac{1}{N}\sum_{i=1}^N \nabla \log p(x_i|\theta) \nabla \log p(x_i|\theta)^T F = N 1 i = 1 ∑ N ∇ log p ( x i ∣ θ ) ∇ log p ( x i ∣ θ ) T
Hessian矩阵为H log p ( x ∣ θ ) = J ( ∇ log p ( x ∣ t h e t a ) ) = J ( ∇ p ( x ∣ θ ) p ( x ∣ θ ) ) = H p ( x ∣ θ ) p ( x ∣ θ ) − ∇ p ( x ∣ θ ) ∇ p ( x ∣ θ ) T p ( x ∣ θ ) p ( x ∣ θ ) = H p ( x ∣ θ ) p ( x ∣ θ ) − ∇ p ( x ∣ θ ) ∇ p ( x ∣ θ ) T p ( x ∣ θ ) p ( x ∣ θ ) = H p ( x ∣ θ ) p ( x ∣ θ ) − ( ∇ p ( x ∣ θ ) p ( x ∣ θ ) ) ( ∇ p ( x ∣ θ ) p ( x ∣ θ ) ) T \begin{aligned}
H_{\log p(x|\theta)} &= J(\nabla \log p(x|theta)) = J(\frac{ \nabla p(x|\theta)}{ p(x|\theta)}) \\
&= \frac{H_{ p(x|\theta)} p(x|\theta)- \nabla p(x|\theta) \nabla p(x|\theta)^T}{ p(x|\theta) p(x|\theta)} \\
&= \frac{H_{ p(x|\theta)}}{ p(x|\theta)}-\frac{ \nabla p(x|\theta) \nabla p(x|\theta)^T}{ p(x|\theta) p(x|\theta)} \\
&= \frac{H_{ p(x|\theta)}}{ p(x|\theta)}-(\frac{ \nabla p(x|\theta)}{ p(x|\theta) })(\frac{ \nabla p(x|\theta)}{ p(x|\theta) })^T
\end{aligned} H log p ( x ∣ θ ) = J ( ∇ log p ( x ∣ t h e t a ) ) = J ( p ( x ∣ θ ) ∇ p ( x ∣ θ ) ) = p ( x ∣ θ ) p ( x ∣ θ ) H p ( x ∣ θ ) p ( x ∣ θ ) − ∇ p ( x ∣ θ ) ∇ p ( x ∣ θ ) T = p ( x ∣ θ ) H p ( x ∣ θ ) − p ( x ∣ θ ) p ( x ∣ θ ) ∇ p ( x ∣ θ ) ∇ p ( x ∣ θ ) T = p ( x ∣ θ ) H p ( x ∣ θ ) − ( p ( x ∣ θ ) ∇ p ( x ∣ θ ) ) ( p ( x ∣ θ ) ∇ p ( x ∣ θ ) ) T E p ( x ∣ θ ) [ H log p ( x ∣ θ ) ] = E p ( x ∣ θ ) [ H p ( x ∣ θ ) p ( x ∣ θ ) ] − E p ( x ∣ θ ) [ ( ∇ p ( x ∣ θ ) p ( x ∣ θ ) ) ( ∇ p ( x ∣ θ ) p ( x ∣ θ ) ) T ] = ∫ H p ( x ∣ θ ) p ( x ∣ θ ) p ( x ∣ θ ) d θ − E p ( x ∣ θ ) [ ∇ log p ( x ∣ θ ) ∇ log p ( x ∣ θ ) T ] = ∫ H p ( x ∣ θ ) p ( x ∣ θ ) p ( x ∣ θ ) d θ − E p ( x ∣ θ ) [ ( S ( θ ) − 0 ) ( S ( θ ) − 0 ) T ] = ∫ H p ( x ∣ θ ) d θ − F = H 1 − F = 0 − F = − F \begin{aligned}
\mathop{E}\limits_{p(x|\theta)}[H_{\log p(x|\theta)}] &= \mathop{E}\limits_{p(x|\theta)}[\frac{H_{ p(x|\theta)}}{ p(x|\theta)}]- \mathop{E}\limits_{p(x|\theta)}[(\frac{ \nabla p(x|\theta)}{p(x|\theta) })(\frac{ \nabla p(x|\theta)}{ p(x|\theta) })^T] \\
&= \int \frac{H_{ p(x|\theta)}}{ p(x|\theta)} p(x|\theta) d\theta - \mathop{E}\limits_{p(x|\theta)}[\nabla \log p(x|\theta) \nabla \log p(x|\theta)^T] \\
& = \int \frac{H_{ p(x|\theta)}}{ p(x|\theta)} p(x|\theta) d\theta - \mathop{E}\limits_{p(x|\theta)}[(S(\theta)-0)(S(\theta)-0)^T] \\
&= \int {H_{ p(x|\theta)}} d\theta -F \\
&= H_1 -F = 0-F \\
&=-F
\end{aligned} p ( x ∣ θ ) E [ H log p ( x ∣ θ ) ] = p ( x ∣ θ ) E [ p ( x ∣ θ ) H p ( x ∣ θ ) ] − p ( x ∣ θ ) E [ ( p ( x ∣ θ ) ∇ p ( x ∣ θ ) ) ( p ( x ∣ θ ) ∇ p ( x ∣ θ ) ) T ] = ∫ p ( x ∣ θ ) H p ( x ∣ θ ) p ( x ∣ θ ) d θ − p ( x ∣ θ ) E [ ∇ log p ( x ∣ θ ) ∇ log p ( x ∣ θ ) T ] = ∫ p ( x ∣ θ ) H p ( x ∣ θ ) p ( x ∣ θ ) d θ − p ( x ∣ θ ) E [ ( S ( θ ) − 0 ) ( S ( θ ) − 0 ) T ] = ∫ H p ( x ∣ θ ) d θ − F = H 1 − F = 0 − F = − F
因为 f ′ ′ ( x ) f''(x) f ′ ′ ( x ) H H H − 1 f ′ ′ ( x ) -\frac{1}{f''(x)} − f ′ ′ ( x ) 1 F F F m i n θ ( L B ( θ ) − ∑ i f i ′ ′ ( θ A ∗ ) ( θ i − θ A , i ∗ ) 2 2 \mathop{min}\limits_{\theta}(L_B(\theta)- \sum_i f''_i(\theta_A^*)\frac{(\theta_i-\theta_{A,i}^*)^2}{2} θ min ( L B ( θ ) − i ∑ f i ′ ′ ( θ A ∗ ) 2 ( θ i − θ A , i ∗ ) 2 m i n θ ( L B ( θ ) − ∑ i F i ( θ i − θ A , i ∗ ) 2 2 \mathop{min}\limits_{\theta}(L_B(\theta)- \sum_i F_i\frac{(\theta_i-\theta_{A,i}^*)^2}{2} θ min ( L B ( θ ) − i ∑ F i 2 ( θ i − θ A , i ∗ ) 2 λ \lambda λ m i n θ ( L B ( θ ) − λ 2 ∑ i F i ( θ i − θ A , i ∗ ) 2 \mathop{min}\limits_{\theta}(L_B(\theta)- \frac{\lambda}{2}\sum_i F_i(\theta_i-\theta_{A,i}^*)^2 θ min ( L B ( θ ) − 2 λ i ∑ F i ( θ i − θ A , i ∗ ) 2
