您的位置: 首页  >  文章  >  误差逆传播计算

误差逆传播计算

分类: 文章 • 2024-02-05 20:02:46
一、考虑以下多层前馈网络，进行前馈传播和误差逆传播相应的计算

误差逆传播计算

(1) 输入在神经网络上进行一次前馈传播， 请计算隐层和输出层中每个神经元的输入和输出；

依题意：根据17363006，则输入为
A=x1=0.17+0.062=0.115 A = x_1= \frac{0.17+0.06}{2} = 0.115

B=x2=0.36+0.062=0.210 B = x_2= \frac{0.36+0.06}{2} = 0.210

然后进行前馈传播，首先是第一层隐层的神经元的输入和输出如下：
C(in)=0.1∗A+0.8∗B=0.1795 C(in) = 0.1*A+0.8*B=0.1795

C(out)=sigmoid(C(in))=11+e−0.1795=0.5448 C(out)=sigmoid(C(in))=\frac{1}{1+e^-0.1795}=0.5448

D(in)=0.4∗A+0.6∗B=0.1720 D(in)=0.4*A+0.6*B=0.1720

D(out)=sigmoid(C(in))=11+e−0.1720=0.5429 D(out)=sigmoid(C(in))=\frac{1}{1+e^-0.1720}=0.5429

E(in)=0.3∗A+0.5∗B=0.1395 E(in)=0.3*A+0.5*B=0.1395

E(out)=sigmoid(E(in))=11+e−0.1395=0.5348 E(out)=sigmoid(E(in))=\frac{1}{1+e^-0.1395}=0.5348

下面是第二层隐层的神经元的输入和输出：
F(in)=0.9∗C(out)+0.2∗D(out)+0.7∗E(out)=0.9733 F(in)=0.9*C(out)+0.2*D(out)+0.7*E(out)=0.9733

F(out)=sigmoid(F(in))=11+e−0.9733=0.7258 F(out)=sigmoid(F(in))=\frac{1}{1+e^-0.9733}=0.7258

G(in)=0.6∗C(out)+0.3∗D(out)+0.7∗E(out)=0.8641 G(in)=0.6*C(out)+0.3*D(out)+0.7*E(out)=0.8641

G(out)=sigmoid(G(in))=11+e−0.8641=0.7035 G(out)=sigmoid(G(in))=\frac{1}{1+e^-0.8641}=0.7035

H(in)=0.1∗C(out)+0.8∗D(out)+0.5∗E(out)=0.7562 H(in)=0.1*C(out)+0.8*D(out)+0.5*E(out)=0.7562

H(out)=sigmoid(H(in))=11+e−0.7562=0.6805 H(out)=sigmoid(H(in))=\frac{1}{1+e^-0.7562}=0.6805

最后是输出层神经元的输入输出：
M(in)=0.4∗F(out)+0.3∗G(out)+0.5∗H(out)=0.8416 M(in)=0.4*F(out)+0.3*G(out)+0.5*H(out)=0.8416

M(out)=sigmoid(M(in))=11+e−0.8416=0.6988 M(out)=sigmoid(M(in))=\frac{1}{1+e^-0.8416}=0.6988

N(in)=0.6∗F(out)+0.2∗G(out)+0.8∗H(out)=1.1206 N(in)=0.6*F(out)+0.2*G(out)+0.8*H(out)=1.1206

N(out)=sigmoid(N(in))=11+e−1.1206=0.7541 N(out)=sigmoid(N(in))=\frac{1}{1+e^-1.1206}=0.7541

(2) 进行一次误差逆传播， 请计算隐层和输出层每个神经元的误差，并更新每个权重参数。

​ 依题意，目标值为
y=(y1,y2)=(0.30+0.062,0.06)T=(0.18,0.06)T y = (y_1,y_2)=(\frac{0.30+0.06}{2},0.06)^T=(0.18,0.06)^T
​ 网络输出值
y^=(y^1,y^2)=(M(out),N(out))T=(0.6988,0.7541)T \widehat{y}=(\widehat{y}_1,\widehat{y}_2)=(M(out),N(out))^T=(0.6988,0.7541)^T
​ 网络输出层总误差：
E=12∗∑j=12(y^j−yj)2=12∗[(0.6988−0.18)2+(0.7541−0.06)2]=0.3755 E= \frac {1}{2}*\sum_{j=1}^2(\widehat{y}_j-y_j)^2=\frac {1}{2}*[(0.6988-0.18)^2+(0.7541-0.06)^2]=0.3755

ps：每个神经元误差 error 记为 δ 且计为负, sigmoid函数求导的性质：f′(x)=f(x)(1−f(x)) ps：每个神经元误差\ error\ 记为\ \delta \ 且计为负,\ sigmoid函数求导的性质：f'(x)=f(x)(1-f(x))

​

​ 输出层每个神经元的 error ：(链式法则)
δM=−∂E∂M(in)=∂E∂M(out)⋅∂M(out)∂M(in)=(y1−M(out))⋅M(out)⋅(1−M(out))=−0.1092 \delta_M = -\frac{\partial E}{\partial M(in)}=\frac{\partial E}{\partial M(out)}\cdot\frac{\partial M(out)}{\partial M(in)}=(y_1-M(out))\cdot M(out)\cdot(1-M(out))=-0.1092

δN=−∂E∂N(in)=∂E∂N(out)⋅∂N(out)∂N(in)=(y2−N(out))⋅N(out)⋅(1−N(out))=−0.1287 \delta_N = -\frac{\partial E}{\partial N(in)}=\frac{\partial E}{\partial N(out)}\cdot\frac{\partial N(out)}{\partial N(in)}=(y_2-N(out))\cdot N(out)\cdot(1-N(out))=-0.1287

第二层隐层神经元的 error：
δF=−∂E∂F(in)=−（∂E∂M(in)⋅∂M(in)∂F(out)⋅∂F(out)∂F(in)+∂E∂N(in)⋅∂N(in)∂F(out)⋅∂F(out)∂F(in)）=−0.0241 \delta_F = -\frac{\partial E}{\partial F(in)}=-（\frac{\partial E}{\partial M(in)}\cdot\frac{\partial M(in)}{\partial F(out)}\cdot \frac{\partial F(out)}{\partial F(in)}+ \frac{\partial E}{\partial N(in)}\cdot\frac{\partial N(in)}{\partial F(out)}\cdot \frac{\partial F(out)}{\partial F(in)}）=-0.0241

δG=−∂E∂G(in)=−（∂E∂M(in)⋅∂M(in)∂G(out)⋅∂G(out)∂G(in)+∂E∂N(in)⋅∂N(in)∂G(out)⋅∂G(out)∂G(in)）=−0.0122 \delta_G = -\frac{\partial E}{\partial G(in)}=-（\frac{\partial E}{\partial M(in)}\cdot\frac{\partial M(in)}{\partial G(out)}\cdot \frac{\partial G(out)}{\partial G(in)}+ \frac{\partial E}{\partial N(in)}\cdot\frac{\partial N(in)}{\partial G(out)}\cdot \frac{\partial G(out)}{\partial G(in)}）=-0.0122

δH=−∂E∂H(in)=−（∂E∂M(in)⋅∂M(in)∂H(out)⋅∂H(out)∂H(in) +∂E∂N(in)⋅∂N(in)∂H(out)⋅∂H(out)∂H(in)）=−0.0343 \delta_H = -\frac{\partial E}{\partial H(in)}=-（\frac{\partial E}{\partial M(in)}\cdot\frac{\partial M(in)}{\partial H(out)}\cdot \frac{\partial H(out)}{\partial H(in)}\ + \frac{\partial E}{\partial N(in)}\cdot\frac{\partial N(in)}{\partial H(out)}\cdot \frac{\partial H(out)}{\partial H(in)}）=-0.0343

第一层隐层神经元的 error：
δC=−∂E∂C(in)=−(∂E∂F(in)⋅∂F(in)∂C(out)⋅∂C(out)∂C(in)+∂E∂G(in)⋅∂G(in)∂C(out)⋅∂C(out)∂C(in)+∂E∂H(in)⋅∂H(in)∂C(out)⋅∂C(out)∂C(in))=−0.0080 \delta_C=-\frac{\partial E}{\partial C(in)}=-(\frac{\partial E}{\partial F(in)}\cdot\frac{\partial F(in)}{\partial C(out)}\cdot \frac{\partial C(out)}{\partial C(in)} + \frac{\partial E}{\partial G(in)}\cdot\frac{\partial G(in)}{\partial C(out)}\cdot \frac{\partial C(out)}{\partial C(in)} + \frac{\partial E}{\partial H(in)}\cdot\frac{\partial H(in)}{\partial C(out)}\cdot \frac{\partial C(out)}{\partial C(in)})=-0.0080

δD=−∂E∂D(in)=−(∂E∂F(in)⋅∂F(in)∂D(out)⋅∂D(out)∂D(in)+∂E∂G(in)⋅∂G(in)∂D(out)⋅∂D(out)∂D(in)+∂E∂H(in)⋅∂H(in)∂D(out)⋅∂D(out)∂D(in))=−0.0089 \delta_D=-\frac{\partial E}{\partial D(in)}=-(\frac{\partial E}{\partial F(in)}\cdot\frac{\partial F(in)}{\partial D(out)}\cdot \frac{\partial D(out)}{\partial D(in)} + \frac{\partial E}{\partial G(in)}\cdot\frac{\partial G(in)}{\partial D(out)}\cdot \frac{\partial D(out)}{\partial D(in)} + \frac{\partial E}{\partial H(in)}\cdot\frac{\partial H(in)}{\partial D(out)}\cdot \frac{\partial D(out)}{\partial D(in)})=-0.0089

δE=−∂E∂E(in)=−(∂E∂F(in)⋅∂F(in)∂E(out)⋅∂E(out)∂E(in)+∂E∂G(in)⋅∂G(in)∂E(out)⋅∂E(out)∂E(in)+∂E∂H(in)⋅∂H(in)∂E(out)⋅∂E(out)∂E(in))=−0.0106 \delta_E=-\frac{\partial E}{\partial E(in)}=-(\frac{\partial E}{\partial F(in)}\cdot\frac{\partial F(in)}{\partial E(out)}\cdot \frac{\partial E(out)}{\partial E(in)} + \frac{\partial E}{\partial G(in)}\cdot\frac{\partial G(in)}{\partial E(out)}\cdot \frac{\partial E(out)}{\partial E(in)} + \frac{\partial E}{\partial H(in)}\cdot\frac{\partial H(in)}{\partial E(out)}\cdot \frac{\partial E(out)}{\partial E(in)})=-0.0106

下面计算各神经元之间的权重参数，设权重参数 wij 为节点 i 到节点 j 的权重 下面计算各神经元之间的权重参数，设权重参数\ w_{ij} \ 为节点 \ i \ 到节点 \ j\ 的权重

第二层隐层与输出层之间的权重参数：
ΔwFM=−η∂E∂wFM=−η∂E∂M(in)⋅∂M(in)∂wFM=−0.0713 \Delta w_{FM} = -\eta \frac{\partial E}{\partial w_{FM}}=-\eta \frac{\partial E}{\partial M(in)}\cdot \frac{\partial M(in)}{\partial w_{FM}} = -0.0713

ΔwGM=−η∂E∂wGM=−η∂E∂M(in)⋅∂M(in)∂wGM=−0.0691 \Delta w_{GM} = -\eta \frac{\partial E}{\partial w_{GM}}=-\eta \frac{\partial E}{\partial M(in)}\cdot \frac{\partial M(in)}{\partial w_{GM}} = -0.0691

ΔwHM=−η∂E∂wHM=−η∂E∂M(in)⋅∂M(in)∂wHM=−0.0669 \Delta w_{HM} = -\eta \frac{\partial E}{\partial w_{HM}}=-\eta \frac{\partial E}{\partial M(in)}\cdot \frac{\partial M(in)}{\partial w_{HM}} = -0.0669

ΔwFN=−η∂E∂wFN=−η∂E∂N(in)⋅∂N(in)∂wFN=−0.0841 \Delta w_{FN} = -\eta \frac{\partial E}{\partial w_{FN}}=-\eta \frac{\partial E}{\partial N(in)}\cdot \frac{\partial N(in)}{\partial w_{FN}} =-0.0841

ΔwGN=−η∂E∂wGN=−η∂E∂N(in)⋅∂N(in)∂wGN=−0.0815 \Delta w_{GN} = -\eta \frac{\partial E}{\partial w_{GN}}=-\eta \frac{\partial E}{\partial N(in)}\cdot \frac{\partial N(in)}{\partial w_{GN}} =-0.0815

ΔwHN=−η∂E∂wHN=−η∂E∂N(in)⋅∂N(in)∂wHN=−0.0788 \Delta w_{HN} = -\eta \frac{\partial E}{\partial w_{HN}}=-\eta \frac{\partial E}{\partial N(in)}\cdot \frac{\partial N(in)}{\partial w_{HN}} =-0.0788

第一层隐层与第二层隐层之间的权重参数：
ΔwCF=−η∂E∂wCF=−η∂E∂F(in)⋅∂F(in)∂wCF=−0.0118 \Delta w_{CF} = -\eta \frac{\partial E}{\partial w_{CF}}=-\eta \frac{\partial E}{\partial F(in)}\cdot \frac{\partial F(in)}{\partial w_{CF}} =-0.0118

ΔwDF=−η∂E∂wDF=−η∂E∂F(in)⋅∂F(in)∂wDF=−0.0118 \Delta w_{DF} = -\eta \frac{\partial E}{\partial w_{DF}}=-\eta \frac{\partial E}{\partial F(in)}\cdot \frac{\partial F(in)}{\partial w_{DF}} =-0.0118

ΔwEF=−η∂E∂wEF=−η∂E∂F(in)⋅∂F(in)∂wEF=−0.0116 \Delta w_{EF} = -\eta \frac{\partial E}{\partial w_{EF}}=-\eta \frac{\partial E}{\partial F(in)}\cdot \frac{\partial F(in)}{\partial w_{EF}} =-0.0116

ΔwCG=−η∂E∂wCG=−η∂E∂G(in)⋅∂G(in)∂wCG=−0.0060 \Delta w_{CG} = -\eta \frac{\partial E}{\partial w_{CG}}=-\eta \frac{\partial E}{\partial G(in)}\cdot \frac{\partial G(in)}{\partial w_{CG}} =-0.0060

ΔwDG=−η∂E∂wDG=−η∂E∂G(in)⋅∂G(in)∂wDG=−0.0060 \Delta w_{DG} = -\eta \frac{\partial E}{\partial w_{DG}}=-\eta \frac{\partial E}{\partial G(in)}\cdot \frac{\partial G(in)}{\partial w_{DG}} =-0.0060

ΔwEG=−η∂E∂wEG=−η∂E∂G(in)⋅∂G(in)∂wEG=−0.0059 \Delta w_{EG} = -\eta \frac{\partial E}{\partial w_{EG}}=-\eta \frac{\partial E}{\partial G(in)}\cdot \frac{\partial G(in)}{\partial w_{EG}} =-0.0059

ΔwCH=−η∂E∂wCH=−η∂E∂H(in)⋅∂H(in)∂wCH=−0.0168 \Delta w_{CH} = -\eta \frac{\partial E}{\partial w_{CH}}=-\eta \frac{\partial E}{\partial H(in)}\cdot \frac{\partial H(in)}{\partial w_{CH}} =-0.0168

ΔwDH=−η∂E∂wDH=−η∂E∂H(in)⋅∂H(in)∂wDH=−0.0168 \Delta w_{DH} = -\eta \frac{\partial E}{\partial w_{DH}}=-\eta \frac{\partial E}{\partial H(in)}\cdot \frac{\partial H(in)}{\partial w_{DH}} =-0.0168

ΔwEH=−η∂E∂wEH=−η∂E∂H(in)⋅∂H(in)∂wEH=−0.0165 \Delta w_{EH} = -\eta \frac{\partial E}{\partial w_{EH}}=-\eta \frac{\partial E}{\partial H(in)}\cdot \frac{\partial H(in)}{\partial w_{EH}} =-0.0165

输入层与第一层隐层之间的权重参数：
ΔwAC=−η∂E∂wAC=−η∂E∂C(in)⋅∂C(in)∂wAC=−0.0008 \Delta w_{AC} = -\eta \frac{\partial E}{\partial w_{AC}}=-\eta \frac{\partial E}{\partial C(in)}\cdot \frac{\partial C(in)}{\partial w_{AC}} =-0.0008

ΔwBC=−η∂E∂wBC=−η∂E∂C(in)⋅∂C(in)∂wBC=−0.0015 \Delta w_{BC} = -\eta \frac{\partial E}{\partial w_{BC}}=-\eta \frac{\partial E}{\partial C(in)}\cdot \frac{\partial C(in)}{\partial w_{BC}} =-0.0015

ΔwAD=−η∂E∂wAD=−η∂E∂D(in)⋅∂D(in)∂wAD=−0.0009 \Delta w_{AD} = -\eta \frac{\partial E}{\partial w_{AD}}=-\eta \frac{\partial E}{\partial D(in)}\cdot \frac{\partial D(in)}{\partial w_{AD}} =-0.0009

ΔwBD=−η∂E∂wBD=−η∂E∂D(in)⋅∂D(in)∂wBD=−0.0017 \Delta w_{BD} = -\eta \frac{\partial E}{\partial w_{BD}}=-\eta \frac{\partial E}{\partial D(in)}\cdot \frac{\partial D(in)}{\partial w_{BD}} =-0.0017

ΔwAE=−η∂E∂wAE=−η∂E∂E(in)⋅∂E(in)∂wAE=−0.0011 \Delta w_{AE} = -\eta \frac{\partial E}{\partial w_{AE}}=-\eta \frac{\partial E}{\partial E(in)}\cdot \frac{\partial E(in)}{\partial w_{AE}} =-0.0011

ΔwBE=−η∂E∂wBE=−η∂E∂E(in)⋅∂E(in)∂wBE=−0.0020 \Delta w_{BE} = -\eta \frac{\partial E}{\partial w_{BE}}=-\eta \frac{\partial E}{\partial E(in)}\cdot \frac{\partial E(in)}{\partial w_{BE}} =-0.0020

权重更新函数：wij=wij+Δwij 下面对权重参数进行更新，重新得到前馈神经网络的参数图 权重更新函数：w_{ij} = w_{ij}+\Delta w_{ij} \ 下面对权重参数进行更新，重新得到前馈神经网络的参数图
代入具体值求得下面结果：
wAC=0.1−0.0008=0.0992     wAD=0.4−0.0009=0.3991     wAE=0.3−0.0011=0.2989wBC=0.8−0.0015=0.7985     wBD=0.6−0.0017=0.5983     wBE=0.5−0.0020=0.4980wCF=0.9−0.0118=0.8882     wCG=0.6−0.0060=0.5940     wCH=0.1−0.0168=0.0832wDF=0.2−0.0118=0.1882     wDG=0.3−0.0060=0.2940     wDH=0.8−0.0168=0.7832wEF=0.7−0.0116=0.6884     wEG=0.7−0.0059=0.6941     wEH=0.5−0.0165=0.4835wFM=0.4−0.0713=0.3287     wFN=0.6−0.0841=0.5159     wGM=0.2−0.0815=0.2309wGN=0.2−0.0815=0.1185     wHM=0.5−0.0669=0.4331     wHN=0.8−0.0788=0.7212 w_{AC}=0.1-0.0008=0.0992 \ \ \ \ \ w_{AD}=0.4-0.0009=0.3991 \ \ \ \ \ w_{AE}=0.3-0.0011=0.2989\\ w_{BC}=0.8-0.0015=0.7985 \ \ \ \ \ w_{BD}=0.6-0.0017=0.5983 \ \ \ \ \ w_{BE}=0.5-0.0020=0.4980\\ w_{CF}=0.9-0.0118=0.8882 \ \ \ \ \ w_{CG}=0.6-0.0060=0.5940 \ \ \ \ \ w_{CH}=0.1-0.0168=0.0832\\ w_{DF}=0.2-0.0118=0.1882 \ \ \ \ \ w_{DG}=0.3-0.0060=0.2940 \ \ \ \ \ w_{DH}=0.8-0.0168=0.7832\\ w_{EF}=0.7-0.0116=0.6884 \ \ \ \ \ w_{EG}=0.7-0.0059=0.6941 \ \ \ \ \ w_{EH}=0.5-0.0165=0.4835\\ w_{FM}=0.4-0.0713=0.3287 \ \ \ \ \ w_{FN}=0.6-0.0841=0.5159 \ \ \ \ \ w_{GM}=0.2-0.0815=0.2309\\ w_{GN}=0.2-0.0815=0.1185 \ \ \ \ \ w_{HM}=0.5-0.0669=0.4331 \ \ \ \ \ w_{HN}=0.8-0.0788=0.7212\\

​

​

​ 下面对权重参数进行更新，重新得到前馈神经网络的参数图

误差逆传播计算

相关推荐