RNN（二）前向和BPTT

标签（空格分隔）： RNN BPTT

basic definition

To simply notation, the RNN here only contains one input layer, one hidden layer and one putput layer. Notations are listed below:

neural layer	node	index	number
input layer	x(t)	i	N
previous hidden layer	s(t)	h	M
hidden layer	s(t-1)	j	M
output layer	y(t)	k	O
input->hidden	V(t)	i,j	N->M
previous hidden->hidden	U(t)	h,j	M->M
hidden->output	W(t)	j,k	M->O

Besides, P is the total number of available training samples which are indexed by l

RNN（二）前向和BPTT
1. input->hidden

n e t j (t) = \sum i N x i (t) v j i + \sum h M s h (t - 1) u j h + θ j netj(t)=\sumiNxi(t)vji+\sumhMsh(t-1)ujh+θj

error for hidden nodes
RNN（二）前向和BPTT

δ l j = - (\sum k O \partial C \partial y l k \partial y l k \partial n e t l k \partial n e t l k \partial s l j) \partial s l j \partial n e t l j = \sum k O δ l k w k j

标签（空格分隔）： RNN BPTT

To simply notation, the RNN here only contains one input layer, one hidden layer and one putput layer. Notations are listed below:

neural layer	node	index	number
input layer	x(t)	i	N
previous hidden layer	s(t)	h	M
hidden layer	s(t-1)	j	M
output layer	y(t)	k	O
input->hidden	V(t)	i,j	N->M
previous hidden->hidden	U(t)	h,j	M->M
hidden->output	W(t)	j,k	M->O

Besides, P is the total number of available training samples which are indexed by l

RNN（二）前向和BPTT
1. input->hidden

n e t j (t) = \sum i N x i (t) v j i + \sum h M s h (t - 1) u j h + θ j netj(t)=\sumiNxi(t)vji+\sumhMsh(t-1)ujh+θj

error for hidden nodes
RNN（二）前向和BPTT

δ l j = - (\sum k O \partial C \partial y l k \partial y l k \partial n e t l k \partial n e t l k \partial s l j) \partial s l j \partial n e t l j = \sum k O δ l k w k j