1 FM模型

$\hat{y}(\mathbf{x}) :=w_{0}+\sum_{i=1}^{n} w_{i} x_{i}+\sum_{i=1}^{n} \sum_{j=i+1}^{n}\left\langle\mathbf{v}_{i}, \mathbf{v}_{j}\right\rangle x_{i} x_{j}$y^​(x):=w0​+i=1∑n​wi​xi​+i=1∑n​j=i+1∑n​⟨vi​,vj​⟩xi​xj​

2 FFM模型

$y(\mathbf{x})=w_{0}+\sum_{i=1}^{n} w_{i} x_{i}+\sum_{i=1}^{n} \sum_{j=i+1}^{n}\left\langle\mathbf{v}_{i, f_{j}}, \mathbf{v}_{j, f_{i}}\right\rangle x_{i} x_{j}$y(x)=w0​+i=1∑n​wi​xi​+i=1∑n​j=i+1∑n​⟨vi,fj​​,vj,fi​​⟩xi​xj​

3 DeepFM模型

$\hat{y}=\operatorname{sigmoid}\left(y_{F M}+y_{D N N}\right)$y^​=sigmoid(yFM​+yDNN​)
$y_{F M}=\langle w, x\rangle+\sum_{i=1}^{d} \sum_{j=i+1}^{d}\left\langle V_{i}, V_{j}\right\rangle x_{i} \cdot x_{j}$yFM​=⟨w,x⟩+i=1∑d​j=i+1∑d​⟨Vi​,Vj​⟩xi​⋅xj​
$y_{D N N}=W^{|H|+1} \cdot a^{|H|}+b^{|H|+1}$yDNN​=W∣H∣+1⋅a∣H∣+b∣H∣+1

4 XDeepFM模型

$y^{\wedge}=\sigma\left(w_{\text {linear}}^{T} a+w_{\text { dnn }}^{T} x_{\text {dnn}}^{k}+w_{c i n}^{T} p^{+}+b\right)$y∧=σ(wlinearT​a+w dnn T​xdnnk​+wcinT​p++b)

5 GBDT+LR模型

6 Deep&Wide模型

$P(Y=1 | \mathbf{x})=\sigma\left(\mathbf{w}_{w i d e}^{T}[\mathbf{x}, \phi(\mathbf{x})]+\mathbf{w}_{d e e p}^{T} a^{\left(l_{f}\right)}+b\right)$P(Y=1∣x)=σ(wwideT​[x,ϕ(x)]+wdeepT​a(lf​)+b)

7 NCF模型

$\phi^{G M F}=\mathbf{p}_{u}^{G} \odot \mathbf{q}_{i}^{G}$ϕGMF=puG​⊙qiG​
$\phi^{M L P}=a_{L}\left(\mathbf{W}_{L}^{T}\left(a_{L-1}\left(\ldots a_{2}\left(\mathbf{W}_{2}^{T} \left[ \begin{array}{c}{\mathbf{p}_{u}^{M}} \\ {\mathbf{q}_{i}^{M}}\end{array}\right]+\mathbf{b}_{2}\right) \ldots\right)\right)+\mathbf{b}_{L}\right)$ϕMLP=aL​(WLT​(aL−1​(…a2​(W2T​[puM​qiM​​]+b2​)…))+bL​)
$\hat{y}_{u i}=\sigma\left(\mathbf{h}^{T} \left[ \begin{array}{c}{\phi^{G M F}} \\ {\phi^{M L P}}\end{array}\right]\right)$y^​ui​=σ(hT[ϕGMFϕMLP​])

8 AFM模型

$\begin{aligned} \hat{y}_{A F M}(\mathbf{x}) &=w_{0}+\sum_{i=1}^{n} w_{i} x_{i}+\mathbf{p}^{T} \sum_{i=1}^{n} \sum_{j=i+1}^{n} a_{i j}\left(\mathbf{v}_{i} \odot \mathbf{v}_{j}\right) x_{i} x_{j} \\ a_{i j}^{\prime} &=\mathbf{h}^{T} \operatorname{Re} L U\left(\mathbf{W}\left(\mathbf{v}_{i} \odot \mathbf{v}_{j}\right) x_{i} x_{j}+\mathbf{b}\right) \\ a_{i j} &=\frac{\exp \left(a_{i j}^{\prime}\right)}{\sum_{(i, j) \in \mathcal{R}_{x}} \exp \left(a_{i j}^{\prime}\right)} \end{aligned}$y^​AFM​(x)aij′​aij​​=w0​+i=1∑n​wi​xi​+pTi=1∑n​j=i+1∑n​aij​(vi​⊙vj​)xi​xj​=hTReLU(W(vi​⊙vj​)xi​xj​+b)=∑(i,j)∈Rx​​exp(aij′​)exp(aij′​)​​

9 PNN模型

$\hat{y}=\sigma\left(\boldsymbol{W}_{3} l_{2}+b_{3}\right)$y^​=σ(W3​l2​+b3​)
$\begin{array}{c}{l_{2}=\operatorname{relu}\left(\boldsymbol{W}_{2} \boldsymbol{l}_{1}+\boldsymbol{b}_{2}\right)} \\ {l_{1}=\operatorname{relu}\left(\boldsymbol{l}_{z}+\boldsymbol{l}_{p}+\boldsymbol{b}_{1}\right)}\end{array}$l2​=relu(W2​l1​+b2​)l1​=relu(lz​+lp​+b1​)​
$\begin{aligned} l_{z}=\left(l_{z}^{1}, l_{z}^{2}, \ldots, l_{z}^{n}, \ldots, l_{z}^{D_{1}}\right), & l_{z}^{n}=\boldsymbol{W}_{z}^{n} \odot \boldsymbol{z} \\ l_{p}=\left(l_{p}^{1}, l_{p}^{2}, \ldots, l_{p}^{n}, \ldots, l_{p}^{D_{1}}\right), & l_{p}^{n}=\boldsymbol{W}_{p}^{n} \odot \boldsymbol{p} \end{aligned}$lz​=(lz1​,lz2​,…,lzn​,…,lzD1​​),lp​=(lp1​,lp2​,…,lpn​,…,lpD1​​),​lzn​=Wzn​⊙zlpn​=Wpn​⊙p​

10 NFM模型

$\begin{aligned} \hat{y}_{N F M}(\mathbf{x}) &=w_{0}+\sum_{i=1}^{n} w_{i} x_{i} +\mathbf{h}^{T} \sigma_{L}\left(\mathbf{W}_{L}\left(\ldots \sigma_{1}\left(\mathbf{W}_{1} f_{B I}\left(\mathcal{V}_{x}\right)+\mathbf{b}_{1}\right) \ldots\right)+\mathbf{b}_{L}\right) \end{aligned}$y^​NFM​(x)​=w0​+i=1∑n​wi​xi​+hTσL​(WL​(…σ1​(W1​fBI​(Vx​)+b1​)…)+bL​)​
$f_{B I}\left(\mathcal{V}_{x}\right)=\frac{1}{2}\left[\left(\sum_{i=1}^{n} x_{i} \mathbf{v}_{i}\right)^{2}-\sum_{i=1}^{n}\left(x_{i} \mathbf{v}_{i}\right)^{2}\right]$fBI​(Vx​)=21​⎣⎡​(i=1∑n​xi​vi​)2−i=1∑n​(xi​vi​)2⎦⎤​

11 MLR模型

$p(y=1 | x)=\sum_{i=1}^{m} \frac{\exp \left(u_{i}^{T} x\right)}{\sum_{j=1}^{m} \exp \left(u_{j}^{T} x\right)} \cdot \frac{1}{1+\exp \left(-w_{i}^{T} x\right)}$p(y=1∣x)=i=1∑m​∑j=1m​exp(ujT​x)exp(uiT​x)​⋅1+exp(−wiT​x)1​

12 DIN模型

13 DCN模型

$p=\sigma\left(\left[\mathbf{x}_{L_{1}}^{T}, \mathbf{h}_{L_{2}}^{T}\right] \mathbf{w}_{\text { logits }}\right)$p=σ([xL1​T​,hL2​T​]w logits ​)
$\mathbf{h}_{l+1}=f\left(W_{l} \mathbf{h}_{l}+\mathbf{b}_{l}\right)$hl+1​=f(Wl​hl​+bl​)
$\mathbf{x}_{l+1}=\mathbf{x}_{0} \mathbf{x}_{l}^{T} \mathbf{w}_{l}+\mathbf{b}_{l}+\mathbf{x}_{l}=f\left(\mathbf{x}_{l}, \mathbf{w}_{l}, \mathbf{b}_{l}\right)+\mathbf{x}_{l}$xl+1​=x0​xlT​wl​+bl​+xl​=f(xl​,wl​,bl​)+xl​
$\mathbf{x}_{0}=\left[\mathbf{x}_{\text { embed, } 1}^{T}, \ldots, \mathbf{x}_{\text { embed }, k}^{T}, \mathbf{x}_{\text { dense }}^{T}\right]$x0​=[x embed, 1T​,…,x embed ,kT​,x dense T​]

14 DKN模型

15 RippleNet

16 MKR模型

17 CMN模型