Table 1 Label, definition, and corresponding derivative, for each activation function
From: Modified state activation functions of deep learning-based SC-FDMA channel equalization system
No | Label | Activation function | Derivative function |
|---|---|---|---|
1 | tanh | \(f\left(x\right)=\frac{{e}^{x}-{e}^{-x} }{{e}^{x}+ {e}^{-x}}\) | \({f}^{\mathrm{^{\prime}}\left(x\right)}=1-{\mathrm{tan}h\left(x\right)}^{2}\) |
2 | Gaussian | \(f\left(x\right)={e}^{-{x}^{2}}\) | \({f}^{\mathrm{^{\prime}}\left(x\right)}=-2x{e}^{-{x}^{2}}\) |
3 | GELU | \(f\left(x\right)=0.5x(1+\mathrm{tan}h\left(\sqrt{2/\pi }\left(x+0.447{x}^{3}\right)\right))\) | \({f}^{\mathrm{^{\prime}}}\left(x\right)=0.5\mathrm{tan}h\left(0.0356{x}^{3}+0.797x\right)+\left(0.0535{x}^{3}+0.398x\right){\mathrm{sec}h}^{2}\left(0.0356{x}^{3}+0.797x\right)+0.5\) |
4 | Cloglogm | \(f\left(x\right)={1-2e}^{-0.7{e}^{x}}+0.5\) | \(f\mathrm{^{\prime}}\left(x\right)=7{e}^{x-0.7{e}^{x}}/5\) |
5 | Modified Elliott | \(f\left(x\right)=\frac{x}{\sqrt{1+{x}^{2}}}+0.5\) | \(f\mathrm{^{\prime}}\left(x\right)=\frac{1}{{({x}^{2}+1)}^{3/2}}\) |
6 | Elliott | \(f\left(x\right)=\frac{0.5x}{1+\left|x\right|}+0.5\) | \(f\mathrm{^{\prime}}\left(x\right)=\frac{0.5}{{(1+\left|x\right|)}^{2}}\) |
7 | Bi-tanh1 | \(f\left(x\right)=\frac{1}{2}\left[\mathrm{tan}h\left(\frac{x}{2}\right)+\mathrm{tan}h\left(\frac{x+1}{2}\right)\right]+0.5\) | \({f}^{\mathrm{^{\prime}}}\left(x\right)=\frac{{\mathrm{sec}h}^{2}\left(\frac{x+1}{2}\right)+{\mathrm{sec}h}^{2}\left(\frac{x}{2}\right)}{4}\) |
8 | Bi-tanh2 | \(f\left(x\right)=\frac{1}{2}\left[\mathrm{tan}h\left(\frac{x-1}{2}\right)+\mathrm{tan}h(\frac{x+1}{2})\right]+0.5\) | \({f}^{\mathrm{^{\prime}}}\left(x\right)=\frac{{\mathrm{sec}h}^{2}\left(\frac{x+1}{2}\right)+{\mathrm{sec}h}^{2}\left(\frac{x-1}{2}\right)}{4}\) |
9 | Rootsig | \(f\left(x\right)=\frac{x}{1+\sqrt{1+{x}^{2}}}+0.5\) | \(f\mathrm{^{\prime}}\left(x\right)=\frac{1}{{\sqrt{{x}^{2}+1}+x}^{2}+1}\) |
10 | Softsign | \(f\left(x\right)=\frac{x}{1+\left|x\right|}+0.5\) | \({f}^{\mathrm{^{\prime}}}\left(x\right)=\frac{1}{{(1+\left|x\right|)}^{2}}\) |
11 | Wave | \(f\left(x\right)=(1-{x}^{2}){e}^{-{x}^{2}}\) | \({f}^{\mathrm{^{\prime}}}\left(x\right)=2x({x}^{2}-2){e}^{-{x}^{2}}\) |
12 | Aranda | \(f\left(x\right)=1-{(1+2{e}^{x})}^{-1/2}\) | \({f}^{\mathrm{^{\prime}}}\left(x\right)={e}^{x}{(2{e}^{x}+1)}^{-3/2}\) |