The output, y, of a hyperbolic tangent activation function (tanh) as a function of its total input, x, is given as follows:
The tanh activation function outputs values in the range [-1, 1], as you can see in the following graph:
Figure 1.7: Tanh activation function
One thing to note is that both the sigmoid and the tanh activation functions are linear within a small range of the input, beyond which the output saturates. In the saturation zone, the gradients of the activation functions (with respect ...
