The activation functions used by the Neuron Class are given below. Let the output be represented by:

O_{i} = g (∑_{j} w_{ji}O_{j}) = g (h_{i})

Where O_{i} is the output of Neuron i, g(x) is the activation function, w_{ji} is the value of the weight connecting the output of Neuron j to the input of Neuron i and h_{i} is the net input to Neuron i given by the summation in the above equation.

The activation functions, g(x), are given by:

• Binary (T = 0)

O_{i} = \{ \begin{matrix}1.0, & if h_{i} > 0.0 \\ 0.0, & if h_{i} < 0.0 \end{matrix}

• Binary (T > 0)

O_{i} = \{ \begin{matrix}1.0, & if rnd ≤ (1 + e^{−2h_{i}/T} )^{−1} \\ 0.0, & otherwise \end{matrix}

• Sigmoid

O_{i} = \frac{1}{1 + e^{−h_{i}}}

• Sign (T = 0)

O_{i} = sgn (h_{i})

• Sign (T > 0)

O_{i} = \{ \begin{matrix}1.0, & if rnd ≤ (1 + e^{−2h_{i}/T} )^{−1} \\ −1.0, & otherwise \end{matrix}

• Tanh

O_{i} = \{ \begin{matrix}tanh (h_{i}) , & if T = 0 \\ tanh (h_{i}/T) , & if T > 0 \end{matrix}

NOTE: rnd is a random number between 0 and 1.